User-agent: * Disallow: /*?* Disallow: /api/hotel-reviews # Block known/undesired crawlers (UA-based). User-agent: AhrefsBot Disallow: / User-agent: Amazing-SearchBot Disallow: / User-agent: AwarioBot Disallow: / User-agent: BitSightBot Disallow: / User-agent: Bytespider Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: DiscountifyBot Disallow: / User-agent: DomainSearch-Bot Disallow: / User-agent: DotBot Disallow: / User-agent: Go-http-client Disallow: / User-agent: HeadlessChrome Disallow: / User-agent: IbouBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: Mediatoolkitbot Disallow: / User-agent: PetalBot Disallow: / User-agent: PounceBot Disallow: / User-agent: PrimepackBot Disallow: / User-agent: Python Disallow: / User-agent: Requests Disallow: / User-agent: Screaming Frog Disallow: / User-agent: SEO Spider Disallow: / User-agent: SemrushBot Disallow: / User-agent: SiteAuditBot Disallow: / User-agent: crawler Disallow: / User-agent: curl Disallow: / User-agent: io_bot Disallow: / User-agent: wpbot Disallow: / # Sitemap URL is appended dynamically in server/src/routes/robots.js # (per-hostname absolute URL), e.g. Sitemap: https://tripx.no/sitemap.xml Sitemap: https://primaholidays.travel/sitemap.xml