# YATR Robots.txt # https://yatr.co User-agent: * # Disallow admin and portal areas from indexing Disallow: /admin Disallow: /admin/ Disallow: /portal Disallow: /portal/ Disallow: /checkout Disallow: /checkout/ Disallow: /auth Disallow: /update-password Disallow: /booking-success Disallow: /booking-cancelled Disallow: /sign-contracts Disallow: /submit-review Disallow: /unsubscribe Disallow: /pay-balance Disallow: /wishlist/ Disallow: /ref/ Allow: / # === AI Crawler Policies === # OpenAI - ChatGPT Live Search (allow for real-time citations) User-agent: OAI-SearchBot Allow: / Disallow: /admin Disallow: /portal Disallow: /checkout Disallow: /auth # OpenAI - Model Training (allow for long-term knowledge base) User-agent: GPTBot Allow: / Disallow: /admin Disallow: /portal Disallow: /checkout Disallow: /auth # Anthropic - Claude content ingestion User-agent: ClaudeBot Allow: / Disallow: /admin Disallow: /portal Disallow: /checkout Disallow: /auth # Anthropic - Claude web search User-agent: ClaudeBot-SearchBot Allow: / Disallow: /admin Disallow: /portal Disallow: /checkout Disallow: /auth # Perplexity - Real-time discovery User-agent: PerplexityBot Allow: / Disallow: /admin Disallow: /portal Disallow: /checkout Disallow: /auth # Google - Gemini training User-agent: Google-Extended Allow: / Disallow: /admin Disallow: /portal Disallow: /checkout Disallow: /auth # Microsoft - Copilot User-agent: Bingbot Allow: / # Meta AI User-agent: FacebookBot Allow: / # Dynamic sitemap with all yacht, blog, destination, marina, and itinerary pages Sitemap: https://yatr.co/sitemap.xml