# ============================================================== # robots.txt — Max MPG Scooters & E-Bikes # https://max-mpgscooters.com/ # Last updated: 2026-05-22 # Purpose: SEO optimization, GEO / AI crawler access, block # low-value SEO audit bots, protect admin paths # ============================================================== # -------------------------------------------------------------- # SITEMAP # -------------------------------------------------------------- Sitemap: https://max-mpgscooters.com/sitemap # ============================================================== # MAJOR SEARCH ENGINE CRAWLERS — Full access # ============================================================== User-agent: Googlebot Allow: / User-agent: Googlebot-Image Allow: / User-agent: Bingbot Allow: / User-agent: DuckDuckBot Allow: / User-agent: Slurp Allow: / User-agent: YandexBot Allow: / User-agent: Baiduspider Allow: / # ============================================================== # AI / LLM CRAWLERS — Full access for GEO visibility # ============================================================== # OpenAI / ChatGPT User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: OAI-SearchBot Allow: / # Anthropic / Claude User-agent: ClaudeBot Allow: / User-agent: Claude-Web Allow: / User-agent: anthropic-ai Allow: / # Google AI (Gemini / AI Overviews) User-agent: Google-Extended Allow: / # Meta / Facebook AI User-agent: meta-externalagent Allow: / User-agent: FacebookBot Allow: / # Apple User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / # Perplexity AI User-agent: PerplexityBot Allow: / # You.com User-agent: YouBot Allow: / # Cohere User-agent: cohere-ai Allow: / # Common Crawl (feeds many LLMs) User-agent: CCBot Allow: / # Diffbot (AI content extraction) User-agent: Diffbot Allow: / # ByteDance / TikTok AI User-agent: Bytespider Allow: / # Amazon Alexa / Rufus AI User-agent: Amazonbot Allow: / # ============================================================== # LOW-VALUE SEO AUDIT BOTS — Blocked # (These consume crawl budget without delivering traffic value) # ============================================================== User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: SeznamBot Disallow: / # ============================================================== # ALL OTHER BOTS — Generic crawl delay + admin path protection # ============================================================== User-agent: * Crawl-delay: 10 # --- CMS / Platform Admin & Utility Paths --- Disallow: /admin/ Disallow: /dashboard/ Disallow: /login/ Disallow: /account/ Disallow: /checkout/ Disallow: /cart/ Disallow: /api/ Disallow: /_api/ Disallow: /_partials/ Disallow: /wix-blog-app/ Disallow: /wix-debug/ Disallow: /editor/ # --- Tracking & Campaign Parameter URLs --- # (Prevents duplicate content from UTM/ad tracking params) Disallow: /*?utm_ Disallow: /*?gclid= Disallow: /*?fbclid= Disallow: /*?msclkid= Disallow: /*?ttclid= Disallow: /*?ref=