🇮🇳 Bharti Airtel
airtel.in · National indices · rank #57 · Telecommunications · live robots.txt ↗
AI crawler access (latest snapshot, 13h ago)
⛔blocked
restricted
✅allowed
faded = inherited from the * wildcard group
✅GPTBot
✅ChatGPT-User
✅OAI-SearchBot
✅ClaudeBot
✅Claude-User
✅Claude-SearchBot
✅anthropic-ai
✅Claude-Web
✅CCBot
✅Google-Extended
✅Applebot-Extended
✅PerplexityBot
✅Perplexity-User
✅Bytespider
✅Amazonbot
✅FacebookBot
✅meta-externalagent
✅meta-externalfetcher
✅cohere-ai
✅AI2Bot
✅Diffbot
✅omgili
✅YouBot
✅DuckAssistBot
✅MistralAI-User
✅PanguBot
✅Timpibot
Current robots.txt 1931 bytes · sha256 7bc91728a52a · raw
User-Agent: * Disallow:/airtellive/* Disallow:/growth-cms-pages/* Disallow:/app* Disallow:/mobile* Disallow:/forme* Disallow:/aes* Disallow:/m/* Disallow:/hfz-hellotunes/* Disallow:/Y/* Disallow:/4g_lte/* Disallow:/ga* Disallow:/personal/* Disallow:/wps/* Disallow:/prepaid-recharge/complete Disallow:/postpaid-bill-pay/complete Disallow:/index-all/product-listing.html Allow:/bank/* Allow: /digital-store/ Disallow:/blog/author/*/*/* Disallow:*&* Disallow:*/digital-store/*/*utm* Disallow:/business/b2b/airtel-iq/api-docs/** # ============================ # AI / LLM Crawlers (Allowed) # ============================ User-agent: GPTBot Allow: / User-agent: PerplexityBot Allow: / User-agent: bingbot Allow: / User-agent: BingPreview Allow: / User-agent: bingGPT Allow: / User-agent: Googlebot Allow: / User-agent: Google-Extended Allow: / User-agent: Bard Allow: / User-agent: Gemini Allow: / User-agent: ClaudeBot Allow: / # ============================ # Restricted Crawlers # ============================ User-Agent: Slurp Disallow: / # ============================ # Sitemap Directives # ============================ Sitemap: https://www.airtel.in/sitemap.xml Sitemap: https://www.airtel.in/infinity-sitemap.xml Sitemap: https://www.airtel.in/isp-sitemap.xml Sitemap: https://www.airtel.in/broadband-sitemap.xml Sitemap: https://www.airtel.in/cms-sitemap.xml Sitemap: https://www.airtel.in/dth-sitemap.xml Sitemap: https://www.airtel.in/bank/sitemap.xml Sitemap: https://www.airtel.in/growth-sitemaps/growth-sitemap6.xml Sitemap: https://www.airtel.in/growth-sitemaps/growth-sitemap5.xml Sitemap: https://www.airtel.in/growth-sitemaps/growth-sitemap4.xml Sitemap: https://www.airtel.in/growth-sitemaps/growth-sitemap3.xml Sitemap: https://www.airtel.in/growth-sitemaps/growth-sitemap2.xml Sitemap: https://www.airtel.in/growth-sitemaps/growth-sitemap1.xml Sitemap: https://www.airtel.in/business-sitemap.xml
Change history
-
initial snapshot
- First snapshot of robots.txt archived