USA Today / robots.txt
← back to usatoday.com · change detected 3mo ago (2026-08-07T19:48:33Z) · modified +6 −0
What changed
- Added Allow: /story/sponsor-story/real-estate-ausa/2026/04/16/sell-your-home-for-top-dollar-by-beating-back-to-school-deadlines/89482853007/ for ChatGPT Operator
- 🤖 AIAdded Allow: /story/sponsor-story/real-estate-ausa/2026/04/16/sell-your-home-for-top-dollar-by-beating-back-to-school-deadlines/89482853007/ for ChatGPT-User
- 🤖 AIAdded Allow: /story/sponsor-story/real-estate-ausa/2026/04/16/sell-your-home-for-top-dollar-by-beating-back-to-school-deadlines/89482853007/ for Google-Extended
- 🤖 AIAdded Allow: /story/sponsor-story/real-estate-ausa/2026/04/16/sell-your-home-for-top-dollar-by-beating-back-to-school-deadlines/89482853007/ for GPTBot
- 🤖 AIAdded Allow: /story/sponsor-story/real-estate-ausa/2026/04/16/sell-your-home-for-top-dollar-by-beating-back-to-school-deadlines/89482853007/ for OAI-SearchBot
- 🤖 AIAdded Allow: /story/sponsor-story/real-estate-ausa/2026/04/16/sell-your-home-for-top-dollar-by-beating-back-to-school-deadlines/89482853007/ for PerplexityBot
Diff old (2026-07-22T19:49:29Z) → new (2026-08-07T19:48:33Z)
| @@ -36,6 +36,7 @@ | ||
| 36 | 36 | Allow: /story/special/contributor-content/ |
| 37 | 37 | Allow: /money/money-management/ |
| 38 | 38 | Allow: /shopping/ |
| 39 | + Allow: /story/sponsor-story/real-estate-ausa/2026/04/16/sell-your-home-for-top-dollar-by-beating-back-to-school-deadlines/89482853007/ | |
| 39 | 40 | |
| 40 | 41 | User-agent: Googlebot-News |
| 41 | 42 | Disallow: /story/sponsor-story/ |
| @@ -52,6 +53,7 @@ | ||
| 52 | 53 | Allow: /story/special/contributor-content/ |
| 53 | 54 | Allow: /money/money-management/ |
| 54 | 55 | Allow: /shopping/ |
| 56 | + Allow: /story/sponsor-story/real-estate-ausa/2026/04/16/sell-your-home-for-top-dollar-by-beating-back-to-school-deadlines/89482853007/ | |
| 55 | 57 | |
| 56 | 58 | User-agent: CCBot |
| 57 | 59 | Disallow: / |
| @@ -62,6 +64,7 @@ | ||
| 62 | 64 | Allow: /story/special/contributor-content/ |
| 63 | 65 | Allow: /money/money-management/ |
| 64 | 66 | Allow: /shopping/ |
| 67 | + Allow: /story/sponsor-story/real-estate-ausa/2026/04/16/sell-your-home-for-top-dollar-by-beating-back-to-school-deadlines/89482853007/ | |
| 65 | 68 | |
| 66 | 69 | User-agent: DataForSeoBot |
| 67 | 70 | Disallow: / |
| @@ -105,6 +108,7 @@ | ||
| 105 | 108 | Allow: /story/special/contributor-content/ |
| 106 | 109 | Allow: /money/money-management/ |
| 107 | 110 | Allow: /shopping/ |
| 111 | + Allow: /story/sponsor-story/real-estate-ausa/2026/04/16/sell-your-home-for-top-dollar-by-beating-back-to-school-deadlines/89482853007/ | |
| 108 | 112 | |
| 109 | 113 | User-agent: Scrapy |
| 110 | 114 | Disallow: / |
| @@ -127,6 +131,7 @@ | ||
| 127 | 131 | Allow: /story/special/contributor-content/ |
| 128 | 132 | Allow: /money/money-management/ |
| 129 | 133 | Allow: /shopping/ |
| 134 | + Allow: /story/sponsor-story/real-estate-ausa/2026/04/16/sell-your-home-for-top-dollar-by-beating-back-to-school-deadlines/89482853007/ | |
| 130 | 135 | |
| 131 | 136 | User-agent: Bytespider |
| 132 | 137 | Disallow: / |
| @@ -359,6 +364,7 @@ | ||
| 359 | 364 | Allow: /story/special/contributor-content/ |
| 360 | 365 | Allow: /money/money-management/ |
| 361 | 366 | Allow: /shopping/ |
| 367 | + Allow: /story/sponsor-story/real-estate-ausa/2026/04/16/sell-your-home-for-top-dollar-by-beating-back-to-school-deadlines/89482853007/ | |
| 362 | 368 | |
| 363 | 369 | User-agent: ChatGPT-PageFetcher |
| 364 | 370 | Disallow: / |