NeuralCrawl

Simon & Schuster / robots.txt

← back to simonandschuster.com · change detected 1mo ago (2026-10-03T02:09:18Z) · modified +8 −1

What changed

Diff old (2026-06-20T01:10:31Z) → new (2026-10-03T02:09:18Z)

@@ -1,4 +1,11 @@
11  User-Agent: *
2- Disallow: /external_assets
2+ Disallow: /search
3+ Allow: /search/books/Category-Biography-Autobiography/Available-For-Sale-Now/_/N-hr0Zpgz/Ne-pgx$
4+ Allow: /search/books/Category-Games-Crosswords/Available-For-Sale-Now/_/N-g6aZpgz/Ne-ffk$
5+ Allow: /search/books/Category-Fiction/Available-For-Sale-Now/_/N-g1hZpgz/Ne-pgx$
6+ Allow: /search/books/Category-History/Available-For-Sale-Now/_/N-ha4Zpgz/Ne-ffk$
7+ Allow: /search/books/Category-Fiction-Mystery-Detective/Available-For-Sale-Now/_/N-g32Zpgz/Ne-pgx$
8+ Allow: /search/books/Category-Political-Science/Available-For-Sale-Now/_/N-hnqZpgz/Ne-ffk$
9+ Allow: /search/books/Available-For-Sale-Now/_/N-pgz/Ne-pgx$
310  Disallow: /about/contact_us
411  Sitemap: https://www.simonandschuster.com/sitemaps/us/sitemap.xml.gz