- Added sitemap http://www.bca.co.id/sitemap.xml
- Removed sitemap https://www.bca.co.id/sitemap.xml
The Pulse
Every change to a monitored robots.txt, archived and explained.
1273sites tracked
1206files archived
5439snapshots
4304changes detected
239this week
- Formatting/comment-only change (no semantic difference)
- Formatting/comment-only change (no semantic difference)
- Formatting/comment-only change (no semantic difference)
- Added sitemap https://www.thetradedesk.com/sitemap.xml
- Removed sitemap https://careers.thetradedesk.com/sitemap.xml
- Formatting/comment-only change (no semantic difference)
- Added sitemap http://www.iqvia.com/image-sitemap.xml
- Added sitemap http://www.iqvia.com/page-sitemap.xml
- Removed sitemap http://127.0.0.1/image-sitemap.xml
- Removed sitemap http://127.0.0.1/page-sitemap.xml
- robots.txt is no longer served (HTTP 404)
- Formatting/comment-only change (no semantic difference)
- Added sitemap https://www.bca.co.id/sitemap.xml
- Removed sitemap http://www.bca.co.id/sitemap.xml
- Added sitemap https://www.lilly.com/medical/sitemap-index.xml
- Formatting/comment-only change (no semantic difference)
- Added Disallow: /*/media/oembed for *
- Added Disallow: /README.md for *
- Added Disallow: /composer/Metapackage/README.txt for *
- Added Disallow: /composer/Plugin/ProjectMessage/README.md for *
- Added Disallow: /composer/Plugin/Scaffold/README.md for *
- Added Disallow: /composer/Plugin/VendorHardening/README.txt for *
- … 25 more change(s)
- robots.txt is no longer served (HTTP 404)
- First snapshot of robots.txt archived
- First snapshot of robots.txt archived
- First snapshot of robots.txt archived
- First snapshot of robots.txt archived
- First snapshot of robots.txt archived
- First snapshot of robots.txt archived
- First snapshot of robots.txt archived
- First snapshot of robots.txt archived
- First snapshot of robots.txt archived
- First snapshot of robots.txt archived
- First snapshot of robots.txt archived
- First snapshot of robots.txt archived
- Added Disallow: /gtwy/api/ecom/purchase-history/order-details/* for *
- Added Disallow: /gtwy/api/ecom/purchase-history/order-list* for *
- Added Disallow: /tsc/cart* for *
- Added Disallow: /tsc/checkout* for *
- Added Disallow: /tsc/orderconfirmation* for *
- Added Disallow: /tsc/orders* for *
- Added Disallow: /*?*&searchQuery=* for *
- Added Disallow: /*?searchQuery=* for *
- Added sitemap https://www.sap.com/sitemap-us-int-only-siteimprove.xml
- Added sitemap https://www.sap.com/sitemap_countries_siteimprove.xml
- Added sitemap https://www.sap.com/sitemap_us_int_only_siteimprove.xml
- Formatting/comment-only change (no semantic difference)