Facebook / robots.txt
← back to facebook.com · change detected 23d ago (2026-10-15T06:25:59Z) · modified +3 −47
What changed
- 🤖 AIBlocked Google-Extended (Google) entirely
- 🤖 AIRemoved Allow: /ajax/bootloader-endpoint/ for Google-Extended
- 🤖 AIRemoved Allow: /ajax/pagelet/generic.php/PagePostsSectionPagelet for Google-Extended
- 🤖 AIRemoved Allow: /careers/ for Google-Extended
- 🤖 AIRemoved Allow: /safetycheck/ for Google-Extended
- 🤖 AIRemoved Disallow: /*/plugins/* for Google-Extended
- 🤖 AIRemoved Disallow: /?*next= for Google-Extended
- 🤖 AIRemoved Disallow: /a/bz? for Google-Extended
- 🤖 AIRemoved Disallow: /ajax/ for Google-Extended
- 🤖 AIRemoved Disallow: /album.php for Google-Extended
- 🤖 AIRemoved Disallow: /business/*&categories for Google-Extended
- 🤖 AIRemoved Disallow: /business/*?categories for Google-Extended
- 🤖 AIRemoved Disallow: /business/help/search*&query= for Google-Extended
- 🤖 AIRemoved Disallow: /business/help/search*?query= for Google-Extended
- 🤖 AIRemoved Disallow: /business/news*&tags for Google-Extended
- 🤖 AIRemoved Disallow: /business/news*?tags for Google-Extended
- 🤖 AIRemoved Disallow: /business/search*&q= for Google-Extended
- 🤖 AIRemoved Disallow: /business/search*?q= for Google-Extended
- 🤖 AIRemoved Disallow: /checkpoint/ for Google-Extended
- 🤖 AIRemoved Disallow: /contact_importer/ for Google-Extended
- 🤖 AIRemoved Disallow: /dialog/ for Google-Extended
- 🤖 AIRemoved Disallow: /fbml/ajax/dialog/ for Google-Extended
- 🤖 AIRemoved Disallow: /feeds/ for Google-Extended
- 🤖 AIRemoved Disallow: /file_download.php for Google-Extended
- 🤖 AIRemoved Disallow: /job_application/ for Google-Extended
- 🤖 AIRemoved Disallow: /l.php for Google-Extended
- 🤖 AIRemoved Disallow: /login.php*&next= for Google-Extended
- 🤖 AIRemoved Disallow: /login.php/?next= for Google-Extended
- 🤖 AIRemoved Disallow: /login.php?next= for Google-Extended
- 🤖 AIRemoved Disallow: /login/*&next= for Google-Extended
- 🤖 AIRemoved Disallow: /login/?next= for Google-Extended
- 🤖 AIRemoved Disallow: /login/device-based/regular/login/*&next= for Google-Extended
- 🤖 AIRemoved Disallow: /login/device-based/regular/login/?next= for Google-Extended
- 🤖 AIRemoved Disallow: /meta-ai-answers/ for Google-Extended
- 🤖 AIRemoved Disallow: /moments_app/ for Google-Extended
- 🤖 AIRemoved Disallow: /p.php for Google-Extended
- 🤖 AIRemoved Disallow: /photos.php for Google-Extended
- 🤖 AIRemoved Disallow: /plugins/ for Google-Extended
- 🤖 AIRemoved Disallow: /share.php for Google-Extended
- 🤖 AIRemoved Disallow: /share/ for Google-Extended
- 🤖 AIRemoved Disallow: /sharer.php for Google-Extended
- 🤖 AIRemoved Disallow: /sharer/ for Google-Extended
- 🤖 AIRemoved Disallow: /tr/ for Google-Extended
- 🤖 AIRemoved Disallow: /tr? for Google-Extended
- 🤖 AIRemoved Disallow: /ufi/reaction/profile/browser/ for Google-Extended
- 🤖 AIRemoved Disallow: /x/oauth/ for Google-Extended
Diff old (2026-08-26T20:03:19Z) → new (2026-10-15T06:25:59Z)
| @@ -5,6 +5,9 @@ | ||
| 5 | 5 | # All authorized user-agents listed on this page must comply with Meta’s |
| 6 | 6 | # Automated Data Collection Terms available at: |
| 7 | 7 | # https://www.facebook.com/legal/automated_data_collection_terms |
| 8 | + User-agent: Google-Extended | |
| 9 | + Disallow: / | |
| 10 | + | |
| 8 | 11 | User-agent: PetalBot |
| 9 | 12 | Disallow: / |
| 10 | 13 | |
| @@ -1207,53 +1210,6 @@ | ||
| 1207 | 1210 | Allow: /careers/ |
| 1208 | 1211 | Allow: /safetycheck/ |
| 1209 | 1212 | |
| 1210 | - User-agent: Google-Extended | |
| 1211 | - Disallow: /*/plugins/* | |
| 1212 | - Disallow: /?*next= | |
| 1213 | - Disallow: /a/bz? | |
| 1214 | - Disallow: /ajax/ | |
| 1215 | - Disallow: /album.php | |
| 1216 | - Disallow: /business/*&categories | |
| 1217 | - Disallow: /business/*?categories | |
| 1218 | - Disallow: /business/help/search*&query= | |
| 1219 | - Disallow: /business/help/search*?query= | |
| 1220 | - Disallow: /business/news*&tags | |
| 1221 | - Disallow: /business/news*?tags | |
| 1222 | - Disallow: /business/search*&q= | |
| 1223 | - Disallow: /business/search*?q= | |
| 1224 | - Disallow: /checkpoint/ | |
| 1225 | - Disallow: /contact_importer/ | |
| 1226 | - Disallow: /dialog/ | |
| 1227 | - Disallow: /fbml/ajax/dialog/ | |
| 1228 | - Disallow: /feeds/ | |
| 1229 | - Disallow: /file_download.php | |
| 1230 | - Disallow: /job_application/ | |
| 1231 | - Disallow: /l.php | |
| 1232 | - Disallow: /login.php*&next= | |
| 1233 | - Disallow: /login.php/?next= | |
| 1234 | - Disallow: /login.php?next= | |
| 1235 | - Disallow: /login/*&next= | |
| 1236 | - Disallow: /login/?next= | |
| 1237 | - Disallow: /login/device-based/regular/login/*&next= | |
| 1238 | - Disallow: /login/device-based/regular/login/?next= | |
| 1239 | - Disallow: /meta-ai-answers/ | |
| 1240 | - Disallow: /moments_app/ | |
| 1241 | - Disallow: /p.php | |
| 1242 | - Disallow: /photos.php | |
| 1243 | - Disallow: /plugins/ | |
| 1244 | - Disallow: /share.php | |
| 1245 | - Disallow: /share/ | |
| 1246 | - Disallow: /sharer.php | |
| 1247 | - Disallow: /sharer/ | |
| 1248 | - Disallow: /tr/ | |
| 1249 | - Disallow: /tr? | |
| 1250 | - Disallow: /ufi/reaction/profile/browser/ | |
| 1251 | - Disallow: /x/oauth/ | |
| 1252 | - Allow: /ajax/bootloader-endpoint/ | |
| 1253 | - Allow: /ajax/pagelet/generic.php/PagePostsSectionPagelet | |
| 1254 | - Allow: /careers/ | |
| 1255 | - Allow: /safetycheck/ | |
| 1256 | - | |
| 1257 | 1213 | User-agent: GPTBot |
| 1258 | 1214 | Disallow: /*/plugins/* |
| 1259 | 1215 | Disallow: /?*next= |