🇦🇺 University of Sydney

sydney.edu.au · Universities · rank #34 · University · live robots.txt ↗

AI crawler access (latest snapshot, 3h ago)

⛔blocked restricted ✅allowed faded = inherited from the * wildcard group

✅GPTBot

✅ChatGPT-User

✅OAI-SearchBot

✅ClaudeBot

✅Claude-User

✅Claude-SearchBot

✅anthropic-ai

✅Claude-Web

✅CCBot

✅Google-Extended

✅Applebot-Extended

✅PerplexityBot

✅Perplexity-User

✅Bytespider

✅Amazonbot

✅FacebookBot

✅meta-externalagent

✅meta-externalfetcher

✅cohere-ai

✅AI2Bot

✅Diffbot

✅omgili

✅YouBot

✅DuckAssistBot

✅MistralAI-User

✅PanguBot

✅Timpibot

Current robots.txt 2138 bytes · sha256 2934536259fb · raw

User-agent: FunnelBack
Disallow: /education_social_work/bulletin/

User-agent: *
Allow: /
Sitemap:https://www.sydney.edu.au/sitemap.xml
Allow: /myuni-content/
Allow: /medicine-health/schools/sydney-school-of-health-sciences/academic-staff/
Allow: /science/about/our-people/academic-staff/


Disallow: /medicine/fmrc/icpc3dev/
Disallow: /handbooks/archive/2011/
Disallow: /medicine/people/academics-old/profiles/
Disallow: /search/

Disallow: /library/images/
Disallow: /library/scripts/
Disallow: /library/styles/
Disallow: /library/test/
Disallow: /library/templates/
Disallow: /library/stream/
Disallow: /library/screens/
Disallow: /library/cgi-bin/
Disallow: /library/unified-search/
Disallow: /library/contacts/email-campaigns/

Disallow: /styleguide/
Disallow: /agents/

Disallow: /errors/
Disallow: /architecture/about/our-people/academic-staff/staff-profile.html
Disallow: /law/about/our-people/academic-staff/staff-profile.html
Disallow: /music/about/our-people/academic-staff/staff-profile.html
Disallow: /engineering/about/our-people/academic-staff/staff-profile.html
Disallow: /medicine-health/about/our-people/academic-staff/staff-profile.html
Disallow: /medicine-health/schools/faculty-of-health-sciences/academic-staff/staff-profile.html
Disallow: /arts/about/our-people/academic-staff/staff-profile.html

Disallow: /content/dam/corporate/documents/agents-portal/
Disallow: /content/dam/corporate/documents/study/guides/usyd-international-guide-2022.pdf
Disallow: /content/dam/corporate/documents/study/guides/usyd-international-guide.pdf
Disallow: /content/dam/corporate/documents/study/how-to-apply/international-admission-guide.pdf
Disallow: /content/dam/corporate/documents/study/getting-started/welcome-guide_international.pdf
Disallow: /content/dam/corporate/documents/business-school/study/mba/part-time-mba-brochure.pdf
Disallow: /business/study/mba/part-time-mba/request-a-brochure/thank-you.html
Disallow: /study/mba/full-time-mba/request-a-brochure/thank-you.html
Disallow: /business/study/mba/global-executive-mba/request-a-brochure/thank-you.html
Disallow: /content/dam/students/documents/graduations/programs/

Change history

initial 3h ago snapshot
- First snapshot of robots.txt archived