Nasdaq 100
The archive β every monitored site, its robots.txt history and AI-crawler stance.
All sites
Top 1000 websites
US 500
Nasdaq 100
AI labs
Cybersecurity
SEO & AI search
Creative rights-holders
Knowledge & UGC
National indices
Banks & fintech
European companies
Social Networks
Governments
Publishers
| # | Company | Domain | Sector | robots.txt | AI bots blocked | Snapshots | Changes | Last change | |
|---|---|---|---|---|---|---|---|---|---|
| 1 | πΊπΈ | Apple |
|
Technology | present | 0 | 1 | 0 | β |
| 2 | πΊπΈ | Microsoft |
|
Technology | present | 0 | 1 | 0 | β |
| 3 | πΊπΈ | Nvidia |
|
Technology | present | 0 | 1 | 0 | β |
| 4 | πΊπΈ | Amazon |
|
Retailing | present | 🤖 23 | 1 | 0 | β |
| 5 | πΊπΈ | Alphabet |
|
Technology | present | 0 | 1 | 0 | β |
| 6 | πΊπΈ | Meta Platforms |
|
Technology | present | 0 | 1 | 0 | β |
| 7 | πΊπΈ | Broadcom |
|
Technology | present | 0 | 1 | 0 | β |
| 8 | πΊπΈ | Tesla |
|
Motor Vehicles | present | 0 | 1 | 0 | β |
| 9 | πΊπΈ | Costco Wholesale |
|
Retailing | present | 0 | 1 | 0 | β |
| 10 | πΊπΈ | Netflix |
|
Media | present | 0 | 1 | 0 | β |
| 11 | πΊπΈ | Advanced Micro Devices |
|
Technology | present | 0 | 1 | 0 | β |
| 12 | πΊπΈ | PepsiCo |
|
Food & Beverage | present | 0 | 1 | 0 | β |
| 13 | πΊπΈ | Adobe |
|
Technology | present | 0 | 1 | 0 | β |
| 14 | πΊπΈ | Linde |
|
Chemicals | present | 0 | 1 | 0 | β |
| 15 | πΊπΈ | Cisco Systems |
|
Technology | present | 0 | 1 | 0 | β |
| 16 | πΊπΈ | Qualcomm |
|
Technology | present | 0 | 1 | 0 | β |
| 17 | πΊπΈ | T-Mobile US |
|
Telecommunications | blocked | 0 | 0 | 0 | β |
| 18 | πΊπΈ | Comcast |
|
Media | present | 0 | 1 | 0 | β |
| 19 | πΊπΈ | Intuit |
|
Technology | present | 0 | 1 | 0 | β |
| 20 | πΊπΈ | Texas Instruments |
|
Technology | present | 0 | 1 | 0 | β |
| 21 | πΊπΈ | Amgen |
|
Health Care | present | 0 | 3 | 2 | 5h ago |
| 22 | πΊπΈ | Honeywell International |
|
Industrials | present | 0 | 1 | 0 | β |
| 23 | πΊπΈ | Intel |
|
Technology | present | 0 | 1 | 0 | β |
| 24 | πΊπΈ | Starbucks |
|
Food Services | present | 0 | 1 | 0 | β |
| 25 | πΊπΈ | Applied Materials |
|
Technology | present | 0 | 1 | 0 | β |
| 26 | πΊπΈ | Booking Holdings |
|
Technology | present | 0 | 1 | 0 | β |
| 27 | πΊπΈ | Mondelez International |
|
Food & Beverage | present | 0 | 1 | 0 | β |
| 28 | πΊπΈ | ADP |
|
Business Services | present | 0 | 1 | 0 | β |
| 29 | πΊπΈ | Gilead Sciences |
|
Health Care | present | 0 | 1 | 0 | β |
| 30 | πΊπΈ | Vertex Pharmaceuticals |
|
Health Care | present | 0 | 1 | 0 | β |
| 31 | πΊπΈ | Lam Research |
|
Technology | present | 0 | 1 | 0 | β |
| 32 | πΊπΈ | Analog Devices |
|
Technology | present | 0 | 1 | 0 | β |
| 33 | πΊπΈ | KLA |
|
Technology | present | 0 | 1 | 0 | β |
| 34 | πΊπΈ | Regeneron Pharmaceuticals |
|
Health Care | present | 0 | 1 | 0 | β |
| 35 | πΊπΈ | Palo Alto Networks |
|
Technology | present | 0 | 1 | 0 | β |
| 36 | πΊπΈ | Snowflake |
|
Technology | present | 0 | 1 | 0 | β |
| 37 | πΊπΈ | Micron Technology |
|
Technology | present | 0 | 1 | 0 | β |
| 38 | πΊπΈ | Intuitive Surgical |
|
Health Care | present | 0 | 1 | 0 | β |
| 39 | πΊπΈ | CrowdStrike |
|
Computer Security | present | 0 | 1 | 0 | β |
| 40 | πΊπΈ | Airbnb |
|
Technology | present | 0 | 1 | 0 | β |
| 41 | πΊπΈ | Cadence Design Systems |
|
Technology | blocked | 0 | 0 | 0 | β |
| 42 | πΊπΈ | Synopsys |
|
Technology | present | 0 | 1 | 0 | β |
| 43 | πΊπΈ | Marriott International |
|
Hotels & Leisure | present | 0 | 1 | 0 | β |
| 44 | πΊπΈ | Workday |
|
Technology | present | 0 | 1 | 0 | β |
| 45 | πΊπΈ | Fortinet |
|
Computer Security | present | 🤖 1 | 1 | 0 | β |
| 46 | πΊπΈ | Paychex |
|
Business Services | blocked | 0 | 0 | 0 | β |
| 47 | πΊπΈ | O'Reilly Automotive |
|
Retailing | present | 0 | 1 | 0 | β |
| 48 | πΊπΈ | Charter Communications |
|
Telecommunications | present | 0 | 1 | 0 | β |
| 49 | πΊπΈ | Keurig Dr Pepper |
|
Food & Beverage | present | 0 | 1 | 0 | β |
| 50 | πΊπΈ | Marvell Technology |
|
Semiconductors | present | 0 | 1 | 0 | β |
| 51 | πΊπΈ | PayPal Holdings |
|
Financials | present | 0 | 1 | 0 | β |
| 52 | πΊπΈ | American Electric Power |
|
Energy | absent | 0 | 0 | 0 | β |
| 53 | πΊπΈ | Exelon |
|
Energy | present | 0 | 1 | 0 | β |
| 54 | πΊπΈ | Copart |
|
Business Services | present | 0 | 1 | 0 | β |
| 55 | πΊπΈ | Ross Stores |
|
Retailing | present | 0 | 1 | 0 | β |
| 56 | πΊπΈ | Cintas |
|
Industrials | present | 0 | 1 | 0 | β |
| 57 | πΊπΈ | Fastenal |
|
Wholesale | present | 0 | 1 | 0 | β |
| 58 | πΊπΈ | Diamondback Energy |
|
Energy | present | 🤖 1 | 1 | 0 | β |
| 59 | πΊπΈ | Datadog |
|
Programming and Developer Software | present | 0 | 1 | 0 | β |
| 60 | πΊπΈ | Autodesk |
|
Programming and Developer Software | present | 0 | 1 | 0 | β |
| 61 | πΊπΈ | Microchip Technology |
|
Semiconductors | present | 0 | 1 | 0 | β |
| 62 | πΊπΈ | IDEXX Laboratories |
|
Health Care | present | 0 | 1 | 0 | β |
| 63 | πΊπΈ | CSX |
|
Transportation | blocked | 0 | 0 | 0 | β |
| 64 | πΊπΈ | Baker Hughes |
|
Energy | wrong response | 0 | 0 | 0 | β |
| 65 | πΊπΈ | Old Dominion Freight Line |
|
Transportation | present | 0 | 1 | 0 | β |
| 66 | πΊπΈ | Biogen |
|
Health Care | present | 0 | 1 | 0 | β |
| 67 | πΊπΈ | GE HealthCare |
|
Health Care | present | 0 | 1 | 0 | β |
| 68 | πΊπΈ | Constellation Energy |
|
Energy | present | 0 | 1 | 0 | β |
| 69 | πΊπΈ | Lululemon Athletica |
|
Apparel | present | 🤖 1 | 1 | 0 | β |
| 70 | πΊπΈ | Electronic Arts |
|
Technology | present | 🤖 4 | 1 | 0 | β |
| 71 | πΊπΈ | DexCom |
|
Health Care | present | 0 | 1 | 0 | β |
| 72 | πΊπΈ | Kraft Heinz |
|
Food & Beverage | absent | 0 | 0 | 0 | β |
| 73 | πΊπΈ | Zscaler |
|
Computer Security | present | 0 | 1 | 0 | β |
| 74 | πΊπΈ | Onsemi |
|
Technology | present | 0 | 1 | 0 | β |
| 75 | πΊπΈ | CDW |
|
Technology | unreachable | 0 | 0 | 0 | β |
| 76 | πΊπΈ | Ansys |
|
Programming and Developer Software | present | 0 | 1 | 0 | β |
| 77 | πΊπΈ | The Trade Desk |
|
Business Services | present | 0 | 1 | 0 | β |
| 78 | πΊπΈ | Cognizant |
|
Technology | present | 0 | 1 | 0 | β |
| 79 | πΊπΈ | Atlassian |
|
Programming and Developer Software | present | 0 | 1 | 0 | β |
| 80 | πΊπΈ | Xcel Energy |
|
Energy | absent | 0 | 0 | 0 | β |
| 81 | πΊπΈ | Paccar |
|
Motor Vehicles | unreachable | 0 | 0 | 0 | β |
| 82 | πΊπΈ | Verisign |
|
Technology | present | 0 | 1 | 0 | β |
| 83 | πΊπΈ | Warner Bros. Discovery |
|
Media | present | 0 | 1 | 0 | β |
| 84 | πΊπΈ | GlobalFoundries |
|
Semiconductors | present | 0 | 1 | 0 | β |
| 85 | πΊπΈ | Align Technology |
|
Health Care | present | 0 | 1 | 0 | β |
| 86 | πΊπΈ | Take-Two Interactive |
|
Technology | present | 0 | 1 | 0 | β |
| 87 | πΊπΈ | Dollar Tree |
|
Retailing | present | 0 | 1 | 0 | β |
| 88 | πΊπΈ | Sirius XM |
|
Media | present | 0 | 1 | 0 | β |
| 89 | πΊπΈ | Walgreens Boots Alliance |
|
Health Care | present | 0 | 1 | 0 | β |
| 90 | πΊπΈ | Enphase Energy |
|
Energy | present | 0 | 1 | 0 | β |
| 91 | πΊπΈ | First Solar |
|
Energy | present | 0 | 1 | 0 | β |
| 92 | πΊπΈ | Super Micro Computer |
|
Technology | present | 0 | 1 | 0 | β |
| 93 | πΊπΈ | MongoDB |
|
Programming and Developer Software | present | 0 | 1 | 0 | β |
| 94 | πΊπΈ | Okta |
|
Computer Security | present | 0 | 1 | 0 | β |
| 95 | πΊπΈ | DoorDash |
|
Technology | present | 0 | 1 | 0 | β |
| 96 | πΊπΈ | Coinbase |
|
Finance | present | 0 | 1 | 0 | β |
| 97 | πΊπΈ | Expedia Group |
|
Travel and Tourism | present | 0 | 1 | 0 | β |
| 98 | πΊπΈ | eBay |
|
Technology | present | 🤖 9 | 1 | 0 | β |
| 99 | πΊπΈ | Zoom Communications |
|
Programming and Developer Software | present | 0 | 1 | 0 | β |
| 100 | πΊπΈ | Roper Technologies |
|
Programming and Developer Software | present | 🤖 1 | 1 | 0 | β |
| 101 | π¬π§ | Arm Holdings |
|
Semiconductors | present | 0 | 1 | 0 | β |
| 102 | πΊπΈ | Palantir |
|
Technology | present | 0 | 1 | 0 | β |