Analyze your robots.txt for AI crawlers
Paste a robots.txt and get a plain per-crawler verdict for every AI and search bot that matters — parsed entirely in your browser.
Parsed entirely in your browser — nothing is sent to our server.
How it works
- Paste your robots.txt; nothing is uploaded — parsing runs in your browser.
- Each crawler is evaluated with real robots.txt group + longest-match rule semantics.
- Verdicts cite each crawler's verified token; the crawler directory links the official docs.
- For server-level blocks (e.g. Cloudflare), use the live crawler check instead.
Need the exact published limit for a specific portal? See the requirements directory — every entry cites its official source and verification date.