Can AI crawlers see your site?
Check in seconds whether GPTBot, ClaudeBot, PerplexityBot and search crawlers can actually reach your pages — robots.txt rules plus a live server-response probe. Free, no account.
- Live probes, not just robots.txt
- No account required
- Checked URLs are not stored
- Free to use
Choose your problem
Why it matters
AI assistants and AI search now decide whether your site gets cited. Blocks hide in two layers: your robots.txt rules, and server/CDN-level bot protection (like Cloudflare's default AI-crawler blocking) that robots.txt checkers never see. Our checker tests both and tells you exactly which bot is blocked where. For each crawler's verified user-agent strings and official documentation, see the crawler directory. Browse the ai crawler directory.
Guides
- How AI-Powered Content Tools Help Local Businesses Get Found Online
Discover how AI content tools help local service businesses create SEO-friendly content, save time, and rank higher in local search results.
- How AI Search Results Are Changing Where Customers Find Local Businesses
AI search tools are reshaping local business discovery. Learn how AI results work, what they mean for your visibility, and how to adapt.
- What Is AI Visibility? How to Make Your Company Searchable
Learn how AI tools discover your business and 5 steps to improve your AI visibility in ChatGPT, Google AI, Perplexity, and other AI search tools.
- How AI Is Changing SEO for Local Services
Discover how AI is reshaping SEO for local service businesses and what you need to know to stay competitive in search rankings.
- How AI is Reshaping Online Visibility for Small Businesses
Learn how small businesses use AI tools to improve search rankings, create content, and reach more customers online.
- What Is AI Visibility and Why It Matters for Your Business in 2026
Learn how AI visibility affects your local business in 2026—why search engine algorithms alone aren't enough and what you can do.
- What Is AI Visibility and Why Local Businesses Need It
AI visibility helps local businesses appear in ChatGPT, Google's AI Overview, and other AI tools. Learn why it matters for your service business.
- Why Your Business Isn't Showing Up in Search: 5 Overlooked Visibility Issues
Discover 5 hidden reasons your business isn't ranking in search. Fix technical issues, keyword gaps, and trust signals to boost visibility.
- What Does AI Visibility Mean for Your Business?
Learn how AI visibility affects your business's ability to attract customers through AI search tools and what local service businesses need to do.
- Why Your Local Business Isn't Showing Up in Google Search
Discover why your local business disappears from Google search results and learn the key fixes to reclaim visibility in your market.
- How Local Businesses Appear in AI Search Results
Learn how local businesses get discovered in AI search tools and what steps to take to improve visibility.
- How AI Chatbots Are Finding (and Recommending) Local Businesses
Learn how AI chatbots discover and recommend local services, and what local businesses need to do to be found.
- Why Small Businesses Struggle With Online Visibility (And What's Changed)
Small businesses face intense competition and evolving algorithms. Learn why visibility is harder—and what new tools are changing the game.
- How AI Search Engines Actually Find Local Business Information
Learn how AI search engines discover and verify local business data, from Google Business Profiles to review aggregators.
- AI search visibility (GEO): making sure the assistants can actually read you
GEO / AI-search visibility has a prerequisite people skip: can the AI engines fetch your pages at all? The reachability checklist and how to verify it.
- Why your robots.txt says 'allowed' but AI crawlers still can't reach you
robots.txt allows AI crawlers but they still can't reach you? The block is at your CDN/server layer. How to confirm which gatekeeper is saying no.
- How to opt out of AI training without hurting your Google Search ranking
Google-Extended lets you opt out of Gemini training with zero effect on Google Search ranking (per Google). How to set it — and why not to block Googlebot by mistake.
- GPTBot vs OAI-SearchBot vs ChatGPT-User: which one to allow
OpenAI's GPTBot (training), OAI-SearchBot (search), and ChatGPT-User (user fetch) are three separate crawlers. Which to allow to stay findable without being trained on.
- Is your site blocked from ChatGPT, Claude, and Perplexity? How to check both layers
Two layers decide whether ChatGPT/Claude/Perplexity can reach your site — robots.txt and your CDN. Most checkers see only one. How to check both.
Requirement news
- Cloudflare's Sept 15 default AI-crawler blocking: what site owners should check
Cloudflare's Sept 15 2026 default blocking of AI crawlers on ad-supported sites means robots.txt can say 'allowed' while the edge says 'blocked' — here's how to check both.