Generative engine optimization (GEO) starts with a boring prerequisite: the AI engines have to be able to fetch your pages at all. Great content the crawler can't reach earns zero citations.
The reachability checklist
- robots.txt — allow the AI search crawlers you want citations from: OAI-SearchBot (ChatGPT), Claude-SearchBot, PerplexityBot. These are separate from the training bots (see the crawler directory).
- Edge/CDN — confirm your host isn't blocking those same crawlers at the network layer (common after Cloudflare's 2026 default change).
- Server health — a crawler that gets timeouts or 5xx errors simply moves on.
Verify, don't assume
Run the live crawler check on your key pages and read the per-crawler results. Fix reachability first; content and structure optimizations only pay off once the engines can actually see the page. This is a technical reachability check — it doesn't judge your content, and it isn't advice about crawling policy or licensing.