Check 0.4 · Crawler access

Are AI crawlers actually visiting your site?

AI crawler visits are being seen. Access that is allowed on paper and access that happens are different things.

By Reflexa Technologies — the team building Reflexa, the agent that works on your site · updated 16 October 2026

What the check looks at

This one cannot be answered from outside. The page check reads your site; whether GPTBot, ClaudeBot or PerplexityBot ever arrive is only visible from inside — in the requests the site receives. The agent on the site records every such request and shows them page by page.

Why it matters

Access that is allowed on paper and access that happens are different things. A site can pass every door check and still never be fetched because nothing links to it, or because a layer nobody checked drops the requests. Observed visits are the proof; their absence is a question.

How to see it on your own site

Search your server or CDN logs for the AI user-agents, verify the source IPs against the vendors' published ranges, and count per page. Or install the agent: it does exactly this on every request and separates a page fetched to answer a live question from one crawled to learn.

AI bot verifier → — free, no signup.

How to fix it

What the agent does: the agent on the site records every AI request as it happens — this is the counting the page check cannot do from outside.

Related checks

Read next

Part of the 23 checks · group: Crawler access — robots.txt, CDN/bot filter, login walls, noindex, HTTP status, sitemap, canonical.

Questions people ask.

Why is this "not checkable from outside"?

Because a visit is an event on your server. An outside check can only see your site the way a visitor does; it cannot see who else is visiting.

Does Google Analytics show AI visits?

Not the crawler fetches — those run no JavaScript, so no tag fires. Analytics may show a person who clicked through from an AI answer, which is a different and much smaller number.

See this check run on your pages.

The page check reads your public pages from outside and runs all 22 — free, no signup, no AI engine involved.

Run the free check →