Google-Extended is not a bot you will see in your logs. Collects pages to train models; blocking it does not affect whether AI can answer about you today.
Google-Extended is not a bot you will see in your logs. It is a control token: a Disallow for Google-Extended tells Google not to use the crawled content to train Gemini or to ground its answers — without affecting Google Search.
Training: collects pages to train models; blocking it does not affect whether AI can answer about you today.
There is no request with this name. Google-Extended is a robots.txt token only: a robots.txt token only — there is no crawler with this user-agent; Googlebot does the fetching and Google-Extended controls whether the content may be used for Gemini training and grounding.
Google publishes its IP ranges as a feed at https://developers.google.com/static/search/apis/ipranges/googlebot.json (the fetch itself is Googlebot). A request claiming to be Google-Extended from an address outside those ranges is an impostor.
AI bot verifier → checks an IP and user-agent against the published ranges and reverse DNS, free.
Blocking Google-Extended keeps your content out of Gemini training and grounding. Google documents that it does not affect ranking in Search or inclusion in AI Overviews, which draw on the Search index.
https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers
Check your site: the AI Crawler Access Check reads your robots.txt for Google-Extended and the other AI crawlers and fetches your homepage with each one's real user-agent — the CDN challenge that robots.txt cannot show.
Blocking Google-Extended keeps your content out of Gemini training and grounding. Google documents that it does not affect ranking in Search or inclusion in AI Overviews, which draw on the Search index.
Add a group for it: User-agent: Google-Extended on one line and Allow: / on the next. If a User-agent: * group has a Disallow, the named group takes precedence for Google-Extended.
User-agent: Google-Extended then Disallow: /. A robots.txt token only — there is no crawler with this user-agent; Googlebot does the fetching and Google-Extended controls whether the content may be used for Gemini training and grounding.
Google publishes its IP ranges as a feed at https://developers.google.com/static/search/apis/ipranges/googlebot.json (the fetch itself is Googlebot). A request claiming to be Google-Extended from an address outside those ranges is an impostor.
The free crawler check reads your robots.txt per bot and fetches your homepage with each one's real user-agent.