Applebot-Extended is Apple's control token, like Google-Extended: a Disallow keeps your content out of Apple Intelligence training without affecting Siri and Spotlight search, which use Applebot.. Collects pages to train models; blocking it does not affect whether AI can answer about you today.
Applebot-Extended is Apple's control token, like Google-Extended: a Disallow keeps your content out of Apple Intelligence training without affecting Siri and Spotlight search, which use Applebot.
Training: collects pages to train models; blocking it does not affect whether AI can answer about you today.
There is no request with this name. Applebot-Extended is a robots.txt token only: a robots.txt token only — Applebot does the fetching; Applebot-Extended controls whether the content may be used to train Apple's models.
Apple publishes no IP-range feed for Applebot-Extended; Apple documents reverse-DNS verification for Applebot. Treat the user-agent as a claim, and rate-limit or block by behaviour if it misbehaves.
Blocking Applebot-Extended removes your pages from Apple's model training only.
https://support.apple.com/en-us/119829
Check your site: the AI Crawler Access Check reads your robots.txt for Applebot-Extended and the other AI crawlers and fetches your homepage with each one's real user-agent — the CDN challenge that robots.txt cannot show.
Blocking Applebot-Extended removes your pages from Apple's model training only.
Add a group for it: User-agent: Applebot-Extended on one line and Allow: / on the next. If a User-agent: * group has a Disallow, the named group takes precedence for Applebot-Extended.
User-agent: Applebot-Extended then Disallow: /. A robots.txt token only — Applebot does the fetching; Applebot-Extended controls whether the content may be used to train Apple's models.
Apple publishes no IP-range feed for Applebot-Extended; Apple documents reverse-DNS verification for Applebot. Treat the user-agent as a claim, and rate-limit or block by behaviour if it misbehaves.
The free crawler check reads your robots.txt per bot and fetches your homepage with each one's real user-agent.