Magazine · Guide

AI SEO: How to Get Found and Cited in AI Search

By GraphoRank Editorial · · 22 min read

There is no single AI SEO switch. Google says nothing extra is required beyond being indexed and eligible for a snippet. ChatGPT, Perplexity, Gemini and Copilot each have their own controls, so you decide bot by bot. A realistic first month is four jobs: let the right bots in, fix what stops pages being read, rewrite key pages so they answer questions directly, and measure.

Key takeaways

  • AI SEO is classic SEO plus a few new decisions: which AI crawlers you allow, whether your key facts are easy to quote, and how you measure visibility inside AI answers.
  • Blocking GPTBot does not remove you from ChatGPT search. Blocking OAI-SearchBot does.
  • robots.txt is a polite request, not a lock, and vendors say some user-triggered fetchers may not follow it.
  • Nearly every study finds fewer clicks when an AI answer appears, but the size of the drop varies a lot and the studies measure different things.
  • llms.txt and special markup are optional. The first 30 days need only a browser, your robots.txt and Search Console.

What is AI SEO?

AI SEO is the practice of making your website easy for AI search engines to crawl, understand, quote and link to, so that your pages appear as cited sources in AI-generated answers.

It sits on top of classic search engine optimization (SEO). You still need pages that can be crawled and indexed, and content that answers real questions. What changes is who reads your page first: often a machine that summarizes it for a person.

You will also see this called AI search optimization, answer engine optimization (AEO) or generative engine optimization (GEO). GEO is the research-led label for improving how generative AI systems use and cite your content; the theory and evidence are in our Generative Engine Optimization guide. This page is the practical side: what each AI engine says it does, which switches you control, and what to do in your first 30 days.

The terms you need:

  • AI Overviews are the AI-written summaries Google shows at the top of some search results.
  • AI Mode is Google’s conversational search experience. Alphabet reported in July 2026 that AI Mode had passed 1 billion monthly active users since its global expansion the previous October.
  • A crawler (or bot) is a program that visits web pages automatically. AI companies run several kinds: some collect content to train models, some fetch pages to build search answers, and some fetch a page only when a person asks.
  • robots.txt is a plain text file at the root of your site that tells crawlers which parts they may visit.
  • A citation is a link or source label an AI answer attaches to a statement.

Is it worth the effort?

The audience is real, and it is growing alongside search, not instead of it.

  • 49% of US adults used AI chatbots in a February 2026 survey Pew Research Center, 2026
  • 33% in 2024 Pew Research Center; the 2024 question was asked of a narrower group

Treat the jump as indicative. The 2024 question was asked of a narrower group than the 2026 one.

The Reuters Institute describes the growth of chatbot use for news as “fast rather than explosive”. People are adding AI to search, not dropping search.

How does each AI search engine find and use your content?

Each AI search engine has its own crawlers and its own rules, so there is no single AI SEO switch. The table summarizes what the vendors’ own documentation says, as we read it on 2026-10-10. Where we found no official guidance, the table says so.

Engine What it draws on Crawler and controls (per vendor docs) What the vendor says
Google AI Overviews and AI Mode Google Search index Normal Googlebot rules; snippet controls (nosnippet, data-nosnippet, max-snippet, noindex) No additional requirements; no special markup needed; page must be indexed and snippet-eligible
ChatGPT search OpenAI’s search index OAI-SearchBot for search, GPTBot for training, ChatGPT-User for user-requested fetches Opted-out sites will not be shown in ChatGPT search answers; robots.txt may not apply to ChatGPT-User
Perplexity Perplexity’s search index PerplexityBot for search, Perplexity-User for user actions PerplexityBot is not used for foundation-model training; Perplexity-User generally ignores robots.txt
Gemini Google models, with grounding Google-Extended robots.txt token (no separate user-agent string) Controls use for training future Gemini models and for grounding; does not affect Search inclusion
Microsoft Copilot Bing Webmaster Tools reports on content cited in Copilot, Bing AI summaries and select partner integrations Bing says it respects robots.txt and other supported controls Microsoft’s AI Performance report shows how often your pages are cited; it advises clear structure, fresh content and IndexNow

Google AI Overviews and AI Mode

Google says there are no extra steps. Its documentation states that “there are no additional requirements to appear in AI Overviews or AI Mode” and that you do not need special schema.org structured data. A page must be indexed and eligible to be shown with a snippet (the short text excerpt Google displays under a result).

Google’s separate guide on optimizing for generative AI features adds more:

  • You do not need new machine-readable files, AI text files, markup or Markdown to appear in Google Search.
  • There is “no requirement to break your content into tiny pieces”.
  • Structured data “isn’t required for generative AI search”.
  • llms.txt “will neither harm nor help” Search visibility.
  • Seeking inauthentic mentions of your brand across the web “isn’t as helpful as it might seem”.

What you control: the standard snippet tools. nosnippet and data-nosnippet stop text being quoted, max-snippet limits how much, and noindex removes the page from Search. Traffic from AI features is counted in Search Console under the “Web” search type, so it is blended with normal results.

Two cautions. This is guidance about eligibility, not proof of what gets cited. And trade press reported that Google later narrowed some of this wording, while Chrome’s own Lighthouse tool now audits for llms.txt, which looks inconsistent. It is not: the two products answer different questions, Search visibility on one side and readiness for browser agents on the other. Re-read Google’s pages before you act on a single sentence.

How often AI Overviews appear also swings. Semrush tracked more than 10 million keywords in 2025:

  • 6.49% of keywords showed an AI Overview in January Semrush, 2025
  • 24.61% at the peak, in July Semrush, 2025
  • 15.69% in November Semrush, 2025

That is vendor data, but it shows why one month’s snapshot should not drive a strategy.

OpenAI documents three separate crawlers. The split matters more than anything else on this page:

  • GPTBot collects content that may be used for training.
  • OAI-SearchBot powers ChatGPT search. OpenAI says opted-out sites “will not be shown in ChatGPT search answers”, and that the change takes about 24 hours to apply.
  • ChatGPT-User fetches a page when a person asks ChatGPT to open it. OpenAI says robots.txt rules may not apply to it.

So blocking GPTBot does not remove you from ChatGPT search, because that is a different bot. Blocking OAI-SearchBot does. For a sense of how common the choice is, the HTTP Archive’s 2025 Web Almanac found GPTBot named in the robots.txt of 4.5% of desktop sites (2.9% in 2024). That counts sites that mention the bot, not sites that block it.

Secondary source. OpenAI’s Publishers and Developers FAQ is reported (we could not open the help page itself, which returned an access error, so this rests on secondary quotation) as saying its ChatGPT Atlas browser uses ARIA labels and roles, the same markup that supports screen readers, and that publishers should give buttons, menus and forms clear roles, labels and states. Check that against OpenAI’s current page before repeating it.

On scale, Similarweb’s panel shows ChatGPT’s share of generative AI web visits shrinking while Gemini’s grows:

  • about 53% ChatGPT share of generative AI web visits in May 2026 Similarweb, 2026
  • about 76% ChatGPT share in June 2025 Similarweb, 2025
  • about 27 to 28% Gemini share in May 2026, up from under 9% in June 2025 Similarweb, 2026

Directional only. Similarweb does not disclose its panel size. Either way, no single engine is worth optimizing for alone.

Perplexity

Perplexity’s documentation lists two bots. PerplexityBot is used to surface and link websites in Perplexity search results and, Perplexity says, is not used to train foundation models. Perplexity-User supports user actions; because those fetches are triggered by a person, Perplexity says it “generally ignores robots.txt rules”.

Whether bots really follow robots.txt is contested. Here is who claims what:

  • Documented by the vendor: the Perplexity-User behavior above.
  • Alleged by a third party: in August 2025, Cloudflare reported that Perplexity used undeclared crawlers that impersonated a normal Chrome browser, and that tests on new domains with robots.txt set to disallow-all still returned content details. Perplexity publicly disputed it, arguing that assistants acting on a user’s request are not crawlers (as reported by press coverage; we could not open Perplexity’s own post).
  • Measured independently: an academic preprint studying 130 self-declared bots found that stricter robots.txt directives are less likely to be obeyed and that some categories, including AI search crawlers, rarely check robots.txt at all.

Weigh the sources. Cloudflare sells bot-management services, so its report is a vendor allegation, not an independent finding. The preprint’s data come from one institution’s websites.

robots.txt is a polite request, not a lock. If content must stay private, put it behind a login.

Gemini

Gemini is Google’s AI assistant. Alphabet reported 950 million monthly active users for the Gemini app in July 2026. For site owners the control is Google-Extended, a robots.txt token (not a separate crawler name).

Google says it controls whether your content is used to train future Gemini models and for grounding, meaning checking an answer against live content. Google also states that it “does not impact a site’s inclusion in Google Search nor is it used as a ranking signal”.

No official guidance on citation. Google documents training and grounding access. On how the Gemini app chooses which sources to cite, no official guidance found. Google’s general Search guidance above is the closest official reference.

Microsoft Copilot

Microsoft’s guidance is about measurement and basics. In February 2026 Bing introduced an AI Performance report in Bing Webmaster Tools (public preview). It shows how publisher content appears across Microsoft Copilot, AI-generated summaries in Bing and select partner integrations.

The report gives you:

  • total citations
  • average cited pages per day
  • the “grounding queries” the AI used to retrieve your content
  • page-level citation activity

Microsoft’s advice to publishers in that post:

  • Use the report to see which content is already cited.
  • Strengthen structure and clarity on pages that are indexed but rarely cited.
  • Keep content fresh and accurate.
  • Use IndexNow to notify participating search engines of changes.

It also says Bing respects content owner preferences set through robots.txt and other supported controls.

No citation guarantee. We did not find a Microsoft document that lists which pages Copilot will cite or guarantees citations. Treat the report as a way to see your own data, and keep the basics: pages that are crawlable, indexable and clear.

What about Claude?

Anthropic documents three bots: ClaudeBot for training, Claude-User for user-directed fetches and Claude-SearchBot for search. It says they respect “do not crawl” signals in robots.txt and that it supports the non-standard Crawl-delay extension, and it advises against blocking by IP address. Claude was not among the engines on our brief, but when you write your robots.txt you face the same three-way decision.

What happens to your clicks when AI answers appear?

Nearly every measurement shows fewer clicks on results when an AI answer is present, but the size of the drop ranges from about a third to about two thirds depending on what is measured and how. Four headline figures:

  • 8% vs 15% of visits with a result click, with vs without an AI summary Pew Research, US panel, March 2025
  • 34.5% lower CTR for the top result when an AI Overview is present Ahrefs, 2025
  • about 58% lower CTR for the top result than forecast Ahrefs, 2026 update
  • 0.70%, 0.52%, 1.45% organic CTR when an AI Overview cites you, does not cite you, or is absent Seer Interactive, Q3 2025

The studies measure different things, so read the middle column of the table first.

Study What it compares Result
Pew Research (US panel, 900 adults, March 2025, 68,879 Google searches) Visits with an AI summary vs visits without Clicked a result on 8% of visits with a summary vs 15% without; 1% clicked a link inside the summary; the session ended on 26% of visits with a summary vs 16% without
Ahrefs, 2025 (300,000 keywords, desktop, March 2024 vs March 2025) Top-ranked page’s click-through rate (CTR) with vs without an AI Overview 34.5% lower CTR for position 1 when an AI Overview is present
Ahrefs, 2026 update (same sample size, December 2023 vs December 2025) Same, over two years, against a forecast About 58% lower CTR for position 1 than forecast (0.016 actual vs about 0.037 expected)
Seer Interactive (3,119 informational queries, 42 client organizations, June 2024 to September 2025; CTRs below are Q3 2025) Organic CTR when an AI Overview cites you, does not cite you, or is absent 0.70% (cited), 0.52% (not cited), 1.45% (no AI Overview)
Google (Liz Reid, August 2025) Total organic click volume from Search to websites “Relatively stable year-over-year”; no figures, period or method published

Why do the numbers disagree?

Each one measures something different.

  • Pew watches real browsing and compares visits. It covers one month in the US and is observational, and Google disputes the framing.
  • Ahrefs uses search console data on 300,000 keywords, desktop only. The 58% figure replaces the earlier 34.5% because the 2026 study spans two years and uses a different baseline. Keywords with no AI Overview also lost clicks: their top-position CTR fell about 49% over the same period (0.076 to 0.039). So not all of the decline can be pinned on AI Overviews.
  • Seer separates cited from not-cited. Its authors report that cited brands got roughly 35% more organic clicks, but they say causation is unproven. Its overall CTR fell in all three groups compared with a year earlier, including queries with no AI Overview.
  • Google is measuring something else: total clicks across all websites, not clicks per query. Both can be true at once if there are more searches overall.

Google’s claim cannot be checked. Google has not published the data behind it.

What should a small business take from this?

Being cited appears to beat not being cited, though the evidence is correlational. A citation does not guarantee a visit, either: in Pew’s data only 1% of visits included a click on a link inside the summary.

Clicks are not the only goal. The Reuters Institute’s 2026 report found that 42% of people who use chatbots for news say they always or often click through to sources, compared with 44% for search.

Handle that comparison with care. It is self-reported, about news only, and the report cautions the two figures come from quite different user bases.

If your business depends on informational traffic, plan for fewer clicks per search and measure what matters to you: enquiries, sign-ups, calls, sales. If people have to come to your site to buy, an AI answer that names you can still help.

What is a practical 30-day AI SEO plan?

A realistic first month has four jobs: let the right bots in, fix what stops pages being read, rewrite key pages so they answer questions directly, and set up a way to measure results. You do not need a tool or a developer for most of it. The plan is editorial advice; where a source backs a step, it is noted.

Week 1: Check access

  1. Open your robots.txt. Type yourdomain.com/robots.txt in a browser. Look for lines mentioning GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, ClaudeBot, Google-Extended. If you see none, no AI-specific rules exist (you would not be unusual: the 2025 Web Almanac found GPTBot named in 4.5% of desktop robots.txt files and ClaudeBot in 3.6%).
  2. Decide on purpose. Do you want to appear in AI search answers? If yes, allow the search bots (OAI-SearchBot, PerplexityBot) even if you block training bots like GPTBot. The vendors document these as separate things.
  3. Confirm key pages are indexed. In Google Search Console, check that your most important pages are indexed and not set to noindex or nosnippet. Google says indexing and snippet eligibility are the entry requirements for AI Overviews and AI Mode.
  4. Check that content is in the page. Open an important page and see whether the text is visible without clicking tabs or waiting for scripts. Content hidden behind interactions is harder for any automated reader to see.

Week 2: Fix the basics machines trip over

  1. List your 10 most important pages (the ones that make you money or define what you do) and make sure each can be reached from your homepage in two or three clicks through normal links.
  2. Find orphan pages. An orphan page is a page that no other page on your site links to. Machines that follow links never find it. Add links to it from related pages, or remove it.
  3. Fix labels. Buttons, menus and forms should have clear names. The Chrome Lighthouse documentation for its Agentic Browsing category says agents rely on the accessibility tree (the structured version of a page that screen readers use) as their primary data model, and describes the category as experimental. If your site works for someone using a screen reader, it is better prepared for an AI agent too. The OpenAI quote above points the same way, but remember its source caveat.
  4. Keep one version of each page. Duplicate and near-duplicate pages split signals. Point duplicates at one main URL (a canonical).

Week 3: Rewrite for answers

  1. Write down 10 real questions customers ask you by email, phone or in sales calls. These are your prompts.
  2. Answer each in the first two or three sentences of a page or section, in plain words, with the subject named (“Our weekly plan costs…”, not “It costs…”). Then give detail below.
  3. Add facts a machine can quote: specific numbers, dates, names, places, with a source or a date where possible. A preprint that audited 1,702 citations across Brave Summary, Google AI Overviews and Perplexity found metadata and freshness, semantic HTML and structured data most associated with being cited. That study is observational and covers English-language B2B software pages, so treat it as a hint, not a rule.
  4. Add a short FAQ to pages that deserve one, using questions customers really ask. Do not add structured data in the belief that Google requires it: Google says it does not.
  5. Make the page date honest. Update the “last updated” date only when the content really changed.

Week 4: Earn mentions and measure

  1. Look for where your market is discussed. A preprint comparing AI search with Google reports a systematic and overwhelming bias towards earned media (coverage by third parties) over brand-owned and social content. Its abstract gives no figures, so take the direction, not a number. In practice: get listed and described accurately in directories, industry publications and partner sites. Google says seeking inauthentic mentions is not as helpful as it might seem, so do not buy or fake them.
  2. Test by hand. Ask each AI engine your 10 questions. Write down whether you are cited, whether the answer is correct, and which pages it cites instead. Do this monthly with the same questions.
  3. Set up measurement. In analytics, look for referrals from chatgpt.com, perplexity.ai and similar. Note that Google’s AI traffic is not separated out in Search Console.
  4. Decide about llms.txt last. An llms.txt file is a plain text file at your site root listing key pages for AI systems. Evidence for its value is thin (see below). If your content system can generate it in minutes, fine. Otherwise skip it. You can check an existing file with our llms.txt checker and read the details in the llms.txt guide.

Why testing by hand matters. Accuracy is a real issue: in a test published in March 2025 by Columbia Journalism Review’s Tow Center, eight AI search tools collectively gave incorrect answers to more than 60% of queries asking them to identify the source article of a news excerpt. It was an identification task, so it does not generalize to every question.

After 30 days you have crawl access decided, pages reachable, key answers written and a baseline to compare against. Our agentic navigability article explains why link structure matters beyond this checklist.

Is llms.txt worth adding?

Not as a priority. Adoption figures range widely because the studies count different things.

Study What it counted Adoption
Web Almanac 2025 Desktop sites 2.13%
SE Ranking, 2025 Nearly 300,000 domains 10.13%
Common Crawl, July 2026 Sampled URLs returning a text file 11.72%
Ahrefs, May 2026 Domains with traffic in Ahrefs Web Analytics, a customer base that skews technical 28%

Not comparable. Common Crawl’s authors say their figure is not comparable to the Web Almanac’s.

On effect, the evidence is thin:

  • SE Ranking found no correlation between having the file and being cited.
  • Ahrefs found that 97% of the files it observed received zero requests in May 2026.
  • Google says it will neither harm nor help Search visibility.
  • Chrome’s Lighthouse agentic audit checks that the file exists, for a different purpose: readiness for browser agents.

A fetch is not proof of use, and our statistics page collects more on adoption: AI search statistics.

Which tools help with AI SEO?

AI SEO tools fall into three categories, and the right one depends on the question you need answered: “Am I mentioned?”, “Can machines read my site?” or “How do I run all my SEO in one place?” Pick by question, not by brand. We name vendors only to show what each category looks like. We have not tested them and list no prices (check each vendor’s pricing page).

The market is also moving. Adobe agreed to acquire Semrush in November 2025 (closed April 2026 per Semrush’s newsroom), and Sitecore announced in June 2026 that it was acquiring Scrunch.

Category The question it answers Examples (as described on vendor pages, October 2026) Good fit
Visibility trackers When people ask AI engines about my topic, am I mentioned and cited? Profound, Peec AI, Otterly.AI, AthenaHQ, Goodie, Rankscale Brands and agencies tracking many prompts over time
Site-structure and readiness analysis Can crawlers and agents find, read and navigate my pages? Cloudflare Agent Readiness (checks protocol items such as robots.txt, sitemap, Markdown for agents), Scrunch (AI-search visibility platform with an agent-facing content layer), page audits in some trackers Anyone whose site is the bottleneck
Classic SEO suites with AI features Can I keep rankings, backlinks and AI visibility in one subscription? Semrush AI toolkit, Ahrefs Brand Radar, SE Ranking AI Search Teams already using these suites

How to choose

  • Visibility trackers work by sending a list of prompts to AI engines and recording which brands and sources appear. They tell you the score but not why. Check which engines and countries a plan covers and how many prompts you get.
  • Structure analysis looks at your site itself. Cloudflare’s Agent Readiness score is protocol-based. In the pages we reviewed, we found no tool that scored a site’s internal link graph on a single number. GraphoRank does that: it scores how well AI agents can navigate, read and cite a site on a 0 to 100 scale, and it sits in this category.
  • Classic suites are the easy add-on if you already pay for one.

Cloudflare’s own dataset gives a sense of where sites stand:

  • 78% of sites have a robots.txt Cloudflare Agent Readiness dataset
  • 4% declare AI usage preferences Cloudflare Agent Readiness dataset
  • 3.9% pass the Markdown-for-agents check Cloudflare Agent Readiness dataset

On demand: a Minuttia survey of 599 SEO and content professionals found 82% had implemented answer engine optimization to some degree, yet almost 40% had no dedicated budget.

Read the survey with care. It was run with a newsletter audience, Minuttia (an agency) acknowledges professional bias, and “implemented to some degree” includes pilots. Our interpretation: it describes SEO professionals, not business owners, so it says little about how many small businesses have adopted AEO.

If you are small, the 30-day plan costs nothing but time.

Quick answers: what should I do first?

  1. Make sure the pages you care about are indexed and reachable by ordinary links.
  2. Decide, bot by bot, which AI crawlers may visit, and write it in robots.txt.
  3. Put the direct answer first on every key page.
  4. Test your own questions by hand, monthly.
  5. Treat llms.txt and special markup as optional, not as the plan.

If you want a measured starting point instead of a manual one, run a scan with GraphoRank to see how well agents can navigate and cite your site, then compare plans on the pricing page.

FAQ

What is AI SEO?

AI SEO is the practice of making a website easy for AI search engines to crawl, understand, quote and link to. It builds on classic SEO: pages must still be crawlable and indexed, and the content must answer real questions clearly. The goal is to be cited in AI answers, not only to rank in a list of links.

Is AI SEO different from classic SEO?

Mostly it is classic SEO plus a few new decisions. Google says there are no additional requirements to appear in AI Overviews or AI Mode beyond being indexed and eligible for a snippet. The new decisions are which AI crawlers you allow, whether your key facts are easy to quote, and how you measure visibility inside AI answers.

Do I need llms.txt for AI SEO?

No evidence in this guide's sources shows that it helps. Google's documentation says llms.txt will neither harm nor help visibility in Google Search. SE Ranking found no correlation between having the file and being cited, and Ahrefs found that 97% of the llms.txt files it observed received zero requests in May 2026. It is cheap to add but not a priority.

Do I need special schema markup to appear in AI Overviews?

Google says no: structured data is not required for generative AI search, and no special schema.org markup is needed for AI Overviews or AI Mode. A preprint on citation behaviour found structured data and semantic HTML associated with more citations, but that study is observational and covers English-language B2B software pages.

If I block GPTBot, will I disappear from ChatGPT search?

No. OpenAI documents GPTBot as the training crawler and OAI-SearchBot as the crawler for ChatGPT search. Blocking GPTBot does not remove you from ChatGPT search answers. Blocking OAI-SearchBot does, and OpenAI says it takes about 24 hours to apply.

Does robots.txt control every AI bot?

No. OpenAI says robots.txt rules may not apply to ChatGPT-User, which acts on a person's request, and Perplexity says Perplexity-User generally ignores robots.txt for the same reason. A study of 130 self-declared bots also found that some categories, including AI search crawlers, rarely check robots.txt.

Does Google-Extended affect my Google rankings or AI Overviews?

Google says Google-Extended controls whether content is used to train future Gemini models and for grounding, and that it does not affect inclusion in Google Search and is not a ranking signal. Google's page does not mention AI Overviews in that description.

How do I know if AI search sends me traffic?

For Google, AI feature traffic is counted in Search Console under the Web search type, mixed with ordinary results, so it cannot be separated there. For other engines, check your analytics for referral sources such as chatgpt.com or perplexity.ai, and use a visibility tracker if you want to see which answers cite you.

Do AI answers actually reduce clicks?

Most measurements say yes, by very different amounts. Pew found people clicked a result on 8% of visits with an AI summary versus 15% without. Ahrefs measured a 58% lower click-through rate for the top result in its 2026 update. Google says total organic click volume has been relatively stable. The studies measure different things.

Do I need an AI SEO tool?

Not to start. The first 30 days of the plan need only a browser, your robots.txt, Search Console and a list of real customer questions. Tools become useful once you want to track many prompts over time or audit a large site. Pick the category (tracker, structure analysis or SEO suite) that matches the question you need answered.

How long does AI SEO take to work?

No source in the research behind this guide measures a typical time to results, so any promise would be a guess. What is verifiable is what you control: crawler access, page quality and structure can be fixed within weeks. Whether an engine then cites you is up to the engine.

Questions and answers →

Sources

  1. Google Search Central: AI features and your website (October 10, 2026)
  2. Google Search Central: Optimizing your website for generative AI features (October 10, 2026)
  3. Google Search Central: Google crawlers (Google-Extended) (October 10, 2026)
  4. OpenAI: crawlers GPTBot, OAI-SearchBot, ChatGPT-User (October 10, 2026)
  5. OpenAI: Publishers and Developers FAQ (quoted by Simon Willison; primary not opened) (October 10, 2026)
  6. Perplexity: crawlers PerplexityBot and Perplexity-User (October 10, 2026)
  7. Anthropic: does Anthropic crawl data from the web and how can site owners block the crawler (October 10, 2026)
  8. Bing Webmaster Blog: Introducing AI Performance in Bing Webmaster Tools (public preview) (October 10, 2026)
  9. Columbia Journalism Review, Tow Center: AI search engines are bad at citing news (October 10, 2026)
  10. Cloudflare: Perplexity stealth crawling (October 10, 2026)
  11. Kim et al.: Scrapers selectively respect robots.txt (arXiv) (October 10, 2026)
  12. Alphabet Q2 2026 earnings remarks (Sundar Pichai) (October 10, 2026)
  13. Pew Research Center: Google users are less likely to click links when an AI summary appears (October 10, 2026)
  14. Ahrefs: AI Overviews reduce clicks (2025) (October 10, 2026)
  15. Ahrefs: AI Overviews reduce clicks, 2026 update (October 10, 2026)
  16. Seer Interactive: AIO impact on organic and paid CTR (September 2025 update) (October 10, 2026)
  17. Google (Liz Reid): AI search is driving more queries and higher quality clicks (October 10, 2026)
  18. Semrush: AI Overviews study (October 10, 2026)
  19. Reuters Institute: Digital News Report 2026, executive summary (October 10, 2026)
  20. Pew Research Center: about half of US adults use AI chatbots (2026) (October 10, 2026)
  21. Similarweb: gen-AI statistics 2026 (October 10, 2026)
  22. Kumar and Palkhouski: AI Answer Engine Citation Behavior, the GEO-16 framework (arXiv) (October 10, 2026)
  23. Chen et al.: Generative Engine Optimization, How to Dominate AI Search (arXiv) (October 10, 2026)
  24. HTTP Archive Web Almanac 2025: SEO (October 10, 2026)
  25. SE Ranking: llms.txt and AI citations (October 10, 2026)
  26. Ahrefs: llms.txt study of 137K sites (October 10, 2026)
  27. Common Crawl: content analysis of llms.txt files, July 2026 crawl (October 10, 2026)
  28. Chrome Lighthouse: Agentic Browsing category (October 10, 2026)
  29. Cloudflare: Agent Readiness score (October 10, 2026)
  30. Minuttia: State of AEO (October 10, 2026)
  31. Sitecore press release: Sitecore acquires Scrunch (October 10, 2026)
  32. Semrush newsroom: Adobe to acquire Semrush (October 10, 2026)