AI search engines cite websites they can reach, understand, recognise and trust. In practice, 12 ranking factors fall into four groups. These are access, structure, entity and authority. Most websites already lose on the first group. They block AI crawlers by accident, or they hide their content behind slow, script-heavy pages.
Your buyers no longer just search; they ask. A founder asks ChatGPT for the best payroll software in India. A shopper asks Google’s AI Mode which electric cycle suits a 10 km commute. The answer names a few brands and cites a few sources. Everyone else is invisible.
This guide explains how ChatGPT, Gemini, Perplexity, Claude and Google AI Overviews choose those sources. It also shows how to check your own site against each factor.
Key numbers from our audits
These figures come from the Spacemen AI Agent Readiness Index, which is based on websites scanned with our readiness tool:
- 94% of sites have no llms.txt file.
- 67% block at least one major AI crawler in robots.txt, often without knowing it.
- 61% publish no Article schema with an author and date.
- 99% have no AI Instructions page.
In this guide
- How AI search engines choose which sources to cite
- Group 1: Access factors
- Group 2: Structure factors
- Group 3: Entity factors
- Group 4: Authority factors
- Google SEO vs AI search: what changes
- What matters less than people think
- A 12-point audit checklist
- Frequently asked questions
How do AI search engines choose which sources to cite?
AI search engines run live web searches, read the top results and pick the passages that answer the question best. They then write an answer and link to the pages they used. Your page must be found, read and judged trustworthy, all within seconds.
Training data vs live retrieval
There are two ways your brand can appear in an AI answer:
- Training data: what the model learned before its cut-off date. Influencing it takes months or years. It relies on your brand being mentioned widely across the web.
- Live retrieval: what the engine finds on the web when the user asks. This is where most citations come from today, and it can change within weeks.
This guide focuses on live retrieval, because it is where you can make a difference fastest.
Query fan-out: one question becomes many searches
When a user asks a long question, the engine splits it into several smaller searches. Take the question “Which CRM is best for a 20-person sales team in India?” The engine may search for CRM pricing in India, CRM reviews, and CRMs for small teams.
The result: you do not need to rank for the full question. You need clear pages that answer each of the smaller questions behind it.
How the main AI engines differ
| Engine | Where it finds sources | What it tends to cite |
|---|---|---|
| ChatGPT search | Its own crawler (OAI-SearchBot) and third-party search providers | Clear explainer pages, reviews, news and well-known brand sites |
| Google AI Overviews and AI Mode | Google’s own search index | Pages that already rank well and answer each sub-question |
| Perplexity | Its own crawler (PerplexityBot) and live web search | Recent, specific pages, often with several sources per answer |
| Gemini | Google Search | Similar to AI Overviews, plus Google properties such as YouTube |
| Claude | Web search when enabled, plus its own crawlers | Authoritative, well-structured pages |
Each engine has its own crawler and index. The same core factors, however, decide whether you are cited on all of them.
For step-by-step help on the two biggest platforms, read our guides on how to get cited by ChatGPT and Google AI Mode optimisation.
Group 1: Access factors (can AI reach your content?)
Access factors decide whether an AI engine can fetch your pages at all. If a crawler is blocked or a page fails to load, nothing else in this guide matters. Fix this group first.
Factor 1: AI crawlers are allowed in robots.txt
Each AI company uses named crawlers. Your robots.txt file tells them what they may read. These are the main ones:
| Crawler | Company | Purpose |
|---|---|---|
| GPTBot | OpenAI | Collects content to train models |
| OAI-SearchBot | OpenAI | Finds pages for ChatGPT search results |
| ClaudeBot | Anthropic | Collects content for Claude |
| PerplexityBot | Perplexity | Indexes pages for Perplexity answers |
| Google-Extended | Controls whether Google may use your content for Gemini models |
Many sites block these crawlers by accident. Security plugins, CDN bot rules and old robots.txt templates are common causes. Check your robots.txt file and your firewall settings.
You can still block model training while allowing search crawlers. For example, you can block GPTBot but allow OAI-SearchBot.
Factor 2: Pages load fast and work without JavaScript
Many AI crawlers do not run JavaScript. If your main content only appears after scripts load, the crawler may see an empty page. Use server-side rendering or static HTML for important content.
Speed matters as well. In our audits, 38% of sites take more than 1.5 seconds to send the first byte of a page. Slow responses can cause crawlers to time out before they read your content.
Factor 3: Content is easy to discover
AI crawlers find pages through links, sitemaps and simple guide files. Three basics help:
- An XML sitemap at a standard location, listed in robots.txt. 23% of the sites we audited have none.
- An llms.txt file: a short Markdown file at your root domain that lists your most important pages for AI tools.
- Clean internal links, so every key page is no more than three clicks from the homepage.
Not sure how llms.txt differs from schema markup? Our guide to llms.txt vs AI Instructions pages vs schema explains what each one does.
Group 2: Structure factors (can AI understand your content?)
Structure factors decide how easily an AI engine can lift a clear answer from your page. Engines quote passages, not whole pages. The page that gives the cleanest answer to a sub-question usually wins the citation.
Factor 4: Each section opens with a direct answer
Start each section with a two- to three-sentence answer of 40 to 60 words. Then add the detail, examples and evidence. This is called an answer block.
Weak opening: “In today’s fast-changing digital landscape, many businesses are starting to wonder what an AEO audit really involves…”
Strong opening: “An AEO audit checks four things. Can AI crawlers reach your site? Do your pages give clear answers? Are your brand facts consistent? Do other trusted sites mention you?”
The strong version can be quoted as it stands. The weak one cannot.
Factor 5: Schema markup describes the page
Schema markup is code that tells machines what a page is about. For example, it can say who wrote an article, when it was updated, what a product costs, or which questions a page answers. Use JSON-LD, the format Google recommends.
The seven types that matter most for AI citations are:
- Organization
- Article (with author and dates)
- FAQPage
- Product or Service
- Person (for authors)
- BreadcrumbList
- LocalBusiness (if you have a physical location)
The median site in our audits uses only three of these seven.
Factor 6: Headings match real questions, and data sits in tables
Write headings as people ask questions, such as “How much does AEO cost?” rather than “Pricing overview”. This matches the sub-questions that fan-out creates.
Put comparisons, prices and specifications in HTML tables and lists, not in images. AI engines read tables well, and they often reuse the data directly in their answers.
Group 3: Entity factors (does AI know who you are?)
Entity factors decide whether AI engines recognise your brand as a distinct, real organisation, with clear facts attached to it. AI models think in entities: brands, people, products and places. If your entity is unclear, the engine may confuse you with another company or leave you out.
Factor 7: Your brand facts are the same everywhere
AI engines compare what different sources say about you. If your website, LinkedIn page, Google Business Profile and directory listings disagree, the engine has less reason to trust any of them.
Check that these match everywhere:
- Company name and short description
- Founding year, headquarters and locations
- Services or product categories
- Contact email and phone number
- Founder and leadership names
Add an Organization schema with sameAs links to your official profiles. This ties them together as one entity. Where possible, create or update your Wikidata entry.
Factor 8: An AI Instructions page states the facts in one place
An AI Instructions page is a public page written for AI agents. It sets out your brand’s verified facts: who you are, what you offer, where you operate and how you want to be described. It gives engines one clear, citable source, so they don’t have to piece the facts together from many pages.
Fewer than 1% of the sites we audited have one. That makes it one of the cheapest ways to stand out.
Factor 9: Content has named authors with real expertise
AI engines and Google both favour content from people with clear experience in the subject. This is the idea behind Google’s E-E-A-T guidelines, which stand for experience, expertise, authoritativeness and trustworthiness.
For each article:
- Show the author’s name, role and photo.
- Link to an author page with their credentials and LinkedIn profile.
- Add Person schema to that author page.
- Show the date the article was published and last updated.
Group 4: Authority factors (do others vouch for you?)
Authority factors decide whether AI engines trust your brand enough to recommend it. What others say about you counts for more than what you say about yourself. When engines list the “best” options in a category, they lean on independent sources.
Factor 10: Trusted third-party sites mention your brand
The brands that appear in AI answers are usually the brands that appear across the web. That includes industry publications, news sites, comparison articles and “best of” lists. Unlinked mentions count too, because language models learn from how often and in what context a brand is named.
Ways to earn these mentions:
- Publish original data that journalists and bloggers want to quote.
- Offer expert commentary to industry media.
- Get listed in relevant comparison and “top tools” articles.
- Appear on podcasts, webinars and conference agendas.
Factor 11: Your brand shows up in community and review platforms
AI engines often cite Reddit threads, Quora answers, YouTube videos and review sites such as G2, Clutch and Google reviews. These platforms show what real users think, so engines treat them as evidence.
You cannot fake this, and you should not try. Instead, take part in genuine discussions in your category. Ask happy customers for reviews, and publish helpful videos that answer common questions.
Factor 12: You cover your topic in depth, and keep it fresh
One good article rarely makes a brand an authority. AI engines and Google trust sites that cover a subject from every angle. That means definitions, how-to guides, comparisons, pricing and case studies, all linked together. This is called topical authority. A useful target is 25 to 50 connected pages on your core subject.
Freshness matters as well. Engines prefer recent sources for questions about prices, tools and trends. Review your key pages every quarter. Update the facts, and change the “last updated” date only when you have made real changes.
Google SEO vs AI search: what changes?
Most of the foundations are the same. Good SEO is still the entry ticket for AI search, especially for Google AI Overviews and AI Mode, which draw on Google’s own index. What changes is the goal. SEO aims for a high ranking and a click. AI search aims to be quoted and named in the answer itself.
| Factor | Google SEO | AI search | How to check it |
|---|---|---|---|
| Crawler access | Googlebot only | Many crawlers: GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot | Review robots.txt and CDN bot rules |
| JavaScript rendering | Google renders most JavaScript | Many AI crawlers do not | View the page with JavaScript turned off |
| Backlinks | Strong ranking signal | Helpful, but brand mentions matter as much | Track mentions as well as links |
| Keyword targeting | One main keyword per page | Many sub-questions per topic | List the questions buyers ask |
| Content format | Long, complete pages | Clear answer blocks that can be quoted | Read the first 60 words of each section |
| Schema | Unlocks rich results | Helps engines understand the page and the brand | Test with a schema validator |
| Brand consistency | Minor factor | Major factor | Compare your facts across profiles |
| Success metric | Rankings, clicks, traffic | Citations, mentions, share of voice | Track a fixed set of prompts each month |
In short: do not run SEO and AEO as two separate projects. Build one strong technical and content foundation, then add the AI-specific steps on top.
What matters less than people think
Some old SEO habits do little for AI citations. Here are four to stop spending time on:
- Keyword density. AI engines look for meaning, not for how often a keyword repeats. A page that answers the question clearly will beat one that repeats a phrase 20 times.
- Word count by itself. Longer is not better. A 900-word page with sharp answers can be cited more often than a 4,000-word page full of filler.
- Exact-match domains. A domain such as best-crm-india.com carries little weight. Engines care about who you are, not what your URL says.
- AI-generated content at scale. Publishing hundreds of thin, similar pages adds no new information. Engines have no reason to cite a page that says what a thousand others already say. Use AI tools to support your writers, not to replace expertise.
How to audit your site against the 12 factors
You can run this check in about an hour. Work through the list in order, because each group depends on the one before it.
Access
- Does robots.txt allow OAI-SearchBot, PerplexityBot, ClaudeBot and Googlebot?
- Does your main content appear with JavaScript turned off, and does the server respond in under a second?
- Do you have an XML sitemap and an llms.txt file at the root domain?
Structure
- Does every key section open with a 40- to 60-word answer?
- Do your pages carry Organization, Article and FAQPage schema?
- Are your headings written as questions, with data in HTML tables?
Entity
- Do your company facts match on your website, LinkedIn, Google Business Profile and directories?
- Do you have an AI Instructions page?
- Does each article show a named author, an author page and an updated date?
Authority
- Do trusted industry sites mention your brand?
- Are you present on review sites and in community discussions in your category?
- Do you have at least 25 linked pages on your core topic, reviewed in the last three months?
To test the technical factors automatically, run the free AI Agent Readiness Check. It scans your site against more than 50 signals in six categories, then shows what to fix first.
Then test the results directly. Ask ChatGPT, Perplexity and Google AI Mode ten questions your buyers would ask. Note which brands and sources appear. Repeat the same questions every month to track your progress.
Frequently asked questions
What are the most important ranking factors for AI search?
Crawler access comes first. If AI crawlers cannot reach and read your pages, nothing else counts. After that, the biggest factors are clear answer-first content, consistent brand facts across the web, and mentions on trusted third-party sites. Schema markup and topical depth support all of these.
Is ranking in AI search different from ranking in Google?
Partly. The technical and content foundations overlap a lot, and strong Google rankings help with AI Overviews and AI Mode. The differences are more crawlers to allow, more weight on brand mentions and consistency, and a different goal. In AI search, you want to be quoted in the answer rather than clicked from a list.
Does schema markup help with ChatGPT citations?
Schema helps AI engines understand what a page is and who is behind it. That makes your content easier to trust and quote. It is not a shortcut on its own, however. A page with perfect schema but vague content will still lose to a page that answers the question clearly.
How long does it take to get cited by AI engines?
Technical fixes, such as unblocking crawlers, can take effect within weeks, once engines re-crawl your pages. Citations that depend on brand mentions and topical authority usually take three to six months to build. Training-data visibility takes longer still.
Do backlinks still matter for AI search?
Yes, but not in the same way. Links from respected sites still signal trust, and they help your pages rank in the search results that AI engines draw from. Unlinked brand mentions also matter in AI search, because language models learn from how often and where your brand is named.
The bottom line
AI search rewards brands that are easy to reach, easy to understand, easy to recognise and widely trusted. Start with access, because it is the cheapest fix and the most common failure. Then work through structure, entity and authority, in that order.
Once the work is under way, you will need to show it is paying off. Here is how to measure AEO ROI.
If you want a team to do this with you, Spacemen’s answer engine optimisation (AEO) services cover all four groups. We work across ChatGPT, Gemini, Perplexity, Claude and Google AI Overviews, and we report on citations, share of voice and pipeline.