Everyone selling AEO has a theory. We wrote down a baseline first.
AEO is sold by people with no measurement. Here is the baseline we ran on our own site, and the second measurement we found that costs nothing.
On this page
- Why do you need a baseline before you change anything?
- What did six prompts tell us?
- Is this prompt worth chasing?
- What was the uncomfortable result?
- Can you measure whether AI assistants read your site?
- What did we actually change?
- What can we not claim?
- How do you run this yourself?
- 1. Write six to twelve prompts a real buyer would type
- 2. Ask them signed out, today, before you change anything
- 3. Record three things per prompt
- 4. Pull your AI crawler traffic for the last week
- 5. Make your changes, and write down what and when
- 6. Ask the same prompts again in three months, and append
- Common questions about answer engine optimisation
- Will you be able to tell in year two?
Nobody selling answer engine optimisation publishes a measurement. So before changing anything on our own site, we wrote down what the assistants already said about our market, signed out, in a form we could repeat. This is that baseline, what six prompts told us, and the second measurement we have since found that costs nothing.
Key takeaways
- Ask an assistant how to get cited and every source it quotes is a company selling AEO software. Nobody publishes a measurement.
- Record a baseline before you change anything. Search Console cannot tell you an assistant quoted you, so if you skip this you can never prove the work did something.
- Sorted by question shape rather than topic, three patterns appear. Only one of them is winnable, and it is the one small agencies with no authority are already winning.
- You can now measure more than we could a month ago. Your CDN can tell you when an assistant fetched a page to answer someone's question. Ours saw about 220 of those in a week, before the blog was public.
- Treat llms.txt as cheap housekeeping, not a lever. Google has said it does not use it.
Ask Perplexity how to get your business mentioned by ChatGPT, and it will tell you. It will also show you where the answer came from. When we asked, in August 2026, the sources were dekai.ai, rankprompt.com, botric.ai, qbizleadsai.com and aioclicks.
Every one of them sells answer-engine-optimisation software.
That is not a conspiracy, it is a vacuum. The people with the strongest incentive to publish about AEO are the people selling AEO tools, so they have published more than anyone else and the assistants have nothing else to draw on. What is missing is anybody showing their working: a before, a change, an after, and an honest account of which parts cannot be attributed.
This is our attempt at the first half of that, from an SEO agency that would rather publish a measurement than a theory. We have not been cited yet. We can tell you exactly how we will know when we are, and since the first version of this article we have found a second way to know that costs nothing.
The short answer: write down what the assistants say about your market today, signed out, in a form you can repeat, before you change anything. Chase the questions shaped "should I choose A or B", because those are being answered from small blogs with no authority. Ignore "who is the best X", which belongs to exact-match domains. Then watch your own traffic for the AI agents that fetch a page on a person's behalf, because that is the citation pathway and it is now visible.
Jump to:
- What six prompts told us
- Check whether a prompt is worth chasing
- How to measure whether assistants read your site
- How to run this yourself
- Common questions
Why do you need a baseline before you change anything?
Because there is no equivalent of Search Console for assistants, and once you have made changes you can never go back and find out what they said before. Here is the trap. You read about AEO, you make a set of changes, and three months later somebody asks whether it worked. You have no idea. Nothing tells you ChatGPT quoted you. Nothing tells you it stopped.
So the first piece of work is not a change at all. It is writing down what the assistants say today, before you touch anything, in a form you can repeat.
Ours took under an hour:
- A fixed set of prompts, phrased the way a buyer would ask, not as keywords. "Should I use an AI website builder or hire an agency" rather than "ai website builder".
- Signed out. A logged-in session personalises the answer, which makes your result unrepeatable by anyone else, including future you.
- Three fields recorded per prompt. Are we named at all, which businesses are named, and which domains are cited.
- Appended, never overwritten. The comparison is the entire value, so editing the baseline destroys the thing you made it for.
That last point sounds pedantic until the first re-run, when the temptation to tidy the old numbers is strong.
What did six prompts tell us?
That the question shape matters more than the topic. We expected to learn where we stood. We learned something more useful.
Sorted that way, the results split cleanly.
| Question shape | Who gets cited | Worth chasing |
|---|---|---|
| "Who is the best X in Y" | Exact-match domains, directories, listicles | No |
| "How do I fix X" | Vendors selling the fix | Rarely |
| "Should I choose A or B" | Small agency blogs, no authority required | Yes |
The first row is a dead end. Asked which Australian agencies build AI agents, the assistant named five businesses, and every single one held an exact-match domain. That is not an assessment of capability, it is pattern matching on domain names. You cannot compete with it and you should not try.
The second row belongs to vendors. Asked how to stop an AI support bot giving wrong answers, the sources were Gorgias and eGain. Both sell customer service software. They publish more about the problem than anyone who has merely solved it once.
The third row is wide open, and this is the finding worth acting on. Asked whether to use an AI website builder or hire an agency, the assistant cited seven sources. Six were small design agencies. One was Australian. The seventh was an AI website builder citing itself.
None of those six has meaningful domain authority. They were cited because they had written something specific about the exact decision being asked about. That is the pattern, and it brings us to the result that stung.
Is this prompt worth chasing?
Three questions about a prompt you want to be cited for. The answer lands as soon as your answers settle it, on the same logic as the table above.
Start at the top. The recommendation lands here as soon as your answers settle it.
Chase it
Recommended when it chooses between options.
This is the shape small sites are already winning, with no authority behind them. Write the article that answers exactly this decision: state the scope in the first lines, lead each section with the answer, give the trade-off, and name the products. Then record the prompt in your baseline today, signed out, so you can prove the article did something.
What would change this: the prompt drifting towards "the best" as you phrase it. "Should I choose A or B" is winnable; "which agency should I choose" is not.
Rarely worth chasing
Recommended when it asks how to fix or do something.
The sources for a how-to are the vendors selling the fix, because they publish more about the problem than anyone who has merely solved it. You can take one from them when you have done the fix yourself and can show the working: the before, the change, the after and the number. Without that, let it go.
What would change this: a specific, measured fix you have shipped. A real number from a real build is the one thing a vendor cannot copy.
Not worth chasing
Recommended when it asks who is the best, or names a place.
Assistants answer this shape by pattern-matching on domain names and directories: every business named in our test held an exact-match domain. That is not an assessment of anything, and you cannot write your way into it. Keep the prompt in the baseline as a control you expect to lose, and spend the writing on a decision question instead.
What would change this: you holding the exact-match domain. Then it is already yours, and the prompt is a measurement rather than a target.
Reshape it first
Recommended when it is none of the three shapes.
A prompt that is none of the three shapes is usually a keyword wearing a question mark: "AI customer service", "headless CMS Australia". Assistants answer the question behind it, so ask that instead. Rewrite it the way a buyer would say it out loud to a colleague, and it will land in one of the three rows.
What would change this: it already being a buyer's sentence. If a real person would type it as written, run it through the questions again.
What was the uncomfortable result?
The sharpest one was about us. We asked: "Webflow vs WordPress vs headless CMS for a marketing team that edits its own pages."
That is, almost word for word, the argument of an article we had already written. Its whole thesis is that you should choose a CMS by who edits it rather than by feature list. Perplexity gave essentially that answer, and cited two small agency blogs for it.
Ours was not cited, because at the time of the test our blog was not public. The article that should own that question already existed and was invisible.
Publishing is not the last step of the work. It is the step that makes the rest of it count.
A drafted, reviewed, technically immaculate article behind a gate is worth exactly as much as no article. We have since seen the same thing from the other side, in the data below: assistants asking for pages of ours that were not public, and getting nothing.
Can you measure whether AI assistants read your site?
More than we could a month ago, and without buying anything. The baseline above tells you what an assistant says. Your own server logs can tell you when an assistant came to look, and the two are different measurements of the same thing.
The key is that the AI companies now send different crawlers for different jobs, and they identify themselves. One kind indexes or trains: GPTBot, ClaudeBot, PerplexityBot, Google-Extended. Another kind fetches a page live, on behalf of a person who just asked a question: ChatGPT-User, Perplexity-User, Claude-User. That second kind is the one that matters, because it only fires when somebody's question led the assistant to open your page. It is the citation pathway, visible in your traffic.
Here is ours, for the seven days to 6 September 2026, on the public site only and before the blog was public. Requests identified by user agent, so read them as "presenting as" rather than verified:
| Crawler | Requests in 7 days | What it is |
|---|---|---|
| Bingbot | 246 | Bing's index, which ChatGPT search and Copilot retrieve from |
| ClaudeBot | 214 | Anthropic's crawl |
| OAI-SearchBot | 165 | ChatGPT search's index |
| ChatGPT-User | 133 | A live fetch to answer a person's question |
| Claude-User | 125 | A live fetch for a Claude user |
| GPTBot | 98 | OpenAI training |
| Perplexity-User | 90 | A live fetch for a Perplexity answer |
| PerplexityBot | 50 | Perplexity's index |
| Claude-SearchBot | 48 | Claude search's index |
| Google-Extended | 46 | Gemini training |
About 220 live fetches in a week. Each one is a person, somewhere, asking an assistant something that made it open one of our pages. We cannot see what they asked or whether we were quoted in the answer. We can see that the retrieval happened, which is a great deal more than nothing, and it is the number to watch after a change.
Our CDN now shows the same thing as a panel, and it added one detail we could not have got any other way: a list of pages assistants tried to fetch and could not. Two case studies were on it. Both were finished, reviewed and gated pending sign-off, and both returned 404 to an assistant that had a question they answered. The uncomfortable result, again, with a timestamp.
One caution on reading it. A fetch is not a citation. An assistant can open your page and quote someone else. The baseline prompts tell you the second part; the traffic tells you the first. You want both, and the traffic is the one that updates daily.
What did we actually change?
Nothing exotic, and we would rather describe it plainly than sell it. Here is the work itself, including what we added after the first version of this article.
Every AI crawler is allowed, at every layer. Our robots.txt does not block GPTBot, ClaudeBot, PerplexityBot or the rest, and neither does the CDN in front of the site, which we checked rather than assumed: a proxy can block a crawler that robots.txt allows, and a user-agent test proves nothing about that. This is a genuine trade-off and reasonable people land differently. Allowing them means your words can be reproduced without a click. We decided that being absent from the answer is worse than being quoted in it. If your business sells the words themselves, decide the other way.
Content signals say so in writing. One line in robots.txt now states that search, AI input and AI training are all permitted. It is advisory rather than enforced, and it is one of the five things a CDN's agent-readiness check looks for. We chose all three deliberately: it is marketing copy, and it is meant to travel.
Every page has a markdown version on request. An assistant that asks for text instead of HTML gets the article stripped of navigation and scripts. For the articles we build that version ourselves, because it can carry things a conversion cannot know: who wrote it and why to believe them, and a decision tool as a plain list an agent can run. For everything else the CDN converts on the fly. The homepage went from 502KB of HTML to 27KB of text.
FAQ blocks are derived from what the page actually shows. Our FAQ markup is generated from questions rendered in the body, never from a separate hidden field. Partly that is Google's structured data policy, which requires the marked-up content to be visible. Mostly it is that the two cannot then drift apart, which is how most FAQ markup rots.
Author identity is wired up, in the text as well as the markup. The byline carries a job title and a link to a real profile in the structured data, and the markdown version closes with the author and the business in prose, because a model quotes sentences rather than schema. A name on its own asserts nothing.
The entity facts agree everywhere. Address, phone and business details match across the site, the schema, and the Google Business Profile. This one is not guesswork: when we asked how assistants decide who to recommend, the answer we got back was that they synthesise from how consistently a business is described across credible sources. Consistency is the mechanism, so contradicting yourself across surfaces is a real cost.
Bing is told when pages change, and ChatGPT is a separate job. Microsoft Copilot retrieves from Bing, so a site Bing has never crawled is invisible there, and most Australian sites have never checked. ChatGPT search is not the same pipe. OpenAI crawls for it with its own OAI-SearchBot, and OpenAI's documentation says a site opted out of that bot will not be shown in ChatGPT search answers. So the job is to allow both, then tell them. Our IndexNow key is live, and Cloudflare's Crawler Hints is switched on, which is meant to announce changed pages to IndexNow for the whole zone. It is a setting rather than a project.
We published an llms.txt, and you should not expect much from it. It is a plain text file describing the business for AI systems. Google has said it does not use it. No major crawler has confirmed it drives retrieval. We published one because it is cheap, correct and easy to keep accurate. Anyone presenting it as a ranking factor is guessing, and you can tell because they never show a measurement.
What can we not claim?
That we have been cited. Not once, in any of the six prompts. The 220 live fetches say assistants are reading; they do not say assistants are quoting.
That is the honest state, and it would be easy to hide it by publishing this in six months with a more flattering number at the top. The reason to publish now is that the baseline is only credible if it goes out before the result is known.
Two limits worth stating plainly. Six prompts is a small sample, enough to show a pattern and not enough to prove a trend. And we tested one assistant. Perplexity shows its sources, which is what made it useful. ChatGPT and Google's AI Overviews behave differently and would need their own baselines. The crawler counts have a third limit: a user agent is a claim, not a proof, and a busy scraper can wear any name it likes.
How do you run this yourself?
With a browser, a text file and your server logs. You do not need a tool for this, and the tools will not tell you anything you cannot see directly.
1. Write six to twelve prompts a real buyer would type
Phrase them as decisions, not keywords, and include a couple you expect to lose as a control. "Should we use Webflow or WordPress for a site marketing edits" is a prompt. "Best CMS" is not.
2. Ask them signed out, today, before you change anything
This is the step almost nobody does, and it is why almost nobody can tell you whether any of this works, including the companies selling it. A logged-in answer is personalised to you and cannot be repeated.
3. Record three things per prompt
Are you named. Who is named. What is cited. Append the date. Never edit an old entry.
4. Pull your AI crawler traffic for the last week
Group requests by user agent and separate the live fetchers, the ones with "User" in the name, from the index and training crawlers. Some CDNs show this now; ours has a panel for it. If the number is zero, check that nothing between the crawler and your server is blocking it before you change a word of content.
5. Make your changes, and write down what and when
The content changes from the section above, in the order that costs least: allow the crawlers, fix the entity facts, add real FAQs, wire up the author, publish what is sitting in draft.
6. Ask the same prompts again in three months, and append
Compare the two, and compare the weekly fetch counts across the same period. One of those will move first, and it is usually the traffic. That is your early signal, and the prompts are your proof.
Common questions about answer engine optimisation
What is answer engine optimisation, in plain terms?
It is the practice of making your content easy for an AI assistant to find, understand and quote when it answers a question. The difference from traditional SEO is that there is often no click: the assistant reads your page and summarises it, so the win is being the source rather than the destination.
Is AEO different from SEO, or just a new name?
The technical foundations overlap almost entirely: crawlable pages, clean structure, accurate schema, real headings. What changes is the target. Traditional SEO wants a ranking position. AEO wants to be the passage an assistant reproduces, which rewards being specific and checkable rather than broadly relevant.
Does llms.txt help?
There is no public evidence that it does. Google has said it does not use it, and no major AI crawler has confirmed it affects retrieval. It costs almost nothing to publish and keep accurate, so it is reasonable housekeeping. Treat anyone calling it a ranking factor with suspicion.
Should I block AI crawlers instead?
It depends on what you sell. If your content is the product, blocking is defensible. If your content is marketing for something else, being absent from AI answers is usually the worse outcome. It is a business decision rather than a technical one, and it should be made deliberately rather than by leaving a default in place. Check the CDN as well as robots.txt; the two can disagree.
How long before I see anything?
Unknown for citations, and be suspicious of anyone who gives you a confident number. Assistants refresh their sources on their own schedules and none of them publish it. The crawler traffic moves faster: a change to what is crawlable shows up in a week. That is precisely why you need both measurements: without them you cannot tell a slow result from no result.
Do I need an AEO tool?
Not to start. Everything described here was done with a browser, a text file and the server logs we already had. Tools may help at scale, but the first useful thing is a recorded baseline, and no tool is required for that.
How do I know if ChatGPT is citing my website?
Directly, you mostly cannot, which is the whole reason for the baseline. Indirectly, two signals: prompts you ask signed out and record over time, and requests in your logs from ChatGPT-User, which only fires when a person's question led ChatGPT to open your page. The first tells you if you were quoted. The second tells you if you were read.
What is the difference between GPTBot and ChatGPT-User?
GPTBot crawls to build training data and OAI-SearchBot crawls to build the search index; both visit on their own schedule. ChatGPT-User fetches a page live because a person asked something and the assistant decided your page might answer it. The first two tell you that you are known. The third tells you that you were useful to somebody today.
Which AI crawlers should I allow?
If the goal is to be cited, all of them, and check it at the proxy as well as in robots.txt. If you want to be found but not trained on, Content Signals let you say search and AI input yes, training no, though it is advisory. Blocking the live fetchers, the ones with User in the name, is the one choice that guarantees you will never be cited.
Does Bing matter for AI search?
More than its search share suggests, though not for the reason usually given. Microsoft Copilot retrieves from Bing, so a site Bing has never crawled is invisible there. ChatGPT search is separate: OpenAI crawls for it with OAI-SearchBot, and OpenAI's own documentation says a site opted out of that bot will not be shown in ChatGPT search answers. So do both. Verify with Bing Webmaster Tools and submit a sitemap, then check your robots.txt allows OAI-SearchBot.
Do FAQ blocks help with AI answers?
Yes, when the questions are ones people actually ask and the answers stand on their own. Assistants lift passages, not pages, so a one-paragraph answer under a real question is the shape they can use. Generate the markup from what is on the page, never from a hidden field, or the two drift and the markup becomes a lie.
Should I write for ChatGPT, Perplexity or Google AI Overviews?
Write for the buyer's question and all three can use it. They differ in where they retrieve from: Perplexity and ChatGPT search lean on their own crawls and on Bing, Google's overviews on Google's index. The content work is the same. The measurement differs, which is why each needs its own baseline if you want proof.
What kind of content gets cited by AI assistants?
In our six prompts, content that answered a specific decision the asker was making, from a site with no particular authority. "Should I choose A or B" articles from small agencies beat the vendors and the directories on that question shape. Specific and checkable beat broad and authoritative. That is the finding, and it is the one worth acting on.
Can I track AI referrals in Google Analytics?
Partly. When an assistant links to you and a person clicks, the referrer often identifies it and analytics records a session. What analytics cannot see is the far larger number of times an assistant read your page and answered without a click. For that you need the server logs or a CDN that shows crawler traffic by agent.
Is there an AEO equivalent of Search Console?
Not from the assistants themselves. The closest thing is your own CDN's crawler reporting, which some now present as a panel of AI retrievals and even the pages assistants tried to fetch and could not. Ours listed two pages that were finished and unpublished. Nothing else would have told us that.
How many prompts do I need for a useful baseline?
Six showed us a pattern. Twelve would show it more convincingly, and neither proves a trend. Pick prompts that span the three shapes, include a couple you expect to lose, and resist adding new ones between runs. A baseline that changes shape is two baselines.
Will you be able to tell in year two?
Set this up so that in year two, when someone asks whether the AI search work did anything, you can answer with two dates and two numbers rather than a feeling.
Every AEO pitch shows the same thing: a screenshot of an assistant naming a client, and no screenshot of the week before. None of them show the baseline, because most of them never took one.
That is the whole difference. Write down what the assistants say today. Watch who comes to read. Publish what you have drafted, because the one question we were best placed to own was being answered by someone else while our answer sat behind a gate. Then ask again in three months and append. It is the same discipline we bring to every search engine optimisation engagement, and the reason we can show working instead of a theory.