FIELD NOTE

The loop that makes a site look like an AI winner

Searchdom's blunt characterization: if a vendor can show you a long list of AI citations with no explanation of how the prompts were built, there is a 90 percent chance the report is circular.

Most AEO/AIO reports do not test how a real buyer would ask an AI assistant. They test how well a prompt written from the client's own website matches the client's own website.

The method is simple. The vendor scrapes the client site. It pulls page titles, service pages, location pages, case studies, FAQs, and differentiators. Then an LLM generates hyper-specific questions that mirror that content. A prompt such as “Who is the top commercial HVAC contractor on 4th street in Austin specializing in VRF systems?” is not a consumer query. It is a description of a website, rewritten as a question.

Then the vendor feeds those prompts into ChatGPT, Perplexity, Claude, Copilot, or another AI answer engine. The engine often cites the site because the prompt contains the exact niche, street, service, and specialization that the site was built to answer. The report then labels that citation as rank #1 in AI search.

That is not market share. It is prompt alignment.

Why the trick works

The trick works because AI search is non-deterministic and not reportable like Google Search Console.

Google has a structured reporting layer: query impressions, clicks, positions, and devices. AI answer engines do not give agencies the same window into what users actually asked. There is no public reporting API that tells you, “Here are the real user prompts that led to citations of your client.”

So vendors replace missing data with generated prompts. They create the demand, test against it, and report the result as demand.

The same prompt can also produce different answers depending on user history, location, model updates, retrieval changes, tool settings, and whether the assistant is browsing. A fixed “AI rank” for an auto-generated prompt is therefore unstable. It measures prompt-to-content fit, not consumer intent.

The HVAC example makes the problem obvious. If the prompt names a street, a service, and a system type, it will usually point to the shop that matches that narrow description. If zero consumers ask that question, the ranking is vanity. It proves the website can be cited when the question is engineered from the website. It does not prove buyers are asking the question.

What vendors do not say

Most vendors do not disclose this weakness. They present the output as evidence of AI visibility. They may say they “test prompts,” “track citations,” or “monitor AI rankings.” They rarely explain:

• Where the prompts came from.
• Whether the prompts were generated from the client site.
• Whether the prompts reflect real user language.
• Whether the same prompt was tested repeatedly.
• Whether the result was compared against referral traffic.
• Whether the test controlled for location, user context, and model version.

That omission matters. A report is only useful if the test is independent of the thing it claims to measure. If the prompt list is built from the client’s own pages, the test is circular.

Searchdom’s position is that no vendor has secret access to private LLM search logs. No agency can claim a complete AI search ranking unless it has access to the underlying query data. Today, that access does not exist in the same way Search Console exists for Google.

What is actually verifiable today

Agencies can still measure AI visibility, but the bar has to be higher.

First, check referral traffic in analytics. Look for referrers such as chatgpt.com, perplexity.ai, claude.ai, copilot.microsoft.com, and other AI answer domains. This is real user behavior, not a generated test. If a client claims strong AI citation performance but has no corresponding referral traffic, the claim needs scrutiny.

Second, manually test brand visibility against broad, realistic industry prompts. Do not use prompts that repeat the client’s page titles. Ask the way a buyer would: “What are the best commercial HVAC contractors in Austin?” or “How do we choose a VRF installer for a mid-rise office building?” Then compare the client against competitors. If the site only appears when the prompt is a mirror of its own service page, the evidence is weak.

Third, audit the sources LLMs pull from. AI assistants may cite Reddit threads, Wikipedia pages, trade publications, industry roundups, government pages, and reputable directories. If a client is not present in those source ecosystems, AI visibility will be fragile. The goal is not to chase a single generated prompt. The goal is to make the brand visible in the material the model can retrieve.

The honest test

The honest test is simple. Ask the provider how the prompt list was built.

If the answer is “we generated prompts from the client website,” the report is a self-test. If the answer is “we use a curated list of buyer questions,” ask for the list and compare it against real traffic, real buyer language, and competitive results. Then cross-reference the claims against actual referral traffic and source visibility.

Searchdom engages only in real, verifiable data. We tell clients which metrics are real, which are directional, and which are invented. We do not dress up prompt alignment as AI market share.

Takeaway: Most AEO/AIO reports are fake because they generate the question from the client’s own website, then report the citation as proof of demand. The real test is not whether a site can win a self-made prompt. It is whether buyers are asking those questions, whether AI referrals show up in analytics, and whether the brand is visible in the sources AI tools actually use.