In an era where generative artificial intelligence is rapidly reshaping the digital discovery landscape, the ability to quantify what actually influences large language models (LLMs) has become the new frontier of search engine optimization. During a recent industry briefing hosted by Search Engine Journal, executives and product leads from the enterprise SEO platform seoClarity revealed groundbreaking data from a series of controlled split tests conducted across multiple AI platforms, including ChatGPT, Claude, Perplexity, Gemini, and Google’s AI Overviews. The central finding of the research—that adding and then removing FAQ sections directly correlates with the rise and fall of AI citations—marks one of the first documented instances of proven causation in the burgeoning field of AI Engine Optimization (AEO).
The webinar featured insights from Mark Traphagen, Vice President of Product Marketing and Training; Mihir Naik, Senior Product Manager for AI; and Suraj Lalchandani, Senior IT Project Manager. Together, the team argued that the industry must move beyond "visibility scores," which merely indicate presence, and toward page-level performance and rigorous split testing to determine which optimizations actually drive results. This shift in methodology comes at a critical time as enterprises struggle to justify investments in AI-specific content strategies without clear evidence of their efficacy.
The FAQ Experiment: Distinguishing Causation from Correlation
The cornerstone of the seoClarity presentation was a specific experiment involving the implementation of FAQ sections on a set of test pages. While many SEO professionals have suspected that structured data and clear question-and-answer formats assist AI models in processing information, the seoClarity team sought a higher standard of proof. They utilized a "reversion" testing model to isolate the variable of FAQ content.
In the first phase of the test, FAQ sections were added to a group of target pages. The researchers observed a significant lift in citations across various AI search engines. However, to rule out the possibility that the lift was caused by a general model update or external factors, the team performed a second phase: they removed the FAQ sections entirely. Following this removal, the citation rates dropped back to their original baseline.
"That reversion is the difference between correlation and causation," the team noted during the session. While many SEO tools can show a correlation between a change and a ranking shift, almost no teams measuring AI search today can produce the level of proof provided by a successful reversion test. This specific outcome demonstrated that the AI models were actively utilizing the FAQ content to generate their responses and were citing the source specifically because of that structural addition.
The Evolution of Measurement: Google Search Console’s AI Reports
A major turning point in the timeline of AI search measurement occurred on June 3, 2024, when Google officially launched dedicated Search Console reports for AI Overviews and AI Mode. This update allows site owners to see, on a page-by-page basis, how often their URLs appear within Google’s AI-generated search features.
Suraj Lalchandani characterized this as the most significant measurement upgrade in the history of AI search testing. Prior to this release, digital marketers were forced to rely on third-party sampling and inferential data to guess their visibility in Google’s AI features. "Everyone was sampling. Everyone was inferring. But now Google is just giving it to you," Lalchandani remarked.
Despite the breakthrough of first-party data from Google, the seoClarity team emphasized that these reports are not a complete solution. While Google Search Console provides a high level of trust for Google’s own ecosystem, it does not provide insight into how a brand is performing on "closed" LLMs like OpenAI’s ChatGPT or Anthropic’s Claude. For these platforms, structured third-party tracking remains a necessity. The team provided a platform-by-platform reference guide during the session, mapping out exactly which gaps the new Google reports close and which remain open for third-party tools to fill.
The "Golden Set" of Prompts: A Strategic Framework for Testing
To run an effective AI testing program, the seoClarity team advocates for the creation of a "golden set" of prompts. This set is designed to span the entire marketing funnel, from initial awareness and discovery through to consideration and retention. Each prompt is tagged by its stage in the buyer’s journey, allowing brands to see where they are winning and where they are invisible.
The methodology involves sorting these prompts into tiers based on current performance:
- Tier 1 Prompts: These are identified as "easy wins." In these instances, the brand is already relevant to the query, but the AI has not yet been provided with a specific URL that it deems worth linking to. Small structural changes, such as adding a direct answer or a clear linkable asset, often result in immediate citations.
- Tier 2 Prompts: These represent a "heavier lift," where the brand may not be viewed as a primary authority for the topic, requiring more substantial content development and authority-building efforts.
- Excluded Prompts: Surprisingly, the team revealed that they often drop certain buckets of prompts from testing entirely. This is a strategic move to focus resources on queries where the brand has a realistic chance of movement, rather than wasting effort on prompts that are fundamentally misaligned with the brand’s core offerings.
The sequencing of these tests is deliberate. By targeting Tier 1 "easy wins" first, SEO teams can generate early success stories. This build-up of "political capital" within an organization is often necessary to secure the budget and executive buy-in required for more complex, long-term AI optimization tests.
Overcoming the Technical Challenges of LLM Split Testing
One of the most significant hurdles in AI search optimization is the inability to run traditional A/B tests. In a standard web environment, a developer can split live traffic 50-50 between two versions of a page. However, because LLMs crawl and "learn" from content rather than serving it in a traditional request-response cycle, this is impossible.
To circumvent this, the seoClarity methodology relies on the construction of a control group consisting of correlated pages. This group acts as a "noise filter" against the unpredictable nature of model updates and algorithmic shifts. Without a control group, any increase in citations could be attributed to a general improvement in the AI model’s capabilities rather than the specific optimizations performed on the website.
Timing is another critical discipline. The team established a strict baseline period before any changes are made, followed by a minimum test window after implementation. Because AI search engines do not always respond to content changes overnight, cutting the test window short can lead to "reading noise" rather than meaningful data. Every test typically results in one of three outcomes: a positive lift, no change, or a negative impact. According to Mihir Naik, every result is a win because it provides empirical evidence that replaces guesswork.
The ROI of Citations in a Zero-Click Environment
A recurring concern among digital marketers is the Return on Investment (ROI) of an AI citation that does not result in a direct click to the website. As AI engines provide more comprehensive answers within their own interfaces, the "zero-click" phenomenon is expected to accelerate.
Mihir Naik addressed this by shifting the focus from traffic to "narrative control." When a brand is cited by an AI, it gains the ability to shape the answer provided to the user. This is particularly vital in comparison queries—such as "Brand A vs. Brand B"—where the AI’s summary can dictate the user’s perception of a brand’s value proposition. If a brand is not cited, the AI may rely on third-party reviews or outdated information to describe the company’s services.
"You want to be cited because you are controlling the answer that is actually going to be showing up," Naik explained. Being the source of the data allows a brand to ensure its Unique Selling Propositions (USPs) are highlighted correctly and that any inaccuracies in the AI’s understanding are corrected at the source.
The Foundational Role of Traditional SEO
Despite the focus on new AI-driven tactics, the webinar participants were unanimous in their belief that traditional SEO remains the foundation of AI findability. Mark Traphagen noted that seoClarity’s most successful clients in the AI search space are those who have maintained technically healthy websites and well-optimized content for years.
The AI bots used by Google, OpenAI, and others still rely on the ability to crawl and render content effectively. If a site has technical barriers, such as poor site speed, broken links, or content hidden behind complex JavaScript, AI engines will struggle to process the information. Suraj Lalchandani added that in their extensive testing, they have rarely found a situation where a tactic that is good for traditional SEO is bad for AI search.
One specific technical area discussed was the use of collapsible content, such as "accordion" style FAQs. The team warned that implementation matters: if the content is hidden in a way that requires a user click to be rendered in the DOM (Document Object Model), AI bots—much like traditional Google crawlers—may not "click" to see it. Ensuring that content is present in the HTML and readable upon initial crawl is essential for both SEO and AEO success.
Implications for the Future of Search
The research presented by seoClarity suggests a maturation of the AI search industry. The transition from "guessing what works" to "proving what works" through scientific split testing marks a significant milestone for enterprise marketing teams. As AI search engines like Perplexity and SearchGPT continue to gain market share, the competition for citations will likely become as fierce as the competition for the "blue links" of the past two decades.
The broader implication for businesses is a move toward "structural excellence." Whether it is through the use of Schema markup, Markdown formatting, or the strategic placement of FAQs, the way information is structured on a page is now just as important as the information itself. For enterprises, the path forward involves a blend of technical SEO rigor and a new, data-driven approach to AI engine optimization. By focusing on causation rather than mere correlation, brands can ensure they remain authoritative voices in an increasingly AI-mediated world.
