It has become quiet around AI Overviews. One month after my initial traffic impact analysis, I updated my data for AIOs. The results are important for anyone who aims for organic traffic from Google as we’re seeing a shift in AIO structures.
Shortly after Google just launched AI Overviews on May 14, I looked at 1,675 queries and found:
- -8.9% fewer organic clicks when a domain is cited in AIOs than regular results.
- A strong relationship between a domain’s organic ranks and AIO citations.
- Variations of referral traffic depending on user intent.
Since then:
- Featured snippets and AIOs confuse users with slightly different answers.
- Google has significantly pulled back AIOs across all industries.
- AIOs cite more sources.
Boost your skills with Growth Memo’s weekly expert insights. Subscribe for free!
AIOs Dropped By Two-Thirds
A few days after Google launched AIOs in the US, users found misleading and borderline harmful answers.
In a post titled “About last week,” VP of Search Liz Reid addressed the issue, but also called out that many queries were phrased in a way that would likely return questionable answers.1
The debate about LLM answers and questionable queries is not new. Yes, you might get a funny answer when you ask an LLM a funny question. Leading queries were used in the NY Times vs. OpenAI lawsuit and backlash against Perplexity and are no different than leading questions that suggest the answer.
After the PR backlash, Google dropped AIOs across almost every industry by an average of two-thirds.
- May 30: 0.6% on desktop, 0.9% on mobile.
- June 28: 0.2% on desktop, 0.3% on mobile.
Industries with the largest drops (data from Semrush Sensor):
- Health: -3.7% desktop, 1.3% mobile.
- Science: -1% desktop, -2.6% mobile.
- People & Society: -2% desktop, -3.9% mobile.
Image Credit: Kevin IndigIt seems that YMYL industries, such as health, science, animals, and law, were most affected. Some industries gained a small amount of AIOs, but not more than a negligible 0.2%.
- Example: SEOmonitor clearly shows the pullback in visibility metrics for the jobs site monster.com.
Image Credit: Kevin IndigFor the 1,675 queries I analyzed, the number of AIOs dropped from 42% to 23% of queries (almost half). Interestingly, the domain was cited more often (31% vs. 25%, more shortly) and ranked more often in the top 10 spots (45% vs. 41%).
Image Credit: Kevin IndigQueries that stopped showing AIOs had, on average, less search volume. However, I couldn’t detect a clear pattern across word count, user intent, or SERP features for queries that gained vs. lost AIOs. The effect applies broadly, meaning Google reduced AIOs for all types of queries.
Image Credit: Kevin IndigAIOs Lean Heavily On No. 1 Web Result For Text Snippets
The before and after comparison allows us to learn more about the structure and behavior of AIOs.
For example, [hair growth products] and [best hair growth products] deliver almost identical AIOs (see screenshots below). The text is the same, but the product list and cited sources are slightly different. Google treats product searches as equal to “best” searches (makes sense).
SERPs for hair growth products (Image Credit: Kevin Indig)
SERPs for best hair growth products (AIO text is identical to screenshot above) Image Credit: Kevin IndigThe biggest difference is that the query for [hair growth products] shows no citation carousel on the side when you click the “show more” button (another example below).
On mobile, the carousel lives at the bottom of the AIO, which is not great for click-throughs. These subtle design differences likely make a big difference when it comes to clicks from AIOs since more prominently featured citations increase the likelihood of clicks.
Citations only expand when users click “show more” (Image Credit: Kevin Indig)For transactional queries like [hair growth products], Google ranks products in the AIO in no apparent order.
I cross-referenced reviews, average ratings, price, organic product carousel and references in top-ranking articles – none indicate a relationship with the ranking in the AIO. It seems Google leans on its Shopping Graph to sort product lists.
To structure the AIO text, Google seems to pick more elements from the organic No. 1 result than others. For example, time.com ranks No. 1 for [best hair growth products]. Even though the citation in the AIO highlights a section about ingredients (purple in the screenshot below), the whole text closely mirrors the structure of the TIME article before it lists products.
The AIO mirrors the text on the No. 1 web result (time.com) (Image Credit: Kevin Indig)AIOs use fragments of top web results because LLMs commonly use Retrieval Augmented Generation (RAG) to generate answers.
I wrote in How SEO might thrive under Bard and Prometheus:
Sridhar says that Neeva uses a technique called Retrieval Augmented Generation (RAG), a hybrid of classic information retrieval and machine learning. With RAG, you can train LLMs (Large Language Models) through documents and “remove” inaccurate results by setting constraints. In plain terms, you can show AI what you want with the ranking score for web pages. That seems to be the same or similar technique Bing uses to make sure Prometheus results are as accurate and relevant as possible.
The best example of Google mirroring the AIO after the No. 1 web result (in some cases) is the answer for [rosemary oil for hair growth]. The AIO pulls its text from MedicalNewsToday (No. 1) and restructures the answer.
Text in the AI Overview vs. a snippet from MedicalNewsToday (Image Credit: Kevin Indig)AIOs And Featured Snippets Still Co-Exist
For more informational queries with a featured snippet, like [dht], [panic attack vs. anxiety attack], or [does creatine cause hair loss], Google closely mirrors the answer in the featured snippets and elaborates further.
High overlap between AIOs and featured snippets (Image Credit: Kevin Indig)In some cases, the elaboration might confuse users. When searching for [which vitamin deficiency causes hair loss], users see a long list in the AIO and a single answer in the featured snippet. While not contradicting each other, the AIO answer makes the featured snippet seem less trustworthy.
Image Credit: Kevin IndigIn my opinion, Google would be best off not showing a featured snippet when an AIO is present. However, that would be bad news for sites ranking in featured snippets.
AIOs Contain More Citations
One way Google seems to have increased the accuracy of AIOs after the PR backlash is by adding more citations. The average number of citations increased from 15 to 32 in the sample of 1,675 keywords I analyzed. I haven’t yet been able to confirm that more citations are used to compile the answer, but more outgoing links to webpages are a good signal for the open web because they increase the chance of getting click-throughs from AIOs.
Both Reddit and Wikipedia were cited more often after the PR Backlash. I counted citations from those two domains because marketers pay a lot of attention to influencing the public discourse on Reddit, while Wikipedia has a reputation for having more gatekeepers.
Image Credit: Kevin IndigKeep in mind that, with 0.8% and 1%, the number of citations is relatively low. It seems AIO heavily diversifies the number of citations. Only 23 keywords in the 1,675 keyword sample returned more than 10% of citations from Reddit after the PR backlash (28 for Wikipedia).
Accountability
We can conclude that:
- Google shows 50-66% fewer AIOs, which reduces the risk of losing organic traffic – for now.
- There seem to be more opportunities to be cited in AIOs, but strong performance in classic web search still largely determines citations and referral clicks from AIOs.
- Featured snippets get fewer clicks when AIOs are present since they elaborate much more on the answer.
Google becomes more accountable as it touches the border to publishing with AI Overviews. Verticals like health, science, and law continuously morph as new evidence comes out. It will be curious to understand whether AIOs are able to factor new evidence and opinions in and at what speed.
It’s not clear how, exactly, AI Overviews evaluate the strength of evidence, or whether it takes into account contradictory research findings, like those on whether coffee is good for you. “Science isn’t a bunch of static facts,” Dr. Yasmin said. She and other experts also questioned whether the tool would draw on older scientific findings that have since been disproved or don’t capture the latest understanding of an issue.2
If AIOs adapt to new information, websites need to monitor AIOs and adapt content at an equal speed. The adaptation challenge alone will provide room for competitive advantages.
1 AI Overviews: About last week
2 Google Is Using A.I. to Answer Your Health Questions. Should You Trust It?
Featured Image: Paulo Bobita/Search Engine Journal
