Google Admits Search Console Cannot Accurately Track AI Overview Rankings, Analysts Urge Shift to Outcome Metrics

Google Search Advocate John Mueller has publicly acknowledged that Search Console cannot accurately measure rankings within AI-generated search features, confirming a technical limitation that SEO practitioners say has been distorting performance data for some time.

Writing in a Reddit thread, Mueller stated that tracking traditional rankings for AI features is "hard to do in a way that makes it useful," according to Search Engine Journal. He confirmed that Search Console still treats entire AI-generated overviews as a single block — a method the publication describes as "block flattening."

What Block Flattening Means in Practice

Under block flattening, Google does not record where an individual link sits inside an AI Overview. Instead, it assigns one overall rank to the entire overview box. Because AI Overviews typically appear above standard organic listings, every link inside the box is recorded as position 1 in Search Console reports — regardless of whether the link appears prominently under the first sentence or is buried inside a "Show More" dropdown that most users never expand.

The approach creates what Search Engine Journal describes as "a serious mismatch between the data and the real world," because a link card shown near the top of an AI summary and a plain link hidden behind an interactive toggle receive identical position data, even though user click likelihood differs substantially between the two placements.

Impression Counting Adds Further Distortion

The position problem is compounded by how Search Console counts impressions. The publication notes that under standard rules, an impression is logged the moment a search result loads, regardless of whether a user scrolls to see it. When an AI Overview loads, all of its default links count as impressions immediately. Links hidden behind a "Show More" toggle, however, do not count as impressions until a user clicks to expand the section.

This creates an asymmetry: links in the main AI Overview text accumulate impressions even when unread, while links behind interactive buttons appear to have near-zero visibility until expanded.

Impact on Click-Through Data

The reporting quirks are occurring alongside a broader decline in organic click-through rates. Research cited by Search Engine Journal found that "AI Overview CTR fell by 61%" and that click-through rates for cited websites dropped noticeably, partly because the volume of automatic impressions skewed the overall calculation. A separate field study covered by the publication found that having an AI Overview at the top of the page "cut clicks to standard organic results by around 38%," as users obtained answers directly from the summary.

Data referenced in the article shows that AI Overviews appear on 21% of searches and are more prevalent for question-based, informational queries — the segment where Google is most likely to answer a query directly rather than route users to external websites.

Average Position Was Already Imperfect

Search Engine Journal's analysis notes that average position in Search Console was a flawed metric even before AI Overviews arrived, because it blends results from different devices, locations, and result formats into a single mathematical average. The article illustrates the point with a hypothetical: a page that ranks position 1 when an AI Overview is present but position 19 in standard results would be reported as an average position of 10 — a figure that matches neither real-world scenario.

Mueller noted in his Reddit comments that Google has tried to address the issue in its help documentation and has asked the SEO community for practical ideas on how position could be measured when search pages no longer resemble a simple list.

Analysts Recommend Focus on Outcomes

In response to the data limitations, Search Engine Journal argues that marketers should treat AI visibility as a binary yes-or-no question of whether a brand is cited at all, rather than attempting to optimize for a specific numerical position. The publication recommends shifting focus to metrics that cannot be distorted by impression-logging rules: actual site visits, genuine inquiries, and conversions. Where direct layout inspection is needed, the article suggests using automated tools that inspect the live screen rather than relying on Search Console's flattened data.

The article concludes that AI visibility tools from third-party vendors are also unable to resolve the underlying measurement problem, characterizing them as tools that "can't provide any deeper or further insights into this problem."