Google AI fails: what AI Overviews got wrong, and what Google changed
Contents · 11 sections
Yes, Google’s AI answers have got things wrong on the public record. Google launched AI Overviews to all US users at its annual developer showcase in mid-May 2024 source. Within days it told some users they could use “non-toxic glue” to keep cheese on pizza, and, the BBC reported, said geologists recommend humans eat one rock per day source. Google admitted the glue answer, said it “made more than a dozen technical improvements”, and restricted when the feature appears source. In May 2026 a Munich court granted two publishers an injunction against Google over a false AI Overview; it became without effect when the parties settled after Google appealed source.
Contents
- What AI Overviews got wrong in May 2024
- What Google changed
- When a wrong AI Overview became Google’s own words: the Munich ruling
- Where AI Overviews get their answers today
- Is Google AI getting things wrong now?
- What to do with an AI Overview you are about to act on
What AI Overviews got wrong in May 2024
AI Overviews puts a generated answer above the ranked links it summarises. When it drew on satire or low-quality user posts, it could put a confident, false statement in the spot readers trust most source.
Two answers and one admission carry the record:
- Glue on pizza. The BBC reported that AI Overviews told some users that to keep cheese from sliding off pizza they could use “non-toxic glue” source.
- Rocks. Its responses said, in the BBC’s reporting, that geologists recommend humans eat one rock per day source.
- Google’s own admission. Google’s follow-up post acknowledged advice “like using glue to get cheese to stick to pizza” source.
Where the wrong answers came from
Google named causes in its own post. It saw AI Overviews “that featured sarcastic or troll-y content from discussion forums”, and in “a small number of cases” saw them “misinterpret language on webpages and present inaccurate information” source. It said it “limited the inclusion of satire and humor content” source. The BBC noted answers based on “articles written by satirical site, The Onion” source.
Google’s first line was smaller. A Google spokesperson told the BBC the cases were “isolated examples” source.
A screenshot of a wrong answer is one run of the system. It proves the answer happened. It says nothing about how often.
What Google changed
On 30 May 2024, Elizabeth Reid, Google’s VP, Search, published a post acknowledging the failures and describing the changes source.
| Change | The wording | Where it appears |
|---|---|---|
| Technical fixes | ”made more than a dozen technical improvements to our systems” | Google’s post; ABC News also reported the figure |
| Fewer queries get an Overview | ”added triggering restrictions for queries where AI Overviews were not proving to be as helpful” | Google’s post |
| Less user content | ”updated our systems to limit the use of user-generated content in responses that could offer misleading advice” | Google’s post |
| Hard news kept out | Google aims “to not show AI Overviews for hard news topics, where freshness and factuality are important” | ABC News, reporting Google’s aim |
Sources for the rows: Google source, ABC News source.
What Google did not publish
Google published a count of its fixes. It did not publish how often the feature was wrong. TIN’s case file puts it this way: “No numeric outcome (failure rate, query volume) was ever published, so this page documents the erroneous outputs, Google’s public acknowledgment, and the restrictions it shipped, not a measured error rate” source.
The count of fixes is Google’s own. Google restricted and scaled back the feature; it did not fully disable it source.
When a wrong AI Overview became Google’s own words: the Munich ruling
In 2026 a wrong AI Overview reached a court.
The plaintiffs were two Munich publishers, Verlagshaus24 and its subsidiary GeraMond source source. Google’s AI Overviews, branded in Germany as “Übersicht mit KI”, generated a summary linking them to a “Betrugsmasche” (fraud scheme) when their company name was searched alongside that word, with allegations of “fraudulent schemes” and “disreputable business practices” source. The court found the AI made claims “that are not even made in the search results” source.
The published decision carries an editorial headnote (redaktioneller Leitsatz): “Bei KI-generierten Übersichten einer Suchmaschine handelt es sich nicht um bloße Suchergebnisse, sondern um eigene, der Suchmaschinenbetreiberin zurechenbare Inhalte.” source In English: an AI overview is the operator’s own attributable content, not a mere display of search results source.
| The Munich ruling at a glance | |
|---|---|
| Court | Landgericht München I |
| Case | 26 O 869/26 |
| Date | 28 May 2026 |
| Headnote (editorial) | An AI-generated overview is the search operator’s own attributable content, not a mere display of search results |
| Order | A preliminary injunction bars Google from repeating the specific false claims |
| Penalty | An administrative fine (Ordnungsgeld) of at most €250,000 per case, or detention (Ordnungshaft) of up to six months, at most 2 years in total, enforced against the members of management |
| Costs | 80 percent to Google, 10 percent to each of the two plaintiffs |
| Status | Google appealed (Az. 18 U 1744/26 Pre e); the parties settled, and the judgment is without effect |
Sources for the rows: the court’s published decision source, The Decoder for the costs source.
The status belongs next to the ruling. The court’s page records that after Google lodged an appeal (Az. 18 U 1744/26 Pre e), the parties ended the dispute by settlement: “Das Urteil zum Az. 26 O 869/26 ist damit gegenstandslos”, so the judgment no longer has effect source. It was German law, one court and one injunction. It does not show how courts elsewhere treat AI Overviews.
What it does show is the shift in who owns the mistake. In 2024 Google pointed to troll-y forum content and satire. In Munich the court treated the Overview’s words as Google’s. For the wider record of courts holding companies to what their AI said, see TIN’s review of the AI chatbot lawsuit record.
Where AI Overviews get their answers today
TIN does not measure whether AI Overviews are right. It measures what they cite. Two of TIN’s studies, both edition 2026-10-02, show where the answers come from.
What kind of source AI Overviews cite
TIN asked seven AI engines, Google AI Overviews among them, the same 60 everyday questions in ten countries and coded every cited site into ten categories source. AI Overviews returned 184 citations, its largest category was company sites (39.1%), and its most-cited sites were youtube.com (17), en.wikipedia.org (8), britannica.com (3), heartfoundation.org.nz (3) and isappscience.org (3) source.
| Source category | Share of AI Overviews’ 184 citations |
|---|---|
| Company sites | 39.1% |
| Social and video | 14.1% |
| Associations | 9.8% |
| Government | 8.7% |
| Reference | 7.1% |
| Academic | 6.5% |
| News | 4.9% |
| Health providers | 4.3% |
| Comparison | 3.3% |
| Single-topic sites | 2.2% |
Source for every row: source.
AI Overviews cited 136 different sites, and its top 10 sites hold 26.1% of its citations source.
Set this beside 2024, when Google pointed to troll-y content from discussion forums. In TIN’s 2026 snapshot, Reddit was “cited only 3 times in AI Overviews”, while it is second in AI Mode with 18 citations source. This does not show that Google fixed a forum problem, or that its 2024 changes caused the shift: the questions, the years and the methods differ, and no before-and-after measure exists.
The scope goes with every figure above: “60 questions are a sample of everyday searches, not of everything people ask AI. Business-to-business questions are not covered” source. “Edition 1 is one weekly run (amendment 3), a snapshot, not a trend”, and “Google AI Overviews returned none for France, Italy and Sweden” source.
How much of it sits outside the first page
TIN’s second study read the Google results and the AI Overview for buyer searches in seven markets source. In the US, the crawler, which sees a logged-out page, recorded an AI Overview on 75.8% of 91 usable searches source.
Across 69 US searches with an AI Overview, 152 of the 495 top-10 organic domains (30.7%) are also cited in that search’s AI Overview source. Turned the other way, of the 411 distinct domains the US AI Overviews cite, 63.0% are not among that search’s top-10 organic domains, a count that includes 35 opaque Google redirect links counted as outside source.
The study’s scope: buyer searches in TIN’s channels, “not a random sample of all searches”, in one crawl window on 2026-10-01 and 2026-10-02, “a snapshot” source. Both studies say of their findings: “These are associations in one snapshot, not causes” source source.
Why this matters for errors
Most of what a US AI Overview cites in TIN’s study does not rank on the first page. An answer built partly from pages beyond the first page passes through fewer of the filters a ranked result does (TIN’s inference, not a measured finding). The study measures overlap, not accuracy.
Is Google AI getting things wrong now?
Wrong answers did not stop in 2024: the Munich case shows a false Overview reaching court in 2026 source.
One accuracy figure has been reported since. An analysis by AI startup Oumi for the New York Times found that Google’s AI Overviews with the current Gemini 3 model answered correctly 91 percent of the time, as The Decoder reported source. TIN has not read that analysis, and it covers the current model, not the 2024 launch.
What to do with an AI Overview you are about to act on
The steps below are TIN’s advice, drawn from the record above.
- Open the cited source. Google’s own post pointed to satire and troll-y forum content. Check whether the page behind the claim is a joke or a comment thread.
- Check the source says what the Overview says. The Munich court found claims “that are not even made in the search results” source. A citation next to a sentence is not proof the sentence came from it.
- Be warier on news. Google itself aims to keep hard-news topics out of AI Overviews, as ABC News reported source, so an Overview on a breaking story is outside what Google says it intends.
- If an Overview misdescribes your business, record the query, the date, the country and the exact wording. In Germany, the Munich publishers obtained a preliminary injunction source; that is one route, under one country’s law, and their case ended in a settlement.
If you are a publisher who wants to be cited rather than corrected, that is a different question, answered in TIN’s guide to showing up in Google AI Overviews.
Read next
- Google scaled back AI Overviews after glue on pizza: the full 2024 record, with every quote and its source.
- The Munich AI Overviews ruling: the case in detail, with the court’s German text.
- Which websites AI engines cite, TIN’s study: the source mix for AI Overviews and six other engines, with the dataset.
- Google top 10 vs AI Overviews, TIN’s study: how much of what AI Overviews cite sits outside page one, by market and position.
- The AI chatbot lawsuit record: the wider set of rulings on who answers for what an AI said.
- Microsoft capped Bing AI chat after unhinged responses: the closest sibling, another AI answer product restricted rather than withdrawn days after launch.
- How to show up in Google AI Overviews: the other side of this page, for publishers who want to be cited.
Get cited. If you want to know what AI Overviews cite for your own market’s searches, tell us what you sell.
Sources
- TIN case file, “Google scaled back AI Overviews after it told users to put glue on pizza and eat rocks”. https://theinternetninja.com/stories/google-ai-overviews-glue-pizza-eat-rocks-scaled-back-2024/
- TIN case file, Landgericht München I, Google AI Overviews (26 O 869/26). https://theinternetninja.com/stories/landgericht-muenchen-google-ai-overviews-defamation-26-o-869-26-2026/
- TIN, “AI citation sources” study, edition 2026-10-02. https://theinternetninja.com/recon/ai-citation-sources-2026/
- TIN, “Google top 10 vs AI Overviews” study, edition 2026-10-02. https://theinternetninja.com/recon/google-top10-vs-ai-overviews-2026/
- TIN, “AI chatbot lawsuit: what the adjudicated record shows”. https://theinternetninja.com/blog/ai-chatbot-lawsuit-what-the-adjudicated-record-shows-2026/
- Google, “AI Overviews: About last week” (Elizabeth Reid, VP, Search), 30 May 2024. https://blog.google/products/search/ai-overviews-update-may-2024/
- BBC, “Google AI search tells users to glue pizza and eat rocks”, 24 May 2024. https://www.bbc.com/news/articles/cd11gzejgz4o
- ABC News, “Google makes adjustments to AI Overviews after a rocky rollout”, 3 June 2024. https://abcnews.go.com/Technology/google-makes-adjustments-ai-overviews-after-rocky-rollout/story?id=110710227
- Landgericht München I, decision in case 26 O 869/26, 28 May 2026, gesetze-bayern.de. https://www.gesetze-bayern.de/Content/Document/Y-300-Z-BECKRS-B-2026-N-11860?hl=true
- PPC Land, “Munich court holds Google liable for AI Overviews defamation - a first”, 10 June 2026. https://ppc.land/munich-court-holds-google-liable-for-ai-overviews-defamation-a-first/
- The Decoder, “Landmark German ruling declares Google’s AI Overviews are Google’s own words and makes it liable for false answers”, 11 June 2026. https://the-decoder.com/landmark-german-ruling-declares-googles-ai-overviews-are-googles-own-words-and-makes-it-liable-for-false-answers/
Questions
Is Google AI getting things wrong?
It has, on the public record. In May 2024 AI Overviews told some users they could use non-toxic glue to keep cheese on pizza and said geologists recommend eating one rock per day, and Google acknowledged the glue answer. In 2026 a Munich court granted two publishers an injunction against Google over a false AI Overview; the judgment became without effect when the parties settled after Google appealed.
Is there a problem with Google AI?
After the May 2024 errors, Google's VP, Search, Elizabeth Reid, wrote that Google made more than a dozen technical improvements, added triggering restrictions and limited user-generated content in responses that could offer misleading advice. Those changes are Google's own account.
Why are Google AI answers wrong?
In the 2024 cases, Google's own post pointed to sarcastic or troll-y content from discussion forums and to AI Overviews misinterpreting language on webpages, and the BBC noted answers based on articles by the satirical site The Onion. In the Munich case, the court found the AI Overview made claims that are not even made in the search results.
How often is Google AI wrong?
No failure rate was published for the 2024 incident. An analysis by AI startup Oumi for the New York Times, as reported by The Decoder, found that Google's AI Overviews with the current Gemini 3 model answered correctly 91 percent of the time. TIN measures what AI Overviews cite, not whether they are right.
What did Google change after the glue-on-pizza answers?
Google said it made more than a dozen technical improvements to its systems, added triggering restrictions for queries where AI Overviews were not proving helpful, and limited user-generated content in responses that could offer misleading advice. ABC News reported Google aims not to show AI Overviews for hard news topics. Google restricted the feature; it did not switch it off.
Can Google be held liable for a wrong AI Overview?
One German court acted on that basis. On 28 May 2026 the Landgericht München I barred Google from repeating specific false claims its AI Overview made about two publishers, and the decision's editorial headnote says AI-generated overviews are the operator's own attributable content, not mere search results. Google appealed, the parties settled, and the judgment is therefore without effect.
Where do Google AI Overviews get their information?
In TIN's study of 60 everyday questions, company sites were the largest category of AI Overviews citations and youtube.com the most-cited site. In TIN's study of US buyer searches, 63.0% of the domains AI Overviews cited were not among that search's top-10 organic domains. Both are associations in one snapshot, not causes. That this bears on errors is TIN's inference, not a measured finding.
- Written by
- TIN research desk. No single byline: the brand is the work.
- Data used
- 2 studies, 3 case files, each linked where it is used: AI citation sources: which sites do AI engines cite? 2,857 citations from 7 engines in 10 countries, AI Overview citations: how often is a Google top-10 site also cited? 30.7% in the US.
- Rules
- Every figure carries its source link, and nothing is rounded or invented. How TIN works →
- Updated
- 2026-10-16