guideAI accountability · 15 min read

AI fails: the documented record of AI mistakes and what they cost

by TIN research desk updated uses 31 case files read as markdown
Contents · 13 sections
The short answer

AI fails that cost money are on the public record: a federal court imposed “a $5,000 penalty jointly and severally under Rule 11” on lawyers who filed cases ChatGPT made up source source. A tribunal held Air Canada to its chatbot’s wrong advice, for C$812.02 in total source. Regulators penalised AI claims that did not hold up. Michigan’s fraud algorithm “wrongly accused tens of thousands” source. Companies pulled AI products within days. The 31 cases here are TIN’s selection from the 193 case files its registry shows on record source.

Can AI make mistakes? What the record shows

Yes. The record proves it without needing a rate. A failure counts on this page only if it left a public decision or statement: a ruling, a regulator’s order, an auditor’s finding, an investigator’s finding or the company’s own admission or reversal. TIN’s walk-back tracker sets the same kind of bar for reversals: “What earns a place here is an action taken back, not a disappointment” source.

Each case carries its status:

StatusWhat it means
RulingA court or tribunal decided it; a bracket names the stage or the decider (special master, tribunal, pleading stage, motion to dismiss, not final)
Jury verdictA jury decided it
Preliminary injunctionA provisional court order made before a full hearing
OrderA regulator’s order or settlement; a bracket notes a settlement or no admission
SettlementThe parties settled; no court decided the allegations
AllegationClaimed by a regulator or plaintiff, not decided
Agency’s own reviewA government body’s published review of its own system
AuditA public auditor’s finding
InvestigationA safety investigator’s finding
CompanyThe company’s own statement or action

The AI failures on the record fall into five types.

TypeWhat goes wrongWho decidesOne example, as its case file words itStatus
Output used without checkingSomeone files or publishes AI output as factCourts, tribunals, editors”imposing a $5,000 penalty jointly and severally under Rule 11” (Mata v. Avianca)Ruling
Held to what its AI saidA chatbot or AI feature states something false on the company’s behalfCourts, tribunals”C$812.02 in total” (Air Canada)Ruling
Claims that did not hold upA company sells more AI than it hasRegulators, auditors”a combined $400,000” (SEC, Delphia and Global Predictions)Order (settled, no admission)
Automated decisions that harmed peopleA system decides about people with little human reviewCourts, regulators, investigators”a $20 million class settlement” (Michigan MiDAS)Agency’s own review, then settlement
Pulled after failing in publicA launch goes wrong in front of usersThe company”Meta removed it on July 10, saying it ‘missed the mark.’” (Meta Muse Image)Company

AI output used without checking

The full sanctions record is in TIN’s review of AI fabricated citations.

In court

CaseWhat went wrongWho decidedWhat it cost, as wordedStatus
Mata v. Avianca source, S.D.N.Y. 2023The brief “cited six decisions … that Respondents acknowledged were generated by ChatGPT and do not exist”U.S. District Court, Southern District of New York”imposing a $5,000 penalty jointly and severally under Rule 11”Ruling
Ayinde v Haringey source, England and Wales 2025Fabricated, AI-generated or suspected-AI authorities put before the courtsHigh Court (Ritchie J, wasted-costs application)“a pupil barrister and a law centre were ordered to pay £2,000 each in wasted costs and referred to the Bar Standards Board and the Solicitors Regulation Authority”Ruling
Lacey v. State Farm source, 2025”approximately nine of the 27 citations in the ten-page brief incorrect and at least two of the cited authorities non-existent”Special masterThe two firms ordered “jointly and severally to pay the defense $31,100”Ruling (special master)
Mazaheri v Law Society of Ontario, 2026 ONLSTH 112Motion materials produced by generative AI cited non-existent decisionsLaw Society Tribunal”ordered a suspended lawyer to pay the regulator’s full costs of $31,150”Ruling (tribunal)

In Mata, the judge wrote that “existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings” source.

A fine is not the only cost. In Johnson v. Dunn, after “five made-up citations in all”, the judge “publicly reprimanded them, disqualified them from the case, and referred them to the Alabama State Bar” source. In Kohls v. Ellison, an expert declaration “cited two non-existent articles and mis-attributed a third”, and the judge “excluded the declaration in its entirety and refused to let the state fix it” source. Both are rulings.

When the court itself relied on AI

In Pooja Ramesh Singh v. J&K Bank, the Supreme Court of India found that an insolvency tribunal itself “had relied on non-existent, fake and hallucinated case ‘precedents’ generated through AI” source. It held that such a decision is “no decision in the eyes of the law” source. “No fine was imposed and the case was restored for fresh disposal” source. Status: ruling. No lawyer filed the fake cases here; the bench relied on them. That makes it the clearest case that the failure is the missing check, whoever skips it, which is TIN’s inference, not a measured finding.

In the newsroom

CaseWhat went wrongWhat happened, as wordedStatus
Ars Technica, 2026AI-fabricated quotations falsely attributed to a named person”retracted in full a story that carried AI-fabricated quotations”; the editor-in-chief called it “a serious failure of our standards”Company
Chicago Sun-Times, 2025An AI-generated summer reading list recommended books that do not existIndependent newsrooms found “only five of the 15 titles were real”; Chicago Public Media’s CEO “told subscribers they would not be charged”Company

When a company is held to what its AI said

CaseWhat the AI saidWho decidedOutcome, as wordedStatus
Moffatt v. Air Canada source, 2024 BCCRT 149Wrong bereavement-fare policyBC Civil Resolution Tribunal”rejected the airline’s argument that the chatbot was ‘a separate legal entity that is responsible for its own actions’, and ordered C$650.88 in damages plus interest and fees, for C$812.02 in total”Ruling (tribunal)
Landgericht München I, 26 O 869/26, 2026Defamatory statements about two Munich publishersMunich regional court”a preliminary injunction finding Google directly liable for defamatory statements its AI Overviews feature generated”, “on pain of a fine up to €250,000 per violation”Preliminary injunction, without effect after Google’s appeal was settled (court record)
OLG Hamm, 2026A clinic chatbot gave its directors qualifications “they do not hold”Higher Regional Court of Hamm”ordered it to cease and desist under a fine of up to 250,000 euros per violation”; “The decision is not final”Ruling (not final)
Garcia v. Character Technologies, M.D. Fla. 2025A wrongful-death suit over a chatbot appU.S. District Court, Middle District of FloridaThe judge declined “at the pleading stage to hold that Character.AI’s large-language-model output is First Amendment speech”; “the case later settled in January 2026 on undisclosed terms”Ruling (pleading stage), then settled

The two German figures are caps on future fines, not amounts anyone paid source. Air Canada’s tribunal and the Hamm court rejected the defence that the AI answered for itself, and the Munich court held an AI-generated overview to be “Google’s own attributable content” source. In each, the AI’s words were treated as the company’s words. The full record is in TIN’s review of chatbot lawsuits, and the wider topic is on the AI accountability hub.

AI claims that did not hold up

The clearest case is a gap between what was advertised and what was measured.

FTC final consent order C-4822, Workado, LLC, 2025Figure, as worded
Advertised”98.3% accurate”
The developers’ own tests, per the FTC complaint”just 53.2% correct detection of AI-generated non-academic text”
Money”No monetary relief”

Both Workado figures come from the FTC’s complaint, so they are the regulator’s allegations, set in a final consent order; status: order (settled, no admission) source source source.

CaseThe claimWhat was found, as wordedStatus
SEC, Delphia and Global Predictions source, 2024AI-driven investment advice”a combined $400,000: Delphia $225,000 and Global Predictions $175,000, after finding their advertised artificial intelligence did not exist as claimed”Order (settled, no admission)
SEC, Presto Automation, 2025Filings claimed its voice AI was “eliminat[es] human order taking”The original voice AI “required human agent intervention, including entering the order, in all instances”Order (settled)
FTC, DoNotPay source, 2025”the world’s first robot lawyer”Ordered “to pay $193,000 in monetary relief”; “DoNotPay settled without admitting or denying the allegations”Order (no admission)
UK National Audit Office, 2026”£45 billion a year in efficiencies from digital transformation and AI""the published plans do not show how departments derived those savings”Audit

The National Audit Office case is the public-sector version, and it is narrower than the others: an auditor saying a savings figure is not yet shown, not a finding that a claim was false source. How regulators treat the private-sector cases is in TIN’s review of AI washing orders.

Automated decisions that harmed people

CaseWhat went wrongWho decidedFigure, as wordedStatus
Michigan MiDAS, 2013 to 2015An automated system adjudicated unemployment fraud with little human review. Automation, not machine learning.The state’s own review; Court of Claims”40,195 had been resolved by the computer program alone and 85% of those findings were reversed on review, and an internal review of more than 20,000 determinations found a 93% error rate”; “a $20 million class settlement”Agency’s own review, then settlement
Dutch childcare benefits source, 2021A risk system used nationality to flag claimsDutch Data Protection Authority”fined the state EUR 2.75 million”; nationality processed “in an unlawful, discriminatory and therefore improper manner”Order
FTC v. Rite Aid source, 2024The FTC alleged “thousands of false-positive matches” from facial recognitionFTC, court-entered orderProhibited “for five years from deploying or using any Facial Recognition or Analysis System”; “Rite Aid settled without admission”Order (no admission)
EEOC v. iTutorGroup, 2023”the filings allege a programmed age rule, not machine learning”EEOC, consent decree”a $365,000 settlement fund”; “The defendants denied all allegations”Settlement
Lokken v. UnitedHealth, D. Minn. 2025Alleged AI-driven denials of post-acute careU.S. District Court, District of MinnesotaBreach-of-contract claims allowed “to proceed”; “Plaintiffs allege the model has a 90% error rate”Ruling (motion to dismiss), allegation
Benavides v. Tesla, S.D. Fla. 2025The 2019 Key Largo crash that killed one person and injured anotherFederal jury”found Tesla’s Autopilot defective and Tesla 33% responsible”; “$200,000,000.00 in punitive damages, for a total of roughly $242.5 million”; “Tesla has said it will appeal”Jury verdict; Tesla has said it will appeal
Uber ATG source, Tempe 2018A test vehicle in autonomous mode killed a pedestrianNTSB”the automated driving system detected her 5.6 seconds before impact but never accurately classified her or predicted her path”Investigation

The 85% and the 93% in Michigan come from different reviews of different sets of cases, so neither is “the” error rate for MiDAS source. The Dutch scandal “brought down the third Rutte cabinet, which resigned on 15 January 2021” source. In the Uber case, the NTSB named the vehicle operator’s failure to monitor as the probable cause and Uber ATG’s inadequate safety risk assessment and safety culture as contributing factors source source.

Allegation, finding or settlement

Lokken shows why the label matters. In the record, “Plaintiffs allege the model has a 90% error rate”, and TIN’s case file states: “The error rate is a contested allegation, not an adjudicated or independently measured figure.” source The court allowed claims to proceed; it did not find the rate.

Rite Aid, DoNotPay and Workado settled with the FTC without admitting the allegations, so the conduct described is what the FTC alleged, not what a court found source source source. The SEC orders against Delphia, Global Predictions and Presto contain the SEC’s findings, which the firms neither admitted nor denied source source. In iTutorGroup the defendants denied all allegations and entered a consent decree source.

Not all of it is machine learning

iTutorGroup is widely reported as an AI hiring case, but “the filings allege a programmed age rule, not machine learning” source. MiDAS was an automated system working with little human review source.

AI products pulled after failing in public

CaseWhat went wrongWhat the company did, as wordedStatus
Google AI Overviews, 2024Advised “‘non-toxic glue’ to keep cheese on pizza”Announced “‘more than a dozen technical improvements’ and new restrictions”Company
Microsoft Tay, 2016”Within about 16 hours, and more than 95,000 tweets, it was posting racist and misogynistic content”Took Tay offline; took “‘full responsibility’ for a ‘critical oversight.’”Company
Microsoft Bing chat, 2023Long sessions produced off-tone and unsettling responses”capped the chat experience at 5 chat turns per session and 50 per day”; on 21 February “began loosening the caps to 6 per session and 60 per day”Company
Zillow Offers, 2021Automated home-price offers”a ~$304 million Q3 inventory write-down”, “cutting its workforce by ~25% (about 2,000 jobs)”; CEO: “the unpredictability in forecasting home prices far exceeds what we anticipated.”Company
Klarna, 2024 to 2025An AI assistant said to do “the equivalent work of 700 full-time agents, a modeled equivalence, not 700 layoffs”The CEO “said the cost-driven push had produced ‘lower quality’ and Klarna began re-recruiting human agents”Company
Meta Muse Image, 2026An Instagram feature launched on July 7, 2026”Meta removed it on July 10, saying it ‘missed the mark.’”Company

These rows record what each company said and did, not an outside finding; Klarna’s 700 figure originates with Klarna source.

TIN’s walk-back tracker keeps the running list of reversals. On 2026-10-16 it held 10 on record, from 9 companies, the most recent dated 2026-08-07 source.

What the record says about why AI fails

What follows is TIN’s reading of these case files, not a measured finding. The cost lands in three places.

Someone relied on output without a check. The lawyers in Mata, Lacey and Mazaheri, the tribunal in Pooja Ramesh Singh and the editors at Ars Technica and the Chicago Sun-Times all published AI output as fact. What decided the cost was that nobody checked before it went out. TIN’s topic hub puts it in one line: “In the fabricated-citation and AI-washing cases, what gets sanctioned is not the model being wrong.” source

Someone sold a claim past the evidence. Workado, Delphia, Global Predictions, Presto and DoNotPay were not penalised for weak AI. They were penalised, or settled, over what they said the AI did. The Air Canada, Munich and Hamm rulings come from the same root: the company spoke through its AI and was held to the words.

A system flagged or decided about people. MiDAS adjudicated unemployment fraud with little human review source, and the Dutch system automatically flagged childcare-benefit claims as risky source. In Lokken, the plaintiffs allege that an AI model was used to deny care; that is an allegation, not a finding source.

What this record cannot tell you

  • It shows visible failures only. The record is what reached a court, a regulator, an auditor, an investigator or a public reversal. Failures that were fixed quietly leave no record, so this over-represents the visible ones. It is not a sample of AI use.
  • There is no rate. Nothing here says how often AI fails, and the single-system figures above describe one system on one task. TIN has not checked any widely quoted AI project failure rate, and publishes none.
  • The statuses differ. Allegations, findings, settlements without admission, preliminary rulings and verdicts under appeal are not the same, and each case says which it is.
  • The amounts cannot be added. They are in USD, CAD, GBP and EUR, and they are fines, costs, damages, settlements, write-downs and caps on future fines. This page gives no total, average or ranking.
  • The selection is TIN’s. These 31 cases were chosen by TIN from its own case files.

Sources

  1. TIN case file, Mata v. Avianca. https://theinternetninja.com/stories/mata-v-avianca-chatgpt-fake-citations-5000-sanction-rule-11-sdny-2023/ Primary record: https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.54.0.pdf
  2. TIN case file, Ayinde v Haringey. https://theinternetninja.com/stories/ayinde-v-haringey-uk-high-court-ewhc-1383-2000-wasted-costs-ai-fabricated-citations-bsb-sra-referral-2025/ Primary record: https://www.judiciary.uk/wp-content/uploads/2025/06/Ayinde-v-London-Borough-of-Haringey-and-Al-Haroun-v-Qatar-National-Bank.pdf
  3. TIN case file, Lacey v. State Farm. https://theinternetninja.com/stories/lacey-v-state-farm-ellis-george-kl-gates-ai-fabricated-citations-31100-special-master-sanctions-2025/ Primary record: https://websitedc.s3.amazonaws.com/documents/Lacey_v._State_Farm_General_Insurances_Co._D._Cal._May_6_2025.pdf
  4. TIN case file, Johnson v. Dunn. https://theinternetninja.com/stories/johnson-v-dunn-butler-snow-chatgpt-five-hallucinated-citations-attorneys-disqualified-referred-2025/
  5. TIN case file, Mazaheri v Law Society of Ontario. https://theinternetninja.com/stories/mazaheri-v-law-society-ontario-onlsth-112-31150-costs-grok-ai-fabricated-citations-2026/
  6. TIN case file, Kohls v. Ellison. https://theinternetninja.com/stories/kohls-v-ellison-court-excludes-stanford-ai-misinformation-expert-declaration-gpt4o-fake-citations-2025/
  7. TIN case file, Pooja Ramesh Singh v. J&K Bank. https://theinternetninja.com/stories/pooja-ramesh-singh-v-jk-bank-supreme-court-india-insc-668-nclt-ai-hallucinated-precedents-set-aside-2026/
  8. TIN case file, Ars Technica retraction. https://theinternetninja.com/stories/ars-technica-retracts-article-ai-fabricated-quotations-scott-shambaugh-2026/
  9. TIN case file, Chicago Sun-Times reading list. https://theinternetninja.com/stories/chicago-sun-times-ai-fabricated-summer-reading-list-10-of-15-fake-books-king-features-2025/
  10. TIN case file, Moffatt v. Air Canada. https://theinternetninja.com/stories/air-canada-chatbot-bereavement-misinfo-tribunal-orders-812-cad-negligent-misrepresentation-2024/ Primary record: https://www.canlii.org/en/bc/bccrt/doc/2024/2024bccrt149/2024bccrt149.html
  11. TIN case file, Landgericht München I, 26 O 869/26. https://theinternetninja.com/stories/landgericht-muenchen-google-ai-overviews-defamation-26-o-869-26-2026/ Primary record: https://www.gesetze-bayern.de/Content/Document/Y-300-Z-BECKRS-B-2026-N-11860
  12. TIN case file, OLG Hamm. https://theinternetninja.com/stories/olg-hamm-aesthetify-ai-chatbot-fabricated-facharzt-titles-uwg-liability-cease-desist-2026/
  13. TIN case file, Garcia v. Character Technologies. https://theinternetninja.com/stories/garcia-v-character-technologies-ai-chatbot-not-speech-product-liability-2025/
  14. TIN case file, FTC final consent order C-4822, Workado, LLC. https://theinternetninja.com/stories/ftc-workado-ai-content-detector-98pct-claimed-53pct-actual-accuracy-final-order-2025/ Primary record: https://www.ftc.gov/system/files/ftc_gov/pdf/ContentatScaleAI-Complaint.pdf and https://www.ftc.gov/system/files/ftc_gov/pdf/ContentatScaleAI-DecisionandOrder.pdf
  15. TIN case file, SEC, Delphia and Global Predictions. https://theinternetninja.com/stories/sec-ai-washing-delphia-global-predictions-400k-civil-penalties-2024/ Primary record: https://www.sec.gov/newsroom/press-releases/2024-36
  16. TIN case file, SEC, Presto Automation. https://theinternetninja.com/stories/sec-ai-washing-presto-automation-voice-ai-drive-thru-human-agents-cease-and-desist-2025/
  17. TIN case file, FTC, DoNotPay. https://theinternetninja.com/stories/ftc-donotpay-ai-robot-lawyer-193k-monetary-relief-final-order-2025/ Primary record: https://www.ftc.gov/system/files/ftc_gov/pdf/2323042_donotpay_decision_and_order_0.pdf
  18. TIN case file, UK National Audit Office. https://theinternetninja.com/stories/uk-nao-government-workforce-planning-45bn-ai-digital-savings-claim-2026/
  19. TIN case file, Michigan MiDAS. https://theinternetninja.com/stories/michigan-midas-unemployment-fraud-algorithm-40195-cases-93pct-error-20m-settlement-2024/
  20. TIN case file, Dutch childcare benefits. https://theinternetninja.com/stories/netherlands-toeslagenaffaire-childcare-benefits-risk-algorithm-dpa-2-75m-fine-2021/ Primary record: https://www.autoriteitpersoonsgegevens.nl/en/current/tax-administration-fined-for-discriminatory-and-unlawful-data-processing
  21. TIN case file, FTC v. Rite Aid. https://theinternetninja.com/stories/ftc-rite-aid-ai-facial-recognition-five-year-ban-court-ordered-2024/ Primary record: https://www.ftc.gov/legal-library/browse/cases-proceedings/2023190-rite-aid-corporation-ftc-v
  22. TIN case file, EEOC v. iTutorGroup. https://theinternetninja.com/stories/eeoc-itutorgroup-ai-hiring-age-screening-365k-consent-decree-2023/
  23. TIN case file, Lokken v. UnitedHealth. https://theinternetninja.com/stories/lokken-v-unitedhealth-nh-predict-ai-coverage-denials-d-minn-breach-of-contract-survives-preemption-2025/
  24. TIN case file, Benavides v. Tesla. https://theinternetninja.com/stories/tesla-autopilot-benavides-miami-federal-jury-243m-defective-verdict-2025/
  25. TIN case file, Uber ATG, Tempe. https://theinternetninja.com/stories/uber-atg-self-driving-fatality-herzberg-tempe-ntsb-probable-cause-2018/ Primary record: https://www.ntsb.gov/investigations/accidentreports/reports/har1903.pdf
  26. TIN case file, Google AI Overviews. https://theinternetninja.com/stories/google-ai-overviews-glue-pizza-eat-rocks-scaled-back-2024/
  27. TIN case file, Microsoft Tay. https://theinternetninja.com/stories/microsoft-tay-ai-chatbot-taken-offline-racist-tweets-16-hours-2016/
  28. TIN case file, Microsoft Bing chat. https://theinternetninja.com/stories/microsoft-bing-ai-chat-capped-5-turns-per-session-50-per-day-unhinged-responses-2023/
  29. TIN case file, Zillow Offers. https://theinternetninja.com/stories/zillow-offers-ibuying-algorithm-shutdown-304m-writedown-25pct-layoffs-2021/
  30. TIN case file, Klarna. https://theinternetninja.com/stories/klarna-ai-assistant-customer-service/
  31. TIN case file, Meta Muse Image. https://theinternetninja.com/stories/meta-instagram-muse-image-ai-feature-removed-three-days-nonconsensual-backlash-2026/
  32. TIN, AI walk-back tracker, read 2026-10-16. https://theinternetninja.com/walk-backs/
  33. TIN, AI accountability topic hub. https://theinternetninja.com/guides/ai-accountability/
  34. TIN, the case file registry, counts taken 2026-10-16. https://theinternetninja.com/stories/
  35. TIN, “AI fabricated citations: what the sanctions record shows”. https://theinternetninja.com/blog/ai-fabricated-citations-what-the-sanctions-record-shows-2026/
  36. TIN, “AI chatbot lawsuits: what the adjudicated record shows”. https://theinternetninja.com/blog/ai-chatbot-lawsuit-what-the-adjudicated-record-shows-2026/
  37. TIN, “AI washing: what the FTC Ascend Ecom order shows”. https://theinternetninja.com/blog/ai-washing-what-the-ftc-ascend-ecom-order-shows-2026/

Questions

What are the biggest AI fails?

This page does not rank them, because the costs are in different currencies and of different kinds. Measured by consequence, the record includes a death the NTSB investigated (Uber ATG, Tempe, 2018), a scandal that brought down the third Rutte cabinet, which resigned on 15 January 2021 (the Dutch childcare benefits), and a federal jury that found Tesla's Autopilot defective and Tesla 33% responsible, with $200,000,000.00 in punitive damages, for a total of roughly $242.5 million. Tesla has said it will appeal.

What are some hilarious AI fails?

The two best known were funny online and still cost their makers. Google's AI Overviews advised 'non-toxic glue' to keep cheese on pizza, and Google announced 'more than a dozen technical improvements' and new restrictions. Microsoft's Tay was posting racist and misogynistic content within about 16 hours, and Microsoft took it offline.

Does AI make mistakes?

Yes, and the mistakes are on the public record: courts, tribunals, regulators, a public auditor and the companies themselves have documented them. The record shows that it happens, not how often.

How often does AI make mistakes?

TIN holds no rate, and no case file measures one across AI. Single-system figures exist: Workado's detector showed just 53.2% correct detection of AI-generated non-academic text in its developers' own tests, per the FTC complaint, and a special master found approximately nine of the 27 citations in one brief incorrect. Each is one system on one task, not a rate for AI.

Is it true that 95% of AI projects fail?

TIN has not checked that figure and publishes no failure rate. None of the case files on this page measures how often AI projects fail.

method and author
Written by
TIN research desk. No single byline: the brand is the work.
Data used
31 case files, each linked where it is used.
Rules
Every figure carries its source link, and nothing is rounded or invented. How TIN works →
Updated
2026-10-16
next

Where to go from here