How do you measure AI ROI?

Ask five questions of any figure. Who produced it? Against what baseline? Was it measured or estimated? Compared with what? Over what period, and at what cost?

Is AI actually turning a profit?

TIN holds no profit figure across companies. Klarna's '$40 million USD in profit improvement to Klarna in 2024' was its own estimate. Zillow wrote down homes it had bought at prices above its own estimates of future selling prices, and Opendoor wrote down homes its pricing model had bought. The Bank of Korea and the Denmark study found small time savings, with no measured effect on output or on earnings and recorded hours so far. That summary is TIN's reading of these case files, not a measured finding.

A percentage cannot be judged without its cost and its period. The 75% among the 26 case files on this page is Aftonbladet's: a sales lift from a company A/B test, not an ROI.

What is the ROI of generative AI?

No cross-company figure for generative AI exists in TIN's record. JPMorgan's own self-reported estimate for its LLM Suite is 'an estimated three to six hours per week', as The Digital Banker reports it. In a randomised trial by MIT researchers published in Science, ChatGPT 'decreased the time it took workers to complete the tasks by 40 percent', per MIT's release. One is a company's estimate; the other is a study of writing tasks.

AI ROI: what companies can show, and what their figures mean, The Internet Ninja

The short answer

In the 26 case files on this page, every AI ROI figure a company published about its own deployment came from the company, its vendor, or press relaying them. Klarna reported customers resolving errands in “less than 2 mins compared to 11 mins previously” source source. Costco’s CFO said its “personalized product recommendation carousels drove over $470,000,000 of ecommerce sales” in Q2 source source. n8n’s case study says Delivery Hero’s “average time locked out dropped from 35 to 20 mins” source source. In TIN’s case files, nobody else has re-measured any of them. The independent numbers are research results: in a Science trial, ChatGPT “decreased the time it took workers to complete the tasks by 40 percent”, per MIT’s release source source.

This guide draws on 26 case files on TIN, read on 2026-10-16, that state a return, saving, cost or time figure from deploying AI or automation, out of the 193 case files the registry shows on record source. Each figure links its case file, and the case file links the record it came from.

What AI ROI means, and what most figures leave out

An ROI compares a gain with what it cost, over a stated period. Only 2 of the 26 case files print a figure called ROI at all; the rest state a saving, a sales lift, a time cut or a loss. One of the two is a projection the FTC alleged was unfounded (see the losses below) source.

The other is Omega Healthcare’s, in two versions. Business Insider reports “30% return on investment since using the AI-powered automation”, with no period source source. UiPath’s release states “a 30% return on investment within the first year” for Omega Healthcare’s customers source source. The two sources give different windows. Its case file is plain about whose return that is: “The ROI claim belongs to Omega’s clients, not Omega” source. The company and its vendor state it; no one else has measured it.

A figure with a gain and no cost beside it is a benefit claim, not a return on investment. That is TIN’s inference, not a measured finding, and it is the lens for the rest of this page.

What deploying companies say they got back

These 15 case files carry a figure about a deploying organisation’s own AI or automation, from the deploying organisation, its vendor, or press. Each figure is worded as its case file words it.

Case	Figure, as the case file words it	Who produced it	Before and after?
Klarna, 2024	”less than 2 mins compared to 11 mins previously”; Klarna “estimated the assistant would drive ‘$40 million USD in profit improvement to Klarna in 2024’”; “doing the equivalent work of 700 full-time agents” source	Company (Klarna’s press release)	Resolution time, yes, company-stated; the profit figure is a forward estimate
Octopus Energy, 2023	”That’s the work of 250 people in the UK alone”	Company (CEO Greg Jackson, in The Times, quoted by Sifted)	No; an equivalence
Notion, with Decagon	”34% faster ticket resolution”, “2x increase in deflection”, “3.4% ask for human rate”; in the body, resolution time “improved up to 34%” source	Vendor (Decagon’s case study); Notion’s Global Head of Customer Experience quoted	Improvement only; no baseline value
Delivery Hero, with n8n	”the average time locked out dropped from 35 to 20 mins. This meant that employees were locked out for a total of 200 hours per month less” source	Vendor (n8n’s case study); client director named	Yes, both values, vendor-stated
Omega Healthcare, with UiPath	”reduced the amount of time workers spend on documentation tasks by 40%” (Business Insider, June 2025); “monthly savings of 6,700 worker hours” (UiPath, Oct 2024) source; “saving employees more than 15,000 hours a month” (Business Insider, June 2025) source	Company (an Omega executive, to Business Insider) and vendor (UiPath’s release)	No; the two hours figures are from different dates and are not merged
Otto, 2017	”Overall, the surplus stock that Otto must hold has declined by a fifth. The new AI system has reduced product returns by more than 2m items a year”	The Economist, sourced from Otto and Blue Yonder; the case file calls the figures “roughly 2017-vintage and unaudited”	Relative change; no baseline value
Coinbase, 2026	”slashed its internal AI spending by nearly half” (Tech Times)	Company (the CEO’s X post and a Coinbase blog), relayed by press	Relative to its peak; no absolute value
JPMorgan Chase, LLM Suite, 2025	”an estimated three to six hours per week” (The Digital Banker); “an hour saved here and three hours there” (the chief analytics officer, AI News); a “10% to 20% productivity increase” for coding (Tearsheet)	Company, self-reported	No; three figures that do not reconcile
JPMorgan COIN, 2017	Work that “until the project went online in June, consumed 360,000 hours of work each year by lawyers and loan officers” (Bloomberg)	Company (its designers, via Bloomberg)	A before value only
Aftonbladet (Schibsted), 2025	”When A/B tested against our previous models, our new approach led to a 75% increase in sales from front-page articles” source	Company (three Schibsted employees, in a blog post)	A company A/B test; no baseline value
Costco, FY2026	The Q2 carousel figure quoted above source	Company (the CFO, on the earnings call)	No; an attribution
Nordea, 2025	”we have seen a 30% reduction in fraud losses” source	Company (an interview Nordea published)	No
UK DWP fraud model	”saving an estimated £4.4 million”; “around three times more effective at identifying fraud risk than a randomised control group sample” source	Government (DWP’s own estimates, relayed by the National Audit Office)	Against a randomised control group run by DWP
Singapore digitalPORT@SG	”is estimated to save the industry 100,000 man-hours per year” (the Deputy Prime Minister, October 2019) source	Government estimate	A forecast made before rollout
UK government, 2026	”Government expects substantial efficiencies from digital transformation and AI, amounting to £45 billion each year” source	Government expectation, as stated by the National Audit Office	A forward expectation

Read the producer column first. These are TIN’s counts of its own 15 case files on 2026-10-16, chosen by TIN, not a sample of AI deployments source:

What TIN counted in these 15 case files	Count
Produced by the deploying organisation, its vendor, or press relaying them	15 of 15
Re-measured by anyone independent, per each case file	0 of 15
Both a before and an after value published	2 of 15 (Delivery Hero, vendor-stated; Klarna’s resolution time, company-stated)
From a controlled comparison the organisation ran itself	2 of 15 (Aftonbladet’s A/B test; DWP’s randomised control group)
Forward estimates or expectations	3 of 15 (Klarna’s profit figure, Singapore’s man-hours, the UK’s efficiencies)

Equivalences are not headcounts

Klarna’s and Octopus Energy’s figures measure workload, not jobs. Klarna’s case file says its figure “is a modeled equivalence, not a headcount cut” source. Octopus’s is “a CEO estimate of workload equivalence, not a headcount cut”, and Jackson said the technology was “unlikely to lead to job cuts” source source. Klarna’s own chief executive later said of the cost-first approach: “what you end up having is lower quality” source. The Klarna walk-back post covers what happened next.

Estimates are not results

Singapore’s man-hours figure “was announced before the industry rollout began” source. MPA’s page today puts it as a ceiling, “can save up to”, and only the builder, NCS, writes that “the shipping community has saved” it source source. “No government source found gives a method or a measured post-rollout figure”, its case file records source.

The UK’s expected efficiencies drew an auditor’s finding: “the published efficiency plans do not provide details of how departments derived their expected workforce efficiencies” source source. Klarna’s profit figure was a company estimate for 2024, made in the release that launched the assistant source.

When there is no figure at all

Mango’s AI-generated fashion campaign is one case: “There is no public figure for how the campaign performed against a conventional shoot: no engagement, conversion, cost, or time number.” source It is not among the 26, because it has no figure to count.

What independent studies measured

Seven case files carry a figure produced by researchers in a study, rather than by a deploying organisation’s own reporting. They are results for the tasks and people studied, not a company’s ROI.

Study	Figure, as the case file words it	Who produced it	Compared against
BCG consultants, GPT-4	”completing 12.2% more tasks and completing them 25.1% more quickly on average” on tasks inside the frontier source	Researchers (a pre-registered field experiment, Organization Science)	A no-AI control group
Noy and Zhang, Science 2023	The writing-time cut quoted above, and “output quality, as measured by independent evaluators, rose by 18 percent” source	Researchers (MIT, a preregistered randomised trial)	A randomised group without ChatGPT
GitHub Copilot, three company trials	A “26.08% increase (SE: 10.3%)” in completed tasks source	Researchers; “two of the six authors are Microsoft employees and Copilot is a Microsoft and GitHub product”	Randomised control groups
Tutor CoPilot, Stanford	”Tutor CoPilot costs only $20 per-tutor annually”; students “4 percentage points (p.p.) more likely to master topics” source	Researchers (a randomised trial, preprint)	Tutors without the tool
John Deere See & Spray	”43 to 59 percent reduction in herbicide use compared to broadcast applications” source	Researchers (University of Arkansas)	Broadcast spraying, in field trials
Bank of Korea, 2026	”AI adoption reduces average work time by 3.8 percent, equivalent to approximately 1.5 hours per week” source	Researchers (the central bank’s research department)	Non-adopters, in household survey data
Denmark, NBER 33777	”adopters in our sample report savings of about 3% of their work hours”; “precise null effects on earnings and recorded hours … ruling out effects larger than 2%” source	Researchers (academic economists)	The time saving is self-reported; earnings and hours come from registers

The task experiments find gains on the tasks they set, and one found the reverse outside them. In the BCG study, on a task chosen to sit outside the frontier, “subjects using AI were 19% less likely to produce correct solutions compared with those without AI” source source.

The economy-wide studies find small time savings and, so far, no measured gain in output or earnings. The Bank of Korea states that “the relationship between time savings and actual output growth is essentially zero” source source. Why time saved does not always become output is argued in TIN’s post on AI and productivity.

The John Deere case file also models a grower saving, which it presents “as a projection rather than a measured outcome” source.

How to measure AI ROI: five questions to ask of any figure

1. Who produced it?

Every Notion figure “comes from Decagon’s own case study, marketing material, and no independent outlet reports these numbers” source. Delivery Hero’s case file says the same of n8n’s source.

2. Against what baseline?

Nordea’s fraud figure “has no period or base to check it against”, its case file says source. Aftonbladet’s lift is “a relative lift against an undisclosed baseline” source. A percentage cannot be judged without its cost and its period, and Aftonbladet’s is a sales lift, not an ROI.

3. Measured or estimated?

JPMorgan’s LLM Suite figure “is JPMorgan’s own self-reported estimate, and different outlets quote it differently” source. COIN’s hours are “a single-origin 2017 corporate claim, relayed by reputable press but never independently measured” source. Print an estimate with the word “estimated” attached, as its producer did.

4. Compared with what?

DWP compared its model against a randomised control group, yet “the underlying numbers are still DWP’s own estimates” source. Aftonbladet ran an A/B test, but “No absolute sales figure, test period, sample size or significance level is disclosed” source. Costco’s figure has no comparison at all: it “is an attribution the company makes to its own carousels rather than an independently measured causal lift” source source.

5. Over what period, and at what cost?

Omega’s return has two windows: “since using the AI-powered automation” in Business Insider and “within the first year” in UiPath’s release source. Coinbase “has published no per-lever numbers, neither absolute cost figures nor how much of the saving came from caching versus the model switch” source source. A saving with no cost beside it cannot be turned into a return.

AI return on investment when the system is wrong

Four case files put a cost on AI or automation that got it wrong.

Case	Figure, as the case file words it	Who produced it
Zillow Offers, 2021	”a write-down of inventory of approximately $304 million within the Homes segment as a result of purchasing homes in Q3 at higher prices than the company’s current estimates of future selling prices” source	Filing (Zillow’s SEC-filed release)
Opendoor, Q3 2022	”an inventory valuation adjustment of $573 million, versus $202 million in 3Q21” source	Filing (shareholder letter)
VA survivors’ benefits, 2026	”resulting in an estimated $2.7 million in improper payments”, which the report records “as questioned costs of $2,727,764” source	Auditor (VA Office of Inspector General)
FTC v. Automators AI, 2024	Projections such as “$200k in one month and 100% ROI (return on investment) in 8 months” source	Regulator (complaint allegations, then a stipulated order)

Zillow’s CEO gave the reason in the same release: “the unpredictability in forecasting home prices far exceeds what we anticipated” source source. Opendoor’s case file attributes its write-down to homes its in-house automated valuation model had bought at a hotter market’s prices source. The VA system is rule-based: “this is process automation, not artificial intelligence” source.

The Automators projections are the second ROI figure in TIN’s record. The FTC said “the vast majority of the defendants’ clients did not make the promised earnings or even recoup their sizable investment” source source. Those are allegations, resolved by a negotiated order rather than a litigated finding source. More costly failures are in TIN’s AI fails guide.

What this record cannot tell you

It is TIN’s selection. These are TIN’s case files, chosen by TIN. Figures that were published are over-represented: a deployment that returned nothing, or that nobody wrote up, leaves no record. It is not a sample of AI deployments.
TIN has measured no ROI. Every figure belongs to its producer, named beside it. TIN holds no profit figure across companies, and no cross-company figure for generative AI.
Research results are local. They hold for the tasks and people studied, not for a company’s deployment.
The record leans American. 15 of the 26 case files carry a US country tag on 2026-10-16 source.
The figures cannot be added. They span 2017 to 2026, in USD, GBP and percentages of different things over different periods. This page gives no total, average or ranking.
There is no failure rate. This page reports no AI project failure rate, and TIN has not checked any widely quoted one source.

Sources

TIN case file, Klarna AI assistant. https://theinternetninja.com/stories/klarna-ai-assistant-customer-service/ Primary record: https://www.klarna.com/international/press/klarna-ai-assistant-handles-two-thirds-of-customer-service-chats-in-its-first-month/
TIN case file, Costco recommendation carousels. https://theinternetninja.com/stories/costco-ai-recommendation-carousels-ecommerce-sales-q2-q3-fy2026/ Primary record: https://www.fool.com/earnings/call-transcripts/2026/03/05/costco-cost-q2-2026-earnings-call-transcript/
TIN case file, Delivery Hero with n8n. https://theinternetninja.com/stories/delivery-hero-n8n-it-ops-automation/ Primary record: https://n8n.io/case-studies/delivery-hero/
TIN case file, Noy and Zhang, Science 2023. https://theinternetninja.com/stories/noy-zhang-mit-chatgpt-writing-rct-40pct-faster-18pct-quality-science-2023/ Primary record: https://news.mit.edu/2023/study-finds-chatgpt-boosts-worker-productivity-writing-0714
TIN case file, Omega Healthcare with UiPath. https://theinternetninja.com/stories/omega-healthcare-uipath-document-automation/ Primary records: https://www.prnewswire.com/in/news-releases/omega-healthcare-processes-60-million-transactions-with-enterprise-ai-and-automation-from-uipath-302285442.html and https://www.businessinsider.com/omega-healthcare-uipath-ai-document-processing-health-transactions-2025-6
TIN case file, Octopus Energy. https://theinternetninja.com/stories/octopus-energy-kraken-ai-support/ Record: https://www.cityam.com/ai-doing-the-work-of-over-200-people-at-octopus-chief-executive-says/
TIN case file, Notion with Decagon. https://theinternetninja.com/stories/decagon-notion-ai-cx/ Primary record: https://decagon.ai/case-studies/notion
TIN case file, Otto. https://theinternetninja.com/stories/otto-germany-ai-stock-replenishment/
TIN case file, Coinbase. https://theinternetninja.com/stories/coinbase-ai-spend-open-weight-routing/ Record: https://www.techtimes.com/articles/319248/20260628/coinbase-cuts-ai-spend-50-chinese-models-legal-risk-its-ceo-didnt-lead.htm
TIN case file, JPMorgan Chase LLM Suite. https://theinternetninja.com/stories/jpmorgan-chase-llm-suite-genai-200000-employees-eight-months-2025/
TIN case file, JPMorgan COIN. https://theinternetninja.com/stories/jpmorgan-coin-contract-intelligence/
TIN case file, Aftonbladet (Schibsted). https://theinternetninja.com/stories/aftonbladet-ml-front-page-subscription-model-75pct-front-page-sales-2025/ Primary record: https://www.inma.org/blogs/ideas/post.cfm/aftonbladet-sees-75-increase-in-subscription-sales-with-front-page-ai-content-recommendations
TIN case file, Nordea. https://theinternetninja.com/stories/nordea-ai-agent-resolves-91-of-in-scope-private-customer-chats-and-cuts-fraud-lo/ Primary record: https://www.nordea.com/en/news/relationships-before-robots-how-nordea-is-leveraging-ai-in-digital-banking
TIN case file, UK DWP fraud model. https://theinternetninja.com/stories/dwp-ml-fraud-model-universal-credit/ Primary record: https://www.nao.org.uk/press-releases/dwp-begins-to-make-headway-tackling-benefit-fraud-and-error/
TIN case file, Singapore digitalPORT@SG. https://theinternetninja.com/stories/singapore-mpa-digitalport-sg-port-clearance-consolidated-into-a-single-digital-s/ Primary records: https://www.pmo.gov.sg/newsroom/dpm-heng-swee-keat-at-the-opening-plenary-of-the-global-maritime-forum-annual-summit-2019/ and https://www.ncs.co/en-sg/knowledge-centre/articles/transforming-the-tech-behind-marine-port-clearance/
TIN case file, UK government efficiencies and the National Audit Office. https://theinternetninja.com/stories/uk-nao-government-workforce-planning-45bn-ai-digital-savings-claim-2026/ Primary records: https://www.nao.org.uk/press-releases/effective-government-workforce-planning-is-key-to-delivering-productive-resilient-and-affordable-public-services/ and https://www.nao.org.uk/wp-content/uploads/2026/07/government-workforce-planning-lessons-learned.pdf
TIN case file, Mango AI campaign. https://theinternetninja.com/stories/mango-ai-generated-fashion-campaign/
TIN case file, BCG consultants and GPT-4. https://theinternetninja.com/stories/bcg-harvard-jagged-frontier-gpt4-consultants-12-2pct-more-tasks-2023/ Primary record: https://pubsonline.informs.org/doi/10.1287/orsc.2025.21838
TIN case file, GitHub Copilot field experiments. https://theinternetninja.com/stories/generative-ai-copilot-three-field-experiments-software-developers-26pct-tasks-2024/ Primary record: https://economics.mit.edu/sites/default/files/inline-files/draft_copilot_experiments.pdf
TIN case file, Tutor CoPilot. https://theinternetninja.com/stories/tutor-copilot-stanford-fev-rct-4pp-mastery-9pp-lower-rated-tutors-2024/ Primary record: https://arxiv.org/abs/2410.03017
TIN case file, John Deere See & Spray. https://theinternetninja.com/stories/john-deere-see-spray-ai-spot-spraying-cuts-herbicide-use-43-59-independently-con/ Primary record: https://aaes.uada.edu/news/see-and-spray-research/
TIN case file, Bank of Korea. https://theinternetninja.com/stories/bank-of-korea-2026-ai-adoption-cuts-work-time-3-8pct-productivity-gain-near-zero/ Primary record: https://www.bok.or.kr/eng/bbs/B0000354/view.do?nttId=10098400&menuNo=400409&relate=Y&depth=400409&programType=newsDataEng
TIN case file, Denmark, NBER 33777. https://theinternetninja.com/stories/denmark-llm-chatbots-adopted-across-11-exposed-occupations-measured-time-savings/ Primary record: https://www.nber.org/system/files/working_papers/w33777/w33777.pdf
TIN case file, Zillow Offers. https://theinternetninja.com/stories/zillow-offers-ibuying-algorithm-shutdown-304m-writedown-25pct-layoffs-2021/ Primary record: https://www.sec.gov/Archives/edgar/data/1617640/000161764021000085/q32021991.htm
TIN case file, Opendoor. https://theinternetninja.com/stories/opendoor-ibuying-avm-pricing-algorithm-928m-q3-2022-net-loss-573m-inventory-writedown-18pct-layoffs/ Primary record: https://www.sec.gov/Archives/edgar/data/1801169/000180116922000106/q32022formxex992sharehol.htm
TIN case file, VA survivors’ benefits. https://theinternetninja.com/stories/va-oig-automated-survivors-death-benefit-decisions-8000-errors-2-7m-improper-payments-2026/ Primary record: https://www.vaoig.gov/reports/review/review-automated-decisions-veterans-service-connected-death-claims
TIN case file, FTC v. Automators AI. https://theinternetninja.com/stories/ftc-automators-ai-ecommerce-scheme-cresto-21-7m-judgment-lifetime-ban-2024/ Primary records: https://www.ftc.gov/news-events/news/press-releases/2023/08/ftc-action-stops-business-opportunity-scheme-promised-its-ai-boosted-tools-would-power-high-earnings and https://www.ftc.gov/news-events/news/press-releases/2024/02/ftc-action-leads-ban-owners-automators-ai-e-commerce-money-making-scheme
TIN, the case file registry, counts taken 2026-10-16. https://theinternetninja.com/stories/
TIN, AI fails guide. https://theinternetninja.com/guides/ai-fails/
TIN, AI in operations topic hub. https://theinternetninja.com/guides/ai-in-operations/
TIN, “Does AI increase productivity: what the measured record shows”. https://theinternetninja.com/blog/does-ai-increase-productivity-what-the-measured-record-shows-2026/
TIN, “Klarna AI layoffs: the 700-agent number and the 2025 reversal, explained”. https://theinternetninja.com/blog/klarna-ai-layoffs-the-2025-reversal-explained-2026/
TIN, “AI back office automation: what the documented numbers show”. https://theinternetninja.com/blog/ai-back-office-automation-what-the-numbers-show-2026/

AI ROI: what companies can show, and what their figures mean