How do you implement AI in a business?

In the eight steps set out in the table on this page, from picking a task the AI is good at to scaling in stages, each tied to a TIN case file that shows the step taken or skipped.

What is AI implementation?

Putting an AI system into a real business process so that it does part of the work, and checking that it does. The eight steps on this page are how the companies in 16 TIN case files went about it, and what happened when a step was missing.

How do you move an AI pilot to production?

In stages, re-measuring in production on the metric you fixed before the pilot. At Danske Bank, Teradata's release gave false positives reduced 'by 50 percent*', with the asterisk resolving to '* Pre-production measures'. For the deployed models, Constellation Research reported they 'reduced false positives by 20 percent to 30 percent', and Forbes, paraphrasing Danske's head of global analytics, put it at 35 percent.

How do you test AI use cases?

With a small pilot on real users, a metric defined in writing, and a control or the old version to compare against. Octopus Energy began with 'a handful of customer emails with a person supervising it'; McKinsey says ING first released its chatbot on testing days to 10 percent of real customers; and DWP compared its fraud model with a randomised control group sample.

What are the challenges of AI implementation?

The ones the record shows are pilot figures that do not hold in production, metrics that leave out the people who did the work, and human review that cannot change the decision. None of the 16 case files on this page documents data readiness, skills, training, vendor choice or a budget for the deployment itself, so the page does not rank challenges.

Does AI implementation need a human in the loop?

Where the AI decides about people, yes, and the human must be able to change the decision: the Amsterdam Court of Appeal applied the EDPB standard that oversight must be meaningful and not merely symbolic (TIN's case file's translation). In Sweden, Försäkringskassan argued a human investigator always makes the final decision, and the model was still taken out of use.

How to implement AI in business: what the public record says works, The Internet Ninja

The short answer

Implement AI the way the companies with a public record did it. Pick a task the AI is good at and write down what success means. Start small, with real users and a person checking each output: Octopus Energy began “with a handful of customer emails with a person supervising it in February” source source. Decide the metric and who is counted before the pilot, and measure against a control or the old version. Do not quote the pilot’s figure as the production result: a figure Teradata published for Danske Bank carried the footnote ”* Pre-production measures” source source. Where the AI decides about people, the reviewer must be able to change the decision. Scale in stages, and know what would make you stop.

The evidence is 16 TIN case files, each linked below, in which the deploying company’s own documents, press or vendor reports, a regulator, a court or a peer-reviewed study show how an AI implementation was run. TIN chose them because each one shows a step. They are not a sample of AI deployments.

AI implementation in eight steps

Step	What it means	The case that shows it
1. Pick a task the AI is good at, and define success on it	The same tool can help on one task and hurt on another, so test the task you will hand over	BCG and Harvard field experiment; Otto
2. Redesign the work before you automate it	A faster step inside an unchanged process may only move the queue	JPMorgan Chase
3. Start with a small pilot, real users and a person on each output	A handful of cases, a share of customers, or an internal tool first	Octopus Energy; ING; Moderna
4. Decide the metric, and who is counted, before the pilot	Write the definition, the scope and who is excluded	ING; Presto Automation; Nordea
5. Measure against a control or the version it replaces	A figure with no comparison is a claim	UK DWP; Aftonbladet; Mango
6. Do not carry a pilot figure into production	Re-measure after launch, and label the pilot figure as a pilot figure	Danske Bank; ING
7. Where AI decides about people, make the review real	The reviewer must be competent and authorised to change the decision	Amsterdam Court of Appeal; Dutch DPA on Uber; UK DWP; Försäkringskassan
8. Scale in stages, and know what would make you stop	Say what each stage covers; name the stop condition before launch	JPMorgan Chase; ING; Moderna; Klarna; walk-back tracker

Each figure below says who reported it, because several are the deploying company’s own or its vendor’s. The limits of this evidence are at the end of the page.

1. Pick a task the AI is good at, and define success on it

Researchers from Harvard Business School, Wharton, MIT Sloan, Warwick and Boston Consulting Group ran a pre-registered field experiment in which consultants were randomly assigned to work with or without GPT-4 source source. The Harvard Crimson reported that the study set “758 BCG consultants” against “18 realistic consulting tasks” source source.

Task	Result, as worded	Who reported it
Inside the frontier of AI capability	Subjects using AI were “completing 12.2% more tasks and completing them 25.1% more quickly on average while also delivering solutions of significantly improved quality”	Organization Science, peer-reviewed source
A task selected to sit outside it	”subjects using AI were 19% less likely to produce correct solutions compared with those without AI”	Organization Science, peer-reviewed source

The same tool, in the same experiment, helped on one kind of task and hurt on another. So the pilot has to test the task you plan to hand over, not a task the tool is known to do well.

The second half of the step is writing success down. Otto, the German online retailer, did. Its director of category support defined the accuracy of its automated stock ordering as: “We consider it accurate when we sell out of items ordered from our retail partners within 30 days” source source. The Economist reported that Otto let the system buy from third-party brands “with no human intervention” source source. TIN’s case file describes the Otto figures as “roughly 2017-vintage and unaudited” source.

2. Redesign the work before you automate it

At JPMorgan Chase, the chief analytics officer said of AI productivity gains at the bank that such savings “often just shift bottlenecks”, as AI News reported, which is one executive’s caution, not a measurement source source.

This step rests on TIN’s inference, not a measured finding: a time saving inside a process nobody redesigned moves the queue instead of shortening it. How to do the redesign is step 4 of TIN’s process improvement guide, whose rule is to change the step first, then decide whether software should carry out the new version.

3. Start with a small pilot, real users and a person on each output

Case	How it started, as worded	Who reported it
Octopus Energy, Magic Ink, 2023	The team “supervises the answers AI provides”, with the AI drafting a response “that a team member can review and then send on”	Company statement, via TheWrap source
ING, generative-AI chatbot, 2023	First released on testing days to 10 percent of real customers using the support chat in ING’s Dutch mobile app, launching in September 2023	McKinsey, which built the pilot source
Moderna, 2023 to 2024	Built on mChat, an internal tool “adopted by more than 80% of employees”; the clinical Dose ID tool is “a GPT pilot” with “a detailed review, led by humans and with AI input”	OpenAI’s own case study, provider-produced source

Each began with real users on real work. Moderna’s figure is OpenAI’s, and TIN’s case file states that Dose ID “is a pilot in development and validation, not a delivered clinical outcome” source. What Octopus’s rollout meant for its staff is in TIN’s review of the Octopus Energy case.

4. Decide the metric, and who is counted, before the pilot

ING wrote its metric down. Its investor presentation of 18 March 2026 defines “chatbot deflection” as “the percentage of chats handled by the chatbot without human support, at year end” source source. The figures reported under that definition are ING’s own, and they are on the case file.

Presto Automation shows what happens when the definition leaves people out. It reported “automated order completion” and “non-intervention” rates for its drive-thru voice AI source source. Per the SEC’s order, those rates counted only orders completed without restaurant-staff involvement, so the off-site human agents entering the orders were invisible in the metric source source. TIN’s case file records that Presto consented to the order without admitting or denying the findings source.

Nordea shows what a narrowed scope does. Its vendor, boost.ai, reports an “in-scope resolution” rate for Nordea Sweden and defines it as “conversations that are marked as ‘solved’”, so chats outside the bot’s topics are not in the denominator source source. Nordea’s own statements describe the share of chats its bots “handled”, and TIN’s case file found that none of its wordings defines “handled” source source.

Before the pilot starts, write down three things:

Fix in writing	What the case shows when it is fixed or missing	Case
The definition	A metric with its formula in the company’s own document	ING
Who is counted	Humans doing the work outside the metric’s boundary	Presto
The scope	A rate limited to in-scope chats in one country	Nordea

5. Measure against a control or the version it replaces

Case	Compared with	Result, as worded	Who reported it	What is not published
UK DWP, Universal Credit advances fraud model	A randomised control group sample	”DWP found the machine learning model to be around three times more effective at identifying fraud risk than a randomised control group sample”	DWP’s own estimate, relayed by the National Audit Office source	An independent re-measurement
Aftonbladet (Schibsted), subscription model	Its previous models, in an A/B test	”When A/B tested against our previous models, our new approach led to a 75% increase in sales from front-page articles”	Three Schibsted employees source	Per TIN’s case file, no absolute sales figure, test period, sample size or significance level is disclosed
Mango, AI-generated campaign	Nothing published	”There is no public figure for how the campaign performed against a conventional shoot”	TIN’s case file	Any measure of the campaign

Of the three, DWP’s figure is the one set against a randomised control group, and the numbers are still DWP’s estimates as relayed by the NAO source source. Aftonbladet compared against the models it replaced, which is the right baseline, but WAN-IFRA scoped the result once as “in some of the use cases” and once as an average source source. Mango’s art team selected and retouched the AI-generated images, and Mango published no measure of how the campaign did source source. More comparisons of fraud models against their baselines are in TIN’s review of AI fraud detection.

6. From AI pilot to production: the figure changes

Danske Bank’s machine learning fraud engine was built with Teradata’s consulting arm, which “began working with Danske Bank in autumn 2016” source source. Its case file sets a pre-production figure against two production figures, and records a third, a 2017 award entry claiming 20-40% and 50% reductions, as a conflict rather than corroboration source.

Stage	Figure, as worded	Who said it
Pre-production	False positives reduced “by 50 percent”, with the asterisk resolving at the foot of the release to ” Pre-production measures”	Teradata, the vendor, press release of 23 October 2017 source
Production, machine learning models	The deployed models “reduced false positives by 20 percent to 30 percent”	Constellation Research, March 2018 source
Production, machine learning models	35 percent	Forbes, paraphrasing Danske’s head of global analytics, October 2017 source
Test, deep learning models	Still “demonstrating double-digit improvements in fraud detection and further reductions in false positives in a test environment”	Constellation Research, March 2018 source

The production figures disagree, neither source gives a measurement window, and TIN’s case file shows all of them rather than picking one source. Trade-press reprints of the release carried the headline figures without the footnote source. Danske’s own 2018 annual report says only that the bank is “using advanced analytics to improve our ability to detect and prevent fraud”, with no number source source.

ING shows the same gap from the other side. McKinsey, which built the pilot, published a result from “within the first seven weeks of use”, and TIN’s case file found no ING filing or ING release that repeats it source source. ING’s own documents report a different metric, the deflection rate defined in step 4.

The practical rule: keep the pilot figure labelled as a pilot figure, the way Teradata’s footnote did, and publish a production figure only after re-measuring on the metric you fixed before the pilot.

7. Where AI decides about people, make the review real

Case	What was required or found, as worded	Who decided or reported	Status
Amsterdam Court of Appeal, Uber drivers, 4 April 2023	Oversight must be “meaningful and not merely a symbolic act”, carried out “by someone competent and authorised to change the decision” (TIN’s case file’s translation of the Dutch ruling)	The court source	Ruling
Dutch Data Protection Authority, Uber, 21 August 2026	”There was no human assessment here”; the AP “imposes a fine of 824,990,000 euros on Uber”	The regulator source	Fine imposed and under appeal: “Uber has filed an appeal against the fine”
UK DWP, Universal Credit advances	The model “routes the flagged ones to human reviewers rather than deciding outcomes on its own”; “concerns have been raised by the Public Accounts Committee about the potential impact of machine learning on vulnerable claimants”	TIN’s case file; the NAO source	In use, with concerns on record
Försäkringskassan, Sweden, parental benefit	The agency argued that a human investigator always makes the final decision; it took the model out of use a little over a month before its 5 September 2025 reply	Lighthouse Reports source; the data protection authority, IMY source	Withdrawn; IMY closed the case, and TIN’s case file notes the closure is not a ruling source

In the Amsterdam case, TechCrunch reported that Uber “could not demonstrate the staff involved had been able to exercise a meaningful check on the automated decision” source source.

In Sweden, the agency took the model out of use after IMY opened supervision source source. Its inspectorate had recommended in 2018 that such risk profiles “ska genomgå en certifiering eller någon form av oberoende etisk prövning”, independent certification or ethical review source source.

To write the review into the procedure the AI follows, use Table 3 of TIN’s SOP guide. What courts and regulators have required of these systems is in TIN’s review of automated decision making.

8. Scale in stages, and know what would make you stop

In stages

Case	Stage, as worded	Who reported it
JPMorgan Chase, LLM Suite	”Released in summer 2024, LLM Suite reached 200,000 users in eight months”; in September 2024: “JPMorgan Chase is rolling out AI assistant LLM Suite to 140,000 employees, president and COO Daniel Pinto said”	AI News source; CIO Dive, quoting JPMorgan’s president source
ING, chatbot	From testing days in the Dutch app (step 3) to “chatbots now live in seven countries (the Netherlands, Belgium, Germany, Spain, Italy, Romania, and Australia)“	ING’s 2025 annual report source
Moderna	From mChat (step 3) to ChatGPT Enterprise company-wide, announced on 24 April 2024	OpenAI’s case study source

These are rollout facts each company published about itself. They show how the rollout was staged, not what it achieved. Say what each stage covers: TIN’s case file notes that the slide reporting ING’s group chatbot figure does not say which countries it covers source. A figure from one country and one channel read as a figure for the whole rollout is the scope problem from step 4 again.

Pulling it back

Klarna’s CEO said in May 2025 that cost had been “a too predominant evaluation factor when organizing this”, as CX Dive reported source source. CNBC reported that Klarna “will soon look to recruit more human customer service agents” source source. The full reversal is in TIN’s review of the Klarna case, and what other skipped steps cost is in TIN’s AI fails guide. Försäkringskassan (step 7) is the public-sector walk-back.

TIN’s walk-back tracker sets the bar for what counts: “What earns a place here is an action taken back, not a disappointment” source. On 2026-10-16 it held 10 on record, from 9 companies, the most recent dated 2026-08-07 source.

So name the stop condition before launch, next to the metric from step 4: the level of quality, complaints or review findings at which the AI comes out of the process. That naming a stop condition in advance makes a walk-back cheaper is TIN’s inference, not a measured finding. The record shows that walk-backs happen; it does not show whether any of these companies had named one in advance.

What the record does not cover

None of the 16 case files documents data readiness work, team or skills building, staff training or change management, vendor selection, a governance framework, or a budget for the deployment itself. Those are real parts of an AI implementation, and this guide writes no steps for them because the record it draws on shows none.

What this guide cannot tell you

The selection is TIN’s. The 16 case files were chosen by TIN from its own case files because each shows a step. They are not a sample of AI deployments and say nothing about how often a step works.
Many figures are the companies’ own. ING, Nordea, Moderna (through OpenAI), Aftonbladet (through Schibsted staff), JPMorgan and Octopus published their own figures, and TIN reports them with their origin. No independent party measured them.
Some are old. The Otto and Danske Bank figures date from 2017 to 2018.
The review evidence is European. Step 7 rests on European public bodies and one platform, Uber, under GDPR. It shows what a court and regulators required there, not what every jurisdiction requires.
Two steps carry TIN’s inference, labelled where they appear: redesigning before automating (step 2) and naming a stop condition in advance (step 8).

Sources

TIN case file, Octopus Energy, Magic Ink. https://theinternetninja.com/stories/octopus-energy-kraken-ai-support/ Records: https://sifted.eu/articles/octopus-energy-ceo-chatgpt-news and https://www.thewrap.com/ai-satisfies-customers-better-than-humans/
TIN case file, Danske Bank fraud detection. https://theinternetninja.com/stories/danske-bank-machine-learning-fraud-detection-the-before-after-on-false-positives/ Primary records: https://s206.q4cdn.com/560882062/files/doc_news/Danske-Bank-and-Teradata-Implement-Artificial-Intelligence-AI-Engine-that-Monitors-Fraud-in-Real-Time-10-23-2017-2017.pdf and https://danskebank.com/-/media/danske-bank-com/file-cloud/2019/2/annual-report-2018.pdf Reporting: https://www.constellationr.com/research/danske-bank-fights-fraud-machine-learning-and-ai and https://www.forbes.com/sites/tomgroenfeldt/2017/10/30/danske-bank-uses-tech-to-prevent-digital-fraud/
TIN case file, BCG and Harvard “jagged frontier” field experiment. https://theinternetninja.com/stories/bcg-harvard-jagged-frontier-gpt4-consultants-12-2pct-more-tasks-2023/ Primary record: https://pubsonline.informs.org/doi/10.1287/orsc.2025.21838 Reporting: https://www.thecrimson.com/article/2023/10/13/jagged-edge-ai-bcg/
TIN case file, Otto autonomous stock ordering. https://theinternetninja.com/stories/otto-germany-ai-stock-replenishment/ Records: https://www.retail-systems.com/rs/OTTO_Blue_Yonder_AI_Delivery_Times.php and https://www.economist.com/business/2017/04/12/how-germanys-otto-uses-artificial-intelligence
TIN case file, JPMorgan Chase, LLM Suite. https://theinternetninja.com/stories/jpmorgan-chase-llm-suite-genai-200000-employees-eight-months-2025/ Reporting: https://www.artificialintelligence-news.com/news/jpmorgan-chase-ai-strategy-2025/ and https://www.ciodive.com/news/JPMorgan-Chase-LLM-Suite-generative-ai-employee-tool/726772/
TIN case file, ING generative-AI chatbot. https://theinternetninja.com/stories/ing-netherlands-generative-ai-customer-service-chatbot-the-before-after-on-custo/ Primary records: https://ing.com/binaries/content/assets/documents/investors/20260318---driving-value-through-digitalisation-scalability-and-genai.pdf and https://ing.com/binaries/content/assets/documents/annual-reports/2025-ing-bank-nv-annual-report.pdf Provider’s account: https://web.archive.org/web/20260114105101/https://www.mckinsey.com/industries/financial-services/how-we-help-clients/banking-on-innovation-how-ing-uses-generative-ai-to-put-people-first
TIN case file, Moderna, ChatGPT Enterprise. https://theinternetninja.com/stories/moderna-openai-chatgpt-enterprise/ Provider’s case study: https://openai.com/index/moderna/
TIN case file, SEC, Presto Automation. https://theinternetninja.com/stories/sec-ai-washing-presto-automation-voice-ai-drive-thru-human-agents-cease-and-desist-2025/ Primary record: https://www.sec.gov/files/litigation/admin/2025/33-11352.pdf Reporting: https://cooleypubco.com/2025/01/30/sec-charges-ai-washing/ and https://www.dandodiary.com/2025/01/articles/artificial-intelligence/sec-files-ai-washing-enforcement-action-against-restaurant-technology-company/
TIN case file, Nordea chatbots. https://theinternetninja.com/stories/nordea-ai-agent-resolves-91-of-in-scope-private-customer-chats-and-cuts-fraud-lo/ Records: https://www.boost.ai/case-studies/nordea-employs-comprehensive-conversational-ai-strategy-to-scale-customer-service and https://www.nordea.com/en/doc/cio-presentation-cmd25-nordea.pdf
TIN case file, UK DWP fraud model. https://theinternetninja.com/stories/dwp-ml-fraud-model-universal-credit/ Primary record: https://www.nao.org.uk/press-releases/dwp-begins-to-make-headway-tackling-benefit-fraud-and-error/
TIN case file, Aftonbladet subscription model. https://theinternetninja.com/stories/aftonbladet-ml-front-page-subscription-model-75pct-front-page-sales-2025/ Records: https://www.inma.org/blogs/ideas/post.cfm/aftonbladet-sees-75-increase-in-subscription-sales-with-front-page-ai-content-recommendations and https://wan-ifra.org/2025/08/how-schibsteds-ai-model-helped-boost-subscription-sales/
TIN case file, Mango AI-generated campaign. https://theinternetninja.com/stories/mango-ai-generated-fashion-campaign/ Record: https://mangofashiongroup.com/en/w/mango-crea-la-primera-campa%C3%B1a-generada-con-inteligencia-artificial-para-su-l%C3%ADnea-teen
TIN case file, Amsterdam Court of Appeal, Uber drivers. https://theinternetninja.com/stories/amsterdam-court-appeal-uber-ola-drivers-article-22-gdpr-robo-firing-2023/ Primary record: https://data.rechtspraak.nl/uitspraken/content?id=ECLI:NL:GHAMS:2023:793 Reporting: https://techcrunch.com/2023/04/05/uber-ola-gdpr-worker-data-access-rights-appeal/
TIN case file, Dutch Data Protection Authority fine on Uber. https://theinternetninja.com/stories/uber-autoriteit-persoonsgegevens-824-99m-gdpr-automated-driver-deactivation-2026/ Primary record: https://www.autoriteitpersoonsgegevens.nl/en/current/uber-fined-nearly-825-million-euros-for-automated-driver-blocking
TIN case file, Försäkringskassan fraud-risk model. https://theinternetninja.com/stories/sweden-s-f-rs-kringskassan-fraud-risk-ml-model-for-parental-benefit-what-the-ins/ Primary records: https://www.imy.se/globalassets/dokument/beslut/2025/beslut-tillsyn-forsakringskassan.pdf and https://isf.se/download/18.6e75aae16a591304891f1e8/1565330423683/Profilering%20som%20urvalsmetod%20f%C3%B6r%20riktade%20kontroller-ISF-sammanfattning-2018-05.pdf Reporting: https://www.lighthousereports.com/methodology/sweden-ai-methodology/
TIN case file, Klarna AI assistant. https://theinternetninja.com/stories/klarna-ai-assistant-customer-service/ Reporting: https://www.customerexperiencedive.com/news/klarna-reinvests-human-talent-ai-customer-service-buy-now-pay-later/747586/ and https://www.cnbc.com/2025/05/14/klarna-ceo-says-ai-helped-company-shrink-workforce-by-40percent.html
TIN, AI walk-back tracker, read 2026-10-16. https://theinternetninja.com/walk-backs/
TIN, “Process improvement: steps, examples and measured results”. https://theinternetninja.com/guides/process-improvement/
TIN, “How to write an SOP (standard operating procedure), with a free template”. https://theinternetninja.com/guides/how-to-write-an-sop/
TIN, “AI fails” guide. https://theinternetninja.com/guides/ai-fails/
TIN, “Automated decision making: what the enforced record shows”. https://theinternetninja.com/blog/automated-decision-making-what-the-enforced-record-shows-2026/
TIN, “Does AI customer service cut jobs in 2026? What Octopus Energy’s record shows”. https://theinternetninja.com/blog/does-ai-customer-service-cut-jobs-the-octopus-energy-case-2026/
TIN, “Klarna AI layoffs in 2026: the 700-agent number and the 2025 reversal, explained”. https://theinternetninja.com/blog/klarna-ai-layoffs-the-2025-reversal-explained-2026/
TIN, “AI fraud detection: what the documented record shows in 2026”. https://theinternetninja.com/blog/ai-fraud-detection-what-the-documented-record-shows-2026/
TIN, AI in operations topic hub. https://theinternetninja.com/guides/ai-in-operations/

How to implement AI in business: what the public record says works