Automation projects that fail because they skipped the assessment

The most cited customer service automation case from 2024 and 2025 is Klarna. It is worth looking at closely because this is not a story about whether AI works. It is about what happens when you automate before assessing where a machine’s advantage ends and a person becomes necessary.

What happened

In 2024, Klarna replaced nearly 700 customer service jobs with an AI assistant developed with OpenAI. The company announced that the system was doing the equivalent of those 700 agents’ work and handling 75% of customer conversations in more than 35 languages. For months, it was the example everyone cited as proof that AI could replace skilled work at scale.

Fifteen months later, CEO Sebastian Siemiatkowski admitted to Bloomberg that the company had gone too far. He said it had focused too much on efficiency and cost, and the result was poorer service quality. Klarna began hiring human agents again, this time using a hybrid model: AI handles routine, high-volume queries, while people take the cases that require judgment, empathy or negotiation.

Why it happened, and why the model was not at fault

The revealing part of this case is not that the model failed technically. It is that nobody had clearly defined how far automation could go without lowering service quality. The initial metrics (volume resolved, first response time and conversations per hour) made the project look like an overwhelming success. They hid a problem that surfaced only later: more customers had to get back in touch because their issue was still unresolved, and dissatisfaction rose with it.

The cost of reversing the decision was underestimated too. Rehiring, training and rebuilding a human customer service team proved expensive, a cost rarely included in the initial savings calculation. The promised savings never fully materialised.

Klarna is not alone. Other companies have repeated the same pattern: IBM replaced much of its human resources function with AI, then hired people again when the system could not handle anything requiring empathy or subjective judgment. Commonwealth Bank of Australia reversed 45 job cuts after finding that the roles it had eliminated were never truly redundant.

The question that should have been asked first

In all three cases, the problem was not automation. It was automating without first answering two specific questions: at what point does service quality decline as automation increases, and how would you know before customers tell you? The second question matters just as much: is there a path back to a human, is it one step away in every interaction, or does the customer have to fight to find it?

You cannot answer either question in a demo. You need to assess the real process first: what share of cases is routine and repeatable, which cases need human judgment, and exactly what happens at the boundary between the two.

The lesson for a small business

Klarna’s mistake was one of scale, but the mechanism is the same in much smaller companies: automating 100% of customer service without keeping aside the part that needs a person. The difference is that a small business does not have the cushion of a fintech valued in the billions to reverse course without serious reputational damage.

An assessment is not bureaucracy that delays a project. It is what determines whether the project really saves money or ends up costing twice as much after a failed implementation and a reversal.