In late 2022 a global bank approved an AI system to automate routine credit decisions. In pilots it was faster than people and made fewer errors. Risk signed it off, compliance approved it, and the CEO talked it up in public. Six months later the system was still scoring every application, and most files were still going through a human review anyway.
That story opens Das Narayandas’s article in the November–December issue of Harvard Business Review. It’s the clearest explanation I’ve read of why AI tools stall in the businesses that bought them. His examples are banks, airlines, UPS and Amazon. The same pattern shows up in a fifteen-person firm. It just looks different.
What the research found
Narayandas argues that these systems rarely fail on the technology. They get approved, and then they never get authority: the right to actually make the call. So they settle into a permanent advisory role, consulted and then overruled.
The reason is accountability. The decision logic moves into the system, but responsibility for the outcome stays with a person. If you’re the one who carries the blame when it goes wrong, overriding the system is the sensible thing to do. A dispatcher at a logistics firm told him that if the system made the call and something went badly, “it is my name on the report.” He had read the incentives correctly.
Two findings stood out to me.
Training doesn’t fix it. The pattern appeared even where change management was done well and staff openly agreed the system was better than them. As long as accountability stays personal, people protect themselves.
Nobody refuses outright. The system gets hemmed in instead: an extra review step here, a lower threshold or a new flag for sensitive cases there. Each one is defensible on its own. Add them up and the AI ends up as a suggestion box with a dashboard.
The organisations that got past this changed the accountability around the system as well as the system itself. UPS started measuring drivers against the route its software prescribed, instead of against their own past performance, so following the algorithm became the safe choice. Amazon let business teams shape the inputs and parameters but made routine overrides of the output need approval. When something missed, the question became which input or rule had failed, and who would fix it.
Whose name is on it in your business?
I spent twenty years selling enterprise software, a lot of it CRM, and every sales leader knows a version of this. The system goes live, the adoption dashboard is green, and the real pipeline lives in a spreadsheet and the sales manager’s head. On the forecast call, the human number wins. Reps liked the tool well enough. Their commission rode on the number, so they trusted their own call over the system’s.
In a bank, accountability is spread across risk, compliance, line managers and committees. In a founder-led firm it mostly sits with one person. The founder’s name is on every client relationship and every piece of advice that leaves the building.
So the founder is the dispatcher. And the team knows it.
That’s why “can you have a quick look before it goes out?” becomes a permanent step, even when the AI draft is good. Your team is reading the situation correctly. If something goes wrong with a client, you’ll ask who sent it. And you keep reviewing because, as far as the client is concerned, it’s your name on it.
This is the brutal binary I find in most founder-led businesses I audit. Either the founder reviews everything the AI produces, or the AI runs loose and nobody is sure what’s underneath it. Narayandas’s research explains why the middle ground is missing. Nobody has written down who is allowed to decide what.
Why this is easier at your size
Here’s the part the HBR article doesn’t say, because it isn’t written for you.
A bank that wants to move authority to a system has to rewrite performance reviews, incentive plans, escalation policies and regulatory filings across thousands of people. A fifteen-person firm has to get one person to make four decisions and write them down.
When the accountability structure is a single person, that person is the bottleneck. They’re also the fastest route to moving authority anywhere. You don’t need a governance board. You need an afternoon, and the discipline to stop re-checking the things you’ve decided not to check.
Signs your AI has been quietly contained
Most firms judge AI adoption by usage. Narayandas points out that usage tells you very little. What matters is whether people follow it. Go through these honestly:
- AI drafts still go to the founder before they reach a client, and nobody remembers deciding that.
- Drafts get rewritten more than they get edited.
- Your list of “not for this client” exceptions keeps growing.
- People describe whatever the AI produces as “a good starting point.”
- Everyone is using it, and nobody’s week has got any shorter.
- You couldn’t write down, today, which decisions the system is allowed to make on its own.
If two or more of those are true, the tool probably works fine. What hasn’t moved is the authority.
Four questions to answer before the next rollout
Narayandas proposes four questions leaders should settle when a system is approved. If leadership doesn’t answer them at the start, staff will answer them later, in favour of keeping things as they were. Here they are, translated for a founder-led firm:
- Whose judgement does this replace, and on which decisions? Name the person and the decision. “Senior consultant, first-draft scoping notes” is an answer. “Helps with proposals” isn’t.
- What can the system now produce or decide without you? Be literal. For example: proposals under €10k go out after a team member checks them against the playbook, without coming to you.
- When someone follows the playbook and a client pushes back, who answers for it? Whoever owns the playbook, which is usually you. Never the person who followed it. If following the process can get someone blamed, they’ll stop following it.
- What happens to the first mistakes? Decide in advance which errors mean “fix the playbook,” which get escalated to you, and which really are on the individual. Toyota’s Andon cord works because pulling it brings help and a root-cause fix, with no automatic blame. Companies that copied the cord without copying that norm got very little from it.
Question 3 is the one founders push back on. It means accepting that a draft you’d have written differently can still be good enough to send. That’s the price of getting out of the review loop, and it’s worth paying.
It gets sharper when the AI acts
Everything above is about AI that suggests. Agents act. They send the follow-up, update the CRM, book the meeting and chase the invoice. When an agent does something within the limits someone set and the result is bad, working out whose name is on it after the fact is much harder.
That’s one reason Automate sits at the top of the five layers. By the time something runs without you in the loop, the rules about who decides and who carries the early mistakes need to exist in writing. In a well-built playbook they’re part of the document, alongside the decision rules and what escalates to whom.
The test that matters
Narayandas ends with a test I’d put to any founder who thinks they’ve adopted AI. If you switched it off tomorrow, would any decision in your business change? Or would everyone carry on as before, with one less browser tab open?
If nothing would change, authority hasn’t moved. Until it does, you’ve paid for a very good adviser that everyone politely works around.