Who is responsible when the agent gets it wrong?
May 28th, 2025 by Jesper Kråkhede
My previous post was about AI agents as identities. The business question is harder: who is responsible when an agent makes a mistake?
Imagine an agent that handles supplier invoices. It approves a payment to a fraudulent account because the invoice looked right. Or an agent in customer service that promises a refund the company never intended to give. Or a security agent that isolates a production server in the middle of the day. Who is accountable?
The answer can´t be ‘the AI’. Accountability must always land on a person or a function. That means a few things for management.
Every agent needs a business owner, not only a technical one. The person who decided that this task should be automated is responsible for the outcome.
The limits must be clear. What may the agent decide on its own, and what must a human approve? Payments above a certain amount, actions against customers and changes to critical systems should have a human in the loop.
Mistakes must be visible. Log what the agent does and why, so errors can be found and corrected.
And contracts with suppliers of AI services must cover what happens when their AI makes a mistake on your behalf.
Delegating a task to an agent is like delegating it to a new employee. You can delegate the work. You can´t delegate the responsibility.