As companies wire AI agents into the core of how they operate, one question keeps going unanswered: when an agent takes an action that causes real damage, who is accountable? On Unscripted, I put that question to someone who has spent thirty years defending systems the world cannot afford to lose.
The short answer from that conversation: most organizations are deploying AI agents faster than they are building the accountability structures around them, and that gap is where the real danger lives. In the high-stakes systems Lena Smart spent her career protecting, the chain of responsibility was settled before anything went live. With AI agents, many companies are skipping that step entirely.
Who is Lena Smart?
Lena Smart is a security leader who spent roughly thirty years defending systems that cannot be allowed to fail. She secured the power grid in the energy sector, protected financial markets, and served as the longtime Chief Information Security Officer at MongoDB, one of the databases a large share of the internet quietly runs on. She is now helping shape some of the first real standards for trusting, and even insuring, AI agents.
When someone with that record tells you where she would and would not let an AI agent run, it is worth paying attention.
What is the AI accountability gap?
The accountability gap is the space between deploying an autonomous AI agent and having a clear answer for who owns the outcome when it acts badly. Is it the vendor, the team that deployed it, the executive who approved it, or the model provider?
Lena's concern is that companies are racing agents into production without resolving that question first. In the systems she defended, responsibility was mapped before go-live. Many organizations are now inverting that order, and most CISOs have not yet fully grasped how different agent accountability is from anything they have managed before. That is a large part of why she is working on standards for insuring these systems: liability forces the question that enthusiasm skips.
What is shadow AI, and why is it more dangerous than shadow IT?
Shadow AI is the AI already running inside your company without security's knowledge: tools employees have adopted on their own and wired into real workflows and real data. It is the successor to shadow IT, and Lena considers it more dangerous.
The reason is scope. Shadow IT was usually an unapproved app. Shadow AI can be an agent connected to sensitive data, taking actions, and making decisions, all outside any governance. You cannot protect what you cannot see, and most organizations have far more unofficial AI in production than they realize.
How do you govern AI without stifling it?
Lena is not a lock-it-all-down security leader. She knows that if security says no to everything, the business simply routes around security, which makes the organization less safe, not more. Her approach is to govern in a way that lets the business move while closing the doors that actually matter.
In practice that means three things: build a taxonomy so everyone shares the same language for AI risk, set a small number of true non-negotiables, and then default to "yes, with conditions" rather than a flat no. The goal is to decide what matters and govern that well, instead of trying to control everything and controlling nothing.
Why does she ask every chatbot for a lasagna recipe?
Lena's signature test is to ask any customer-facing chatbot for a recipe for lasagna. If it happily obliges, it has abandoned its actual job and is burning the company's compute to make her dinner. As she put it, she is using that company's tokens to make herself a meal.
It is a funny habit that makes a serious point. If a support bot can be talked off task that easily, its guardrails are weak, and someone with worse intentions can push it toward worse outcomes. Weak guardrails and wasted compute are two symptoms of the same underlying problem: a system deployed without enough thought about how it can be misused.
What do insurance and liability change about AI risk?
Insurance changes the conversation because it attaches a price to risk. Once an insurer or a liability regime is involved, "we will figure out accountability later" stops being acceptable. Companies have to define what the agent is allowed to do, what happens when it fails, and who is responsible, before coverage exists. Lena's work on standards for insuring AI agents is aimed squarely at forcing that discipline into the deployment process.
Why build security in from the start?
One of the hardest numbers in the episode: retrofitting security after the fact cost her organization about four times more than building it in from the beginning. It is far cheaper and far safer to design security into an AI system at the start than to bolt it on once the system is already in production. For any team standing up AI agents right now, that is the difference between the companies that will be ready and the ones scrambling later.
Frequently asked questions
Who is responsible when an AI agent causes harm? There is no settled answer yet, which is the core problem. Responsibility could sit with the vendor, the deploying team, the approving executive, or the model provider. Organizations should define accountability explicitly before deploying an agent, not after something goes wrong.
What is shadow AI? Shadow AI is AI tooling that employees adopt and connect to company data and workflows without security's knowledge or approval. It is the successor to shadow IT and is considered more dangerous because agents can access sensitive data and take autonomous actions.
How can a company govern AI without slowing the business down? Build a shared taxonomy for AI risk, set a small number of non-negotiables, and default to approving use cases with conditions rather than blocking them. Governing what matters well beats trying to control everything.
Why do security experts test chatbots with off-topic requests? Getting a bot to do something unrelated to its purpose, like producing a recipe, reveals weak guardrails and wasted compute. If a bot is that easy to redirect, it can likely be pushed toward more harmful behavior.
Is it better to build security in early or add it later? Early, by a wide margin. Retrofitting security into an already-deployed AI system can cost several times more than designing it in from the start, and it leaves the system exposed in the meantime.
Listen to the full conversation with Lena Smart, and catch every episode, on the Unscripted with Jeff Pedowitz page.