Most large companies govern AI the same way: write a policy, get it approved, then spend the rest of the time enforcing it. On Unscripted, philosopher and AI ethics advisor Reid Blackman made a blunt case that this approach is fundamentally broken, and he laid out what he thinks should replace it.
The short answer: policy-first AI governance is too slow and goes obsolete before it ships. The alternative is to stop starting with abstract values and start with a single question instead, what are the nightmares? Name the specific bad outcomes for a given AI system, decide what resources and training you need to avoid them, and let cross-functional teams do that problem-solving. It is an engineering-style discipline, not a compliance exercise.
Reid Blackman spent about fifteen years as a philosophy professor specializing in ethics, then left the academy to help large organizations actually operationalize AI ethics. He has advised Amazon, the FBI, and the Canadian government on its federal AI regulations. He wrote Ethical Machines, and his new book, The Ethical Nightmare Challenge, lays out the method described below.
Reid practiced the standard approach for about eight years before concluding it was wrong for AI. The approach is top-down and policy-driven: draft a policy, align the C-suite, get board approval, then enforce compliance. His central objection is pace. It is catastrophically slow.
In a Fortune 500 company, passing an enterprise-wide policy takes about a year. His team can draft one in a couple of days, but then working groups weigh in, outside stakeholders weigh in, the C-suite weighs in, it has to reach the board's agenda, and the lawyers get involved. A year later, two things are true. First, nobody cares about the policy. Second, you have often implemented an out-of-date policy, because AI moved underneath you. You pass a policy on narrow AI and generative AI arrives. You pass one on generative AI and agents arrive. Regulators face the same trap: the EU had to reopen its AI Act when generative AI appeared, and it still does not cover agents well.
The Ethical Nightmare Challenge is Reid's replacement method, designed to stand up real AI governance in weeks rather than months or years. Instead of starting with values that get translated into generic checklists, it starts with three questions for a specific AI system:
If a team can answer those three questions for a given agent, Reid argues, it has done its job. There is a seven-step method underneath for scoring and prioritizing nightmares and deciding whether risk has been reduced enough to deploy, but the frame is those three questions. It is a problem-solving endeavor, not a compliance one, and it is intentionally tech-agnostic, so it flexes to quantum, blockchain, mixed reality, and combinations of them.
Because agents are wildly context-specific. Who is using the agent, for what purpose, connected to which tools, and in what order it calls those tools, all vary enormously from one agent to the next.
Reid's illustration: if a value statement generates twenty procedures, roughly ten will be relevant to a given agent, ten will be irrelevant checkbox compliance, and there will be another ten things you actually needed to do that never appeared on the list. Stamping the same procedures on every agent misses how different, in his word "snowflakey," each one really is.
Specific enough to act on. "Data gets leaked" is too abstract. Reid's example: an agent is connected to a database of customers' financial information and also to an email server, and it is supposed to email internal documents. The nightmare is that it emails sensitive client information to the wrong person. Concretely, it sends the file to Brian Johnson at the company domain when it should have gone to Barry Johnson, and now someone has access to data they should never have seen. That level of specificity is what lets a team design a real safeguard rather than a slogan.
Reid is careful about his own role here: he does not tell a company what its nightmares are or what its risk appetite should be. That is the client's call. His job is to give them the method and the shared worksheet that lets very different people work the problem together.
Reid's answer is that no single person can. Making one executive wholly accountable is unrealistic, because no individual can understand all the ins and outs of these systems. Routing everything through a centralized risk board does not scale either, because as agents proliferate, that board drowns, and almost every agent use case is high risk.
The only workable path, he argues, is pushing more accountability to the front line, on the condition that you have given those people the education, tools, and method to carry it. That is the role of the cross-functional nightmare-challenge teams: a data scientist, a risk professional, and the actual business owner working together. The data scientist can reduce hallucinations with retrieval but not to zero. The business owner knows where to put a human in the loop. Neither can lower the risk enough alone. Together they can.
Reid's most common blind spot in large companies is what he calls non-culpable ignorance. Leaders simply do not know the risks of AI, especially agentic AI, and many assume it is technical work meant for data scientists. He considers that deferral disastrous, because the people who understand how things go wrong in HR, marketing, or clinical settings are exactly the people whose insight the data scientists lack. Without that cross-functional input, the real nightmares stay invisible until they happen.
Reid and I disagreed on timeline, which was one of the more interesting exchanges. He expects five to ten years before AI drives real organizational upheaval, arguing that adoption in large organizations is slower than the headlines suggest, and that this is probably healthy. I put it closer to two to three years, because we are just now moving past the easy phase of using AI to cut costs and into the harder phase where the actual work and the org design start to change.
The exponential jump in risk from generative to agentic AI. Reid put it plainly: moving from narrow AI to generative AI was a big step, but generative to agentic is a mountain, and most people do not grasp it yet. They still picture an agent as a coherent entity, like talking to another person. In reality it is a complex engineered system, closer to a commercial jet or a nuclear plant, an LLM wired to dozens of tools with many ways to call them, pulling from the open internet. That is where cascading and emergent failures live, and it is where attention is currently thinnest.
Why is traditional AI governance too slow? Passing an enterprise-wide policy in a large company takes about a year. By the time it is approved, the technology has often moved from one paradigm to the next, so the policy is obsolete on arrival and rarely changes behavior.
What is the Ethical Nightmare Challenge? A method from Reid Blackman's book that replaces values-first policy with three questions for each AI system: what are the nightmares, what resources avoid them, and what training do people need. It aims to establish governance in weeks rather than years.
Why don't standard AI risk checklists work? AI agents are highly context-specific. A generic list applied to every agent will include irrelevant items, miss critical ones, and reduce governance to checkbox compliance rather than real risk reduction.
Who should be accountable for AI risk? Not a single executive or a centralized risk board, both of which fail to scale. Accountability should be pushed to cross-functional front-line teams that have been given the method, tools, and training to handle it.
What is the difference in risk between generative and agentic AI? Agentic AI connects a model to many tools and lets it take actions, which introduces cascading and emergent failures. Reid describes the jump from generative to agentic as a mountain, with risk increasing exponentially rather than incrementally.
Listen to the full conversation with Reid Blackman, and catch every episode, on the Unscripted with Jeff Pedowitz page.