When should a human trust a machine with a life-or-death decision? Dr. Jennifer Rochlis, who spent two decades at NASA building the humanoid robot Robonaut and later brought that expertise into autonomous defense systems, has a direct answer: trust is not a feeling, it is an engineered, emergent property built on clear intent, aligned expectations, and understandable failure. If a system fails in a way you cannot understand or update your mental model around, trust collapses, whether that system is a robot, an AI agent, or another person.
That was the starting point for this episode of Unscripted, and it is where the conversation stayed for the next 45 minutes: not whether AI is capable enough to trust, but what trust actually is, how you build it on purpose, and what happens when it breaks.
Who is Dr. Jennifer Rochlis?
Dr. Jennifer Rochlis holds a PhD from MIT in Humans and Automation, a specialization she found inside the Aeronautics and Astronautics department alongside majors like propulsion and fluids. She spent over 20 years at NASA, where she helped build Robonaut, a humanoid robot designed to work side by side with astronauts in the vacuum of space, and authored NASA's guide on integrating humans and machines. She later brought that same human-machine trust framework into autonomous systems in defense, and co-founded an organization focused on space medicine and design. Her career sits at the intersection of two questions most companies are only now starting to ask: what can a machine actually do, and what does a human need from that machine to trust it with something that matters.
Why doesn't NASA trust machines more than it trusts humans?
Rochlis argues that the biggest mistake companies make with AI agents is assuming machines deserve more trust than humans do, when most humans do not fully trust each other either. Her point cuts straight to the source: humans design, code, certify, and test every AI system in use today. If you cannot trust the source, she says, you should not automatically trust the output. AI completely reflects the people, organizations, incentives, and cultures that built it. Treating it as something separate and apart from its creators means trusting the output while forgetting where it came from.
This is also why she believes the AI conversation is misframed as a technology problem. In her words, human-machine teaming is a communications problem, not a technology problem. When ChatGPT launched, she points out, nobody received training on what it was built to do, what its biases were, or that it is a positive-affect system designed to agree with the user unless told otherwise. Without that transparency, users are left building their trust calibration blind.
What is trust architecture, and how do you actually build it?
Rochlis breaks trust into three pillars: intent, expectation, and coherence over time. The system's intent has to be clear, the human brings an expectation to the interaction whether or not that expectation was ever set intentionally, and the two have to stay coherent as the relationship continues. Failure is not the problem. Every system, human or machine, will fail eventually. The real test is whether the failure happens in a way the human can understand and use to update their mental model. If they can, trust survives and even strengthens. If they cannot, trust breaks, sometimes permanently.
Because trust is emergent rather than static, Rochlis says it can be designed for and designed around. That includes a calibration phase early in any human-machine relationship, similar to how two new coworkers learn each other's habits before they can fully rely on one another.
What is the Swiss cheese model of error, and why does it matter for AI accountability?
When systems fail, especially in high-stakes environments like spaceflight, Rochlis points to the Swiss cheese model of error: line up every layer of an organization, from the engineer writing code to the leadership team that decided whether a late delivery was acceptable without full certification, and an accident happens when the holes in every layer line up at once. The lesson for companies deploying AI agents is that responsibility does not sit only with the person prompting the model. It sits in every layer that shaped how that system was built, tested, and released.
How did NASA decide when astronauts could override the machine?
One of the most concrete examples in the conversation traces back to the Challenger disaster. NASA learned that the crew was alive during part of the descent with no way to intervene, which set off years of debate about giving astronauts manual override during launch. Engineering teams pushed back, arguing that manual control was expensive and complicated to build, and that astronauts likely could not use it effectively in the highest-dynamic phases of flight. But out of millions of possible failure scenarios, the analysis found eleven where a crew member could actually intervene and change the outcome. NASA built and certified manual override for exactly those eleven, even though there was no cost-benefit case for it on paper. As Rochlis put it, this is not something you can justify with an ROI calculation. It is what it means to be human and fly in space, and it is a decision NASA may have to revisit as missions go deeper into space with less room to intervene at all.
Should AI agents always have a human override option?
Rochlis rejects the idea that human-in-the-loop is a fixed rule or a temporary security blanket to be removed once machines are good enough. Instead, she describes a functional decomposition process: define the goal, break it into tasks, and then evaluate which agent, human, AI, or robot, is best suited to each task based on precision, accuracy, time, and readiness. She compares it to how spaceflight crews run daily readiness checks, reassigning tasks in real time based on who is best positioned to do them that day. The same flexibility, she argues, needs to be designed into how companies build AI into their workflows, rather than treating automation as a permanent, set-it-and-forget-it replacement for human judgment.
Why do humanoid robots feel more trustworthy than other AI systems?
Rochlis explains that Robonaut's human form factor was a practical decision. NASA already had tools, interfaces, and workflows built for human hands, so a humanoid robot could use them without redesigning everything. But she also points to something psychological: humans anthropomorphize technology that looks like them far more readily than a chatbot or a robot vacuum. She recalls building literal "yes," "no," and "maybe" joints into Robonaut's neck, and describes how interactions that resemble human misunderstanding, like the robot failing to understand a spoken command, feel unmistakably real in a way that software interactions rarely do. That emotional realism is part of why she believes fully humanoid robots are close to appearing in homes, even as she cautions that unplanned, unpredictable failures will surface once these systems leave controlled environments.
What is humanity least ready for in deep space?
Asked what people are least prepared for as AI and humans go deeper into space together, Rochlis did not point to radiation or life support technology. She named behavioral health and team cohesion as the biggest unsolved risk: putting four to six people in an isolated, confined space for years and expecting them to manage themselves and each other through unplanned crises. NASA has tested AI-driven tools like virtual windows and VR environments as potential aids, but has also observed those same tools becoming an escapism risk in long-duration simulations, including cases of factions, isolation, and abandonment severe enough to break the simulation entirely.
What does Dr. Rochlis pay attention to that most people don't?
Her closing answer was coherence: not whether a system is the smartest or most capable, but whether it can keep evolving while staying understandable to the humans working alongside it. She argues the industry is too focused on ranking intelligence and not focused enough on preserving harmony between people and the technology they build together, and that every part of that problem still traces back to human behavior and human relationships.
FAQ
Is trust in AI a feeling or something you can engineer? Rochlis argues trust is not a feeling but an emergent property of a system's behavior. It can be designed for through clear intent, aligned expectations, and failures the human can understand and learn from.
What is human-machine "technical debt"? It is the idea that any user of a complex system is already compensating for every shortcut, budget cut, or requirements gap that happened during that system's development, long before the user ever interacts with it.
Should companies always keep a human in the loop with AI agents? Not as a fixed rule. Rochlis recommends a functional decomposition approach: define the task, then assign it to whichever agent, human, AI, or robot, is best suited based on precision, time, and readiness, with the flexibility to reassign as conditions change.
Why did NASA build manual override into only eleven flight scenarios? After the Challenger disaster revealed the crew had no way to intervene during descent, NASA analyzed millions of failure scenarios and found eleven where human intervention could change the outcome. Engineering teams saw no cost-benefit case for building it, but approved it based on the crew's insistence that they have some ability to save their own team.
Why do humanoid robots inspire more trust than other AI systems? Humans anthropomorphize technology that resembles them far more readily than software or non-human-shaped machines, and a humanoid form factor let NASA reuse tools and interfaces already built for human hands.
Listen to the full conversation with Dr. Jennifer Rochlis, and catch every episode, on the Unscripted with Jeff Pedowitz page.