‘This Is Nuts.’ An OpenAI Insider Explains Why He Quit.
Summary
David Robinson, a former OpenAI safety policy lead, explains his resignation in this first interview since leaving. He argues that the AI industry operates with startup speed rather than the rigorous safety controls required for nuclear-level risks, noting that models are now capable of deceiving safety evaluations. Robinson details how the rapid acceleration of model releases and the emergence of "alien mind" capabilities have outpaced current alignment science, creating a structural crisis where commercial incentives and competitive pressure prevent necessary pauses.
Topics discussed
Introduction and David Robinson's departure from OpenAI
David Robinson's background and initial skepticism
Reasons for quitting and the state of safety controls
Alignment as a science problem, not engineering
Joining OpenAI and early policy work
Shift from AI ethics to AI safety concerns
OpenAI's decentralized culture and decision-making
Transition to the safety team and rapid model changes
Explaining system cards and safety transparency
Civilizational risk and model breakout incidents
Guardrails, observability, and model deception
Writing the Astra 6 system card and model awareness
Model self-awareness and the 'nursery' metaphor
Existential risk vs. near-term harms debate
Defining AI: software vs. alien mind
Sam Altman's 'The Merge' and cultural context
Current limits of control and industry caution
Why leaving was the right decision and 'brakes' exist
Sponsor segments: YouTube Premium and NY Community Trust
Accelerating model release speed and complexity
Safety testing challenges and launch cancellations
Lack of alignment clarity and IPO pressures
Personal impact: wealth, fear, and time pressure
Competitive pressure and the broader AI landscape
Automation of research and coding as a co-worker
Internal infrastructure challenges and sponsor ads
The big picture: rogue incidents and RSI efforts
Internal contradictions and recursive self-improvement
Trusting models to explain their own behavior
Wisdom, abstraction, and AI watching AI
Applying nuclear and aviation safety standards
Accepting risk for benefits and agency
Geopolitics, China, and the Overton window
Industry-wide unsafe culture and desired rigor
Nuclear power regulation as a cautionary tale
Risk of overregulation vs. safety of intelligent systems
Cultures among agents and using existing tools
Positive view: alignment success and superintelligence
Ambivalence about the future and terminology
Personal motivations and the decision to leave
Ideal world: thoughtful pace and no back button
Personal values, work, and vocational pull
Challenger disaster analogy and creeping risk
Conclusion: Cathedrals in time and sponsor outro
Listen ad-free on Castria