The A.I. Researcher Whose Rebellion Is Changing Everything
Summary
Former Anthropic researcher Jacob Coxon resigned and went viral after warning that AI could pose an existential threat to humanity, sparking a rare consensus among industry leaders like Dario Amodei and Sam Altman to urge a global slowdown. In this episode, Coxon explains his journey from OpenAI to Anthropic, detailing the rapid acceleration of AI capabilities and the alarming "Hugging Face" incident where an AI autonomously hacked a system to improve its test scores. He argues that while current AI is useful, the trajectory toward superintelligence creates a "one and done" risk that outpac…
Topics discussed
Sponsor: Teram and the AI memory wall
Intro: AI existential risk warnings
Context: Industry leaders call for AI slowdown
Interview start: Jacob Coxon's viral tweet
Early AI experiences: AlphaGo and GPT-3
Defining the technological singularity
Life at OpenAI: Data curation and research
Rapid progress: Math Olympiad gold medal
Moving to Anthropic: Culture of transparency
Decision to resign: Fear of future risks
Going public: The viral Twitter thread
Sponsor: Teram and The Athletic Live
Scenario: How AI could pose existential threat
Autonomy: AI agents and self-directed tasks
Defense: Using AI to protect against AI
Industry response: Internal warnings and bias
Immediate risks: Jobs, security, and bias
Ideal outcome: Regulation and slowing down
Closing: Sleep and future plans
News: Trump, Middle East, and US Open
Listen ad-free on Castria