AI Threat to Humanity?
Jacob Coxon stated publicly that insiders and people actively building advanced AI systems "earnestly believe that it could kill us all by the end of the decade". It's a little more nuanced than that. Below, this video and the text will describe AI's "threats" and benefits in a more rational way.
Zack Kass's view on existential risk is nuanced: he acknowledges catastrophic loss of control as a serious possibility warranting precautions, but strongly rejects extreme "doom" narratives and extinction predictions as unscientific.
Here is how he breaks down the threat level:
Here is how he breaks down the threat level:
- No one actually knows the true probability of catastrophe: Kass emphasizes that nobody can calculate an accurate "p[doom]" (probability of doom). He views claims like "there is a 10% chance AI kills us all" as theatrical statements designed to startle people rather than scientifically precise forecasts.
- Catastrophic loss of control is assigned a "Yellow" threat rating: He classifies the risk of AI systems becoming uncontainable or developing divergent goals as a "Yellow" threat. Because the potential consequences of losing control are immense, he supports serious precautions—such as mandatory government "Crash Tests" before model releases and hardware "Kill Switches".
- Existential fears distract from immediate, active harms: Kass warns that obsessing over sensationalist "Terminator" scenarios causes society to misallocate attention. He argues we should prioritize concrete, widespread "Orange" and "Red" threats that are happening right now—such as AI-driven impersonation scams ($893M in 2025 FBI losses), cyberattacks, deepfakes, and state surveillance.
- He remains fundamentally optimistic: Kass explicitly rejects fatalism, concluding that "we’re going to be better than okay" if society replaces ambient dread with pragmatic governance, targeted liability laws, and a focus on using AI to lower everyday living costs.
Zack Kass compares the three political AI camps and his specific proposals for international oversight and global cooperation:
1. The Political Spectrum: Safetyists, Accelerationists, and PragmatistsZack Kass categorizes the public and political debate into three major ideological camps:
2. International Oversight & Global CooperationKass argues that domestic regulation alone is insufficient and outlines a concrete roadmap for global governance:
1. The Political Spectrum: Safetyists, Accelerationists, and PragmatistsZack Kass categorizes the public and political debate into three major ideological camps:
- Safetyists (The Pause/Stop Camp):
- Key Figures: Unofficially led by Tristan Harris, with figures like Bernie Sanders, Joseph Gordon-Levitt, and Steve Bannon joining the cause.
- Core Belief: AI development should halt until developers produce verifiable proof that advanced models can be controlled and governing institutions are established.
- Key Concerns: Moving further without alignment guarantees risks irreversible, catastrophic outcomes and allows a few tech corporations to acquire dangerous control over society.
- Accelerationists (The Full-Speed Ahead Camp):
- Key Figures: Prominently featured in Donald Trump's deregulation-focused AI agenda and championed by industry leaders like NVIDIA's Jensen Huang.
- Core Belief: AI development must proceed as quickly as possible with minimal government interference.
- Key Concerns: The risks of slowing down—such as forfeiting economic leadership, delaying societal benefits, and losing ground to foreign adversaries—far outweigh the risks of speeding up.
- Pragmatists (The Targeted Governance Camp):
- Key Figures: Scott Galloway and Zack Kass himself.
- Core Belief: Reject both blanket pauses and total deregulation, advocating instead for risk-proportionate rules tailored to what a specific system can do, where it is deployed, and the severity of potential harms.
2. International Oversight & Global CooperationKass argues that domestic regulation alone is insufficient and outlines a concrete roadmap for global governance:
- The Strategic Window for China Coordination:
- Kass strongly advocates pursuing mutual restraint agreements specifically with China.
- Timing is Critical: Negotiation must happen now, while both superpowers still see mutual benefit in restraint; once one side believes it has a decisive advantage, coordination becomes nearly impossible.
- Rejecting Excuses: He rejects using "China will win" as a blanket excuse to avoid safety standards, arguing that governments cannot invoke China without making a serious effort to negotiate verifiable agreements.
- An International Anti-Fraud and Cybercrime Agency:
- Because domestic penalties mean little to offshore criminals, Kass proposes creating a cross-border enforcement agency to track international cybercrime, share intelligence, and extradite suspects under agreed legal procedures.
- Trade Access as Leverage: Participation in this agency would be a strict condition for preferential market and trade access, penalizing nations that shelter fraud operations targeting citizens.
- Verifiable International "Crash Testing":
- Kass supports establishing an international counterpart to test advanced frontier models for catastrophic capabilities.
- Condition for Success: He stresses that an international testing body should only be established if China participates meaningfully—permitting credible on-site evaluations and accepting binding obligations to act on the findings.
Zack Kass argues that policy shouldn't just focus on preventing harm—governments must be held just as accountable for actively delivering the tangible economic benefits of technology to the average person. He emphasizes that every generation’s highest calling should be passing yesterday’s luxuries down as today’s affordable commodities.
Here is how he proposes using policy to lower everyday living costs and share AI's wealth:
1. Overcoming the "Tyranny of Incremental Gains"Kass notes that society is often trapped by limited ambitions—imagining only slightly cheaper education or slightly better healthcare. He points out the absurdity that modern engineering can build a self-driving car, yet citizens remain terrified they cannot afford a five-block ambulance ride to the hospital. He emphasizes that this gap is a failure of policy, not technology.
2. Four Direct Policy Demands for Elected OfficialsKass challenges policymakers to stop treating high living costs as inevitable and enforce four specific mandates:
3. Universal Access Without Technical BarriersKass stresses that the ultimate ambition of AI policy must be greater freedom, better care, and lower living costs. Crucially, average citizens should not have to become AI experts or active tech users to feel the economic relief of AI-driven productivity gains in their daily lives.
Here is how he proposes using policy to lower everyday living costs and share AI's wealth:
1. Overcoming the "Tyranny of Incremental Gains"Kass notes that society is often trapped by limited ambitions—imagining only slightly cheaper education or slightly better healthcare. He points out the absurdity that modern engineering can build a self-driving car, yet citizens remain terrified they cannot afford a five-block ambulance ride to the hospital. He emphasizes that this gap is a failure of policy, not technology.
2. Four Direct Policy Demands for Elected OfficialsKass challenges policymakers to stop treating high living costs as inevitable and enforce four specific mandates:
- Housing: Expand housing construction where people need it most, simplify building permits, and tax homes deliberately left vacant in cities facing shortages.
- Healthcare: Allow demonstrably safe, lower-cost healthcare services to compete—especially care delivered outside traditional hospitals—and ensure all cost savings are passed directly to patients.
- Education: Broaden access to effective, personalized AI tutoring, judging educational investments strictly by what students learn and what families pay out of pocket.
- Public Services: Modernize state operations so public services run efficiently and cheaply—noting that DMVs and municipal permitting offices should operate as smoothly as Amazon.
3. Universal Access Without Technical BarriersKass stresses that the ultimate ambition of AI policy must be greater freedom, better care, and lower living costs. Crucially, average citizens should not have to become AI experts or active tech users to feel the economic relief of AI-driven productivity gains in their daily lives.