AI Employee Quits Over Safety Concerns

AI Employee Quits Anthropic, Raising Fresh AI Safety Fears Over Human Control

Artificial Intelligence

An AI employee has quit Anthropic, warning that AI Safety risks are growing faster than regulation and human control can keep pace.

Jacob Coxon resigned from Anthropic over growing AI Safety concerns.
He warned that powerful AI systems could eventually threaten human control.
Nearly 1,400 AI employees have called for stronger regulation and slower development.
Anthropic researcher Evan Hubinger said advanced AI could pose an existential threat, he estimated risk below 10%.
The resignations raise a wider debate over whether AI Safety measures can keep pace with rapid technology development.

An AI employee resignation has intensified AI Safety concerns have intensified after Jacob Coxon, a 27-year-old researcher who spent three years conducting pretraining research at OpenAI and Anthropic, resigned from Anthropic over fears that technology development is moving too quickly.

Coxon announced his resignation on X, arguing that both companies are moving toward self-improving superintelligence despite the potential consequences. He warned that future systems could eventually hack computer networks, transform entire industries, and acquire real-world resources and influence.

But what happens when technology development moves faster than the ability to control it?

Coxon is not alone. Researchers and employees across major Artificial Intelligence companies have increasingly raised safety concerns about whether existing safeguards can keep pace with increasingly capable systems. In July, nearly 1,400 AI company employees signed an open letter calling for stronger regulation and a slower approach to development.

Coxon argued that AI companies remain trapped in competition with one another, even when researchers understand the potential dangers. He called for stronger coordination between governments and technology companies, suggesting that temporarily limiting improvements in model capabilities could help reduce technology risks.

Is competition pushing safety decisions into the background?

Anthropic researcher Evan Hubinger supported Coxon’s broader warning, saying researchers genuinely believe advanced AI could pose an existential threat. Hubinger later estimated his personal chance of AI killing all humans within the next decade at under 10%.

Anthropic has maintained that AI can deliver major benefits while also creating unprecedented risks. The company said it continues developing models with strong safeguards.

The concerns extend beyond Anthropic. OpenAI chief scientist Jakub Pachocki has also warned that AI capabilities are advancing faster than researchers’ ability to reliably monitor and control them.

Who should decide how fast powerful AI systems are developed?

The debate now reaches beyond individual employee resignation. AI Safety increasingly involves regulation, corporate responsibility, research oversight, and human control. Companies may argue that safeguards are improving, Business Honor can safety measures keep pace with the systems they are designed to govern?

The answer could shape how governments regulate AI and how companies approach the next stage of technology development.

Frequently Asked Questions

Coxon resigned over AI Safety concerns, arguing that major AI companies are moving too quickly toward increasingly powerful systems.

He warned about technology risks including hacking, rapid industry disruption, resource acquisition, and the possibility of losing human control.

Some employees believe stronger regulation could slow risky technology development and establish clearer safety requirements.

Hubinger said advanced AI could potentially threaten humanity, while personally estimating the chance of such an outcome within a decade at under 10%.

Human control is central to ensuring increasingly capable systems remain monitored, constrained, and aligned with safety requirements.