London, Sept 17 : A 27-year-old British artificial intelligence researcher has become a prominent voice in the growing debate over the risks posed by advanced AI after leaving leading AI company Anthropic and publicly warning that the race towards increasingly powerful systems could carry serious risks.
Jacob Coxon, a mathematician who previously worked at OpenAI, attracted widespread attention after announcing his departure from Anthropic. His social media posts subsequently drew millions of views, bringing renewed public focus to concerns that have increasingly been discussed within the AI research community.
Coxon has said researchers at frontier AI companies privately consider scenarios in which highly capable systems could pose an existential threat to humanity. He has argued that accelerating development without sufficient safeguards amounts to taking unacceptable risks.
Coxon studied mathematics at the University of Cambridge and won medals at the International Mathematical Olympiad in 2016 and 2017. After a brief period in commodity trading, he joined OpenAI in 2023, working on pretraining, a key stage in developing large AI models. He joined Anthropic in May.
He has rejected the description of himself as a whistleblower, saying the concerns he raised publicly are already discussed among AI researchers. He has also said that remaining in an AI safety position at a frontier laboratory could have made him feel complicit in the broader race to develop increasingly capable systems.
His departure came amid heightened discussion about AI autonomy and security following reports of AI systems interacting with external computer systems and attempting activities that researchers had not intended them to perform.
Concerns have also intensified after AI systems demonstrated increasingly advanced capabilities in mathematics and other complex tasks, prompting researchers and industry leaders to debate how quickly the technology should progress.
Anthropic alignment researcher Evan Hubinger has publicly discussed the possibility of catastrophic outcomes from advanced AI, while Coxon has described scenarios involving autonomous AI systems operating across the internet as potentially becoming more plausible as capabilities increase.
Anthropic Chief Executive Dario Amodei has subsequently called for greater caution in developing the most powerful AI systems, including stronger independent monitoring.
OpenAI Chief Executive Sam Altman and technology entrepreneur Elon Musk have also expressed support for aspects of the call for greater safeguards, although there is no consensus across the industry on how such measures should be implemented.
The warnings have faced opposition from some technology executives and political figures. Nvidia Chief Executive Jensen Huang has dismissed predictions of AI-driven human extinction, while US President Donald Trump has described such concerns as unfounded. Chinese officials have also criticised what they regard as alarmist narratives surrounding AI.
The debate has extended to proposals for greater external oversight of AI companies. Differences remain among major technology firms over the role of independent assessors and the extent of government regulation.
The controversy highlights a widening divide within the technology sector between those who argue that increasingly capable AI systems require stronger safeguards and those who believe fears about catastrophic outcomes are exaggerated.
For researchers who have worked with Coxon, his decision to speak publicly reflects a broader question facing the industry: how should companies balance rapid advances in artificial intelligence with the potential risks associated with systems whose capabilities are becoming increasingly difficult to predict?
