Anthropic researcher resigns, stating that AI may go out of control next year

Anthropic researcher Jacob Coxon announced his resignation and is preparing to leave the AI industry. He has conducted pre-training research at OpenAI and Anthropic over the past three years. He pointed out that neither company has responsibly advanced AI and are competing to create superintelligence that can continue to enhance its own capabilities, "betting our lives on it."

He switched from OpenAI to Anthropic this year because the latter places more emphasis on model safety. Coxon acknowledged that Anthropic's safety work is serious, but believes that as long as companies are racing to be the first to create AGI (artificial general intelligence that surpasses humans in a wide range of tasks), safety compromises are unavoidable, and self-discipline from a single company cannot solve the problem. His assessment of the two companies is: many people inside OpenAI still do not truly realize how great this risk is; Anthropic knows the risks but believes others are unreliable, so they must create it themselves first.

Coxon stated that some AI practitioners are much more pessimistic privately than they express publicly, and are even worried that AI could destroy humanity before the end of this decade. He personally judges that in the most radical scenario, AI could go out of control by the end of next year, so such decisions should not be made solely by the engineers of a few companies.