Jacob Coxon, a former researcher at both Anthropic and OpenAI, sent a grim warning that neither company was acting responsibly in the artificial intelligence race, arguing that the AI firms are “gambling with our lives.”
“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon wrote in a post on X, after announcing he had resigned from Anthropic.
Evan Hubinger, who currently heads up alignment science at Anthropic, said “we really do earnestly believe AI could kill all humans,” adding he thinks there is a greater than 10 percent chance this occurs within the next decade.
Hubinger thinks that Anthropic is trying its best, but does not have a plan to solve the dangerous alignment issues AI companies are faced with.
In the AI industry, alignment refers to the work by AI developers to ensure that the system behaves in accordance with human values and intentions.
Samuel Marks, another Anthropic employee, agrees that AI developers believe human extinction or other bad outcomes are a risk over the next few years, but stays at Anthropic in hopes that his work will reduce that risk
“Why do AI developers continue despite the risk?” Marks wrote on X. “Due to a mixture of commercial incentives and a belief that they are in a race with other, less responsible AI developers that will abuse the technology or develop it less safely.”