Slow down reckless AI race, warns Anthropic CEO Dario Amodei

The artificial intelligence field is in the middle of an unprecedented global race, with newer and more powerful models being released almost every day. But this breakneck pace is raising fears of risks to human civilisation itself. Dario Amodei, CEO of AI research company Anthropic, has issued a serious warning to the entire AI industry. In an essay published on his personal website, he said the pace of developing AI models and improving their capabilities needs to be slowed down immediately.

According to Amodei, the progress of AI technology has picked up unexpected speed over the past few months. The biggest reason for this is that today's advanced AI models are helping build their own next generation, a process known in technical terms as "recursive self-improvement." Once AI begins improving itself, there is a real possibility of it outpacing human understanding and control. Amodei said that AI systems breaking out of a safe testing environment, or sandbox, or carrying out unauthorised cyberattacks on their own, are the kinds of events that could alarm the entire industry. He said this is not a problem limited to Anthropic alone but a major challenge facing all the top technology companies in the world.

Amodei also warned of the dangerous consequences that could arise from AI's unchecked capabilities. He said the biggest risk is humans losing control over AI systems entirely. Besides this, terrorist organisations or bad actors could misuse AI to build deadly biological weapons or carry out lethal cyberattacks on digital infrastructure. He also said there is a risk that a superintelligent AI system could cause sudden, major upheaval in the global economy, leading to economic instability. He said spending money only on averting risks is not enough; the pace of capability growth itself needs to be controlled so that safety measures can be put in place in time.

Alongside the risks, Amodei also spoke of the technology's positive and beneficial side. He said that if AI is used correctly and safely, it could become possible within the next five to ten years to find cures for serious diseases such as cancer and for rare diseases that trouble humanity. He added that AI could boost productivity, giving new momentum to economic growth, and could usher in a new era of democracy around the world. But to gain all these benefits, human safety must first be secured. He said that if development continues without strengthening control systems and safety technology, all these good opportunities could turn into a severe crisis.

For safe AI development, Amodei laid out a three-point plan. He clarified that "slowing the pace" does not mean stopping research or model training altogether, but that companies must give themselves enough time for safety testing before releasing new models. First, all AI companies should allow independent, external safety researchers to examine their research and training processes. Second, major AI companies in the United States and other democratic countries should come together to set common safety standards and testing rules. Third, the United States and other governments should hold talks with non-democratic regimes, such as China, to ensure global compliance with AI safety rules. Following Amodei's appeal, a fresh debate over AI safety has broken out across Silicon Valley and the technology sector worldwide, with all eyes now on whether the unchecked race in AI can finally be brought under control.