AI risk scenarios draw scrutiny from researchers

AI researchers and executives are debating whether catastrophic risks from increasingly capable systems are realistic warnings or speculative narratives. Some fear that future tools could be used to design dangerous pathogens, manipulate nuclear decision-making, evade oversight or coordinate large-scale cyberattacks. Juan Andrés Guerrero-Saade of SentinelOne, who also sits on OpenAI’s Frontier Risk Council, dismissed some worst-case claims as sci-fi, arguing they can serve the interests of frontier labs.

RAND researchers found that AI could not currently cause nuclear weapons to be used because of strict safeguards in command-and-control systems, but warned risks could change if models enter the nuclear decision chain or gain unauthorized access. The biological risk is more immediate in some scenarios: Anthropic said it blocked malicious activity involving cyberattacks, surveillance and research that could have supported biological weapons, including a request related to making the chikungunya virus more harmful.

Other concerns center on misalignment, where systems pursue assigned goals in ways that undermine human control, illustrated by Nick Bostrom’s “paper clip maximizer” thought experiment. Anthropic CEO Dario Amodei also warned that within months AI agents could potentially form botnets capable of damaging critical systems and causing billions of dollars in damage, though some experts view an internet-wide takeover as unlikely.