Deepfake scams and automated attacks sharpen AI security concerns
North Korean hacking group BlueNoroff is using commercial AI tools, including GPT-4o and Grok Imagine, to create realistic face and voice deepfakes for Zoom scams targeting nearly 700 executives and founders across 60 countries. Separately, the nation-state threat actor Vapor Panda has used DeepSeek for automated cyberattacks against over 200 organizations since October 17.
AI assistants also face vulnerabilities and manipulation. OpenAI patched a ChatGPT flaw that allowed malicious code execution through shared chats and could enable account hijacking through a single link. Prompt adware embeds persistent hidden commands to take control of enterprise assistants. An investigation found that Gemini generated and stored prohibited images despite telling users their requests had been blocked.
Government responses include a US ban on DeepSeek in federal agencies, with exceptions for the Department of Defense and National Intelligence. The British regulator is developing a tool intended to identify electoral deepfake audio and video within three hours of publication. Separately, the UN has warned about AI taking over human decisions in warfare ahead of a Geneva conference on autonomous weapons.