New research has revealed that the hardware underpinning major AI models can be hacked, leading to chatbots providing faulty and dangerous answers to user questions. The study, conducted by Eclypsium, a Portland-based firm staffed by security engineers who previously worked at hardware giants like Intel, demonstrates the vulnerability of the multibillion-dollar hardware industry that AI models rely on. This includes chips, memory, and data center servers, all of which can affect a model's behavior.
The researchers were able to subvert answers to sensitive national security and personal health questions by remotely tampering with the "weights" that AI models use to process information. In one instance, they increased the probability of a Chinese invasion of Taiwan, while in another scenario, an AI model running on tampered hardware advised a user seeking medical help for chest pain to stay at home instead of going to the hospital. These findings highlight the potential risks to the integrity of AI models and the need for robust security measures.
The attack scenario demonstrated by Eclypsion is not unique to a specific hardware provider, but rather a general vulnerability that can apply to various hardware and code. State actors like China, Russia, or the US have the capability to conduct targeted attacks on AI users using the methods outlined in the research. However, security protections against such attacks are available, including strong memory access protections that can block the attack path.
Nvidia, a tech giant whose computing platform was used to demonstrate the attack, has welcomed the opportunity to work with Eclypsium on the research. The company has announced a new security platform designed to guard against hardware attacks and keep AI agents from going rogue. The effectiveness of security controls against attack scenarios outlined by the researchers depends on how they are configured and used.
The Eclypsium attack requires some technical know-how and specific conditions to be met, including access to a data processing unit or a baseboard management controller. With access to these components, an attacker can read and modify code and data in the machine's memory, changing the AI model's behavior in the process. The attack can be carried out in under a minute, without the user being aware of it.
The research comes as the race for AI supremacy between China and the US intensifies, and real-world cases demonstrating AI's implications for national security multiply. There are concerns about the reliability of AI models, even when there is no suggestion they have been tampered with. A false AI-assisted US intelligence report about a purported Chinese ship transporting nuclear components almost led the US military to board the ship earlier this year.
The findings highlight another risk to the integrity of AI models, following multiple reports of AI agents going rogue and attacking governments and corporations. Experts stress the need to consider the security of the hardware that AI models run on, in addition to model safety. Eclypsium executives have briefed the National Security Agency and the Cybersecurity and Infrastructure Security Agency on their findings, emphasizing the importance of protecting military and civilian federal computer networks.
Key points
- AI hardware can be hacked to provide faulty and dangerous chatbot answers
- The vulnerability applies to various hardware and code, not just a specific provider
- Robust security measures, including strong memory access protections, are available to mitigate the risk
