OpenAI recently discovered and banned Russian and Iranian accounts using its artificial intelligence tools, including ChatGPT, for influence operations in multiple countries. According to internal reports produced using ChatGPT, the operatives managed to get their false narratives picked up by news outlets and even prompted a response from a member of the European Parliament. OpenAI investigated the impact of these campaigns and found that those focused on placing false narratives in established media outlets appeared to have reached a wider audience.
The Russian operations primarily targeted countries in Latin America, aiming to undermine support for Ukraine and influence local politics. The accounts mostly used ChatGPT to write reports detailing their work to unspecified higher-ups. One report described the widest-reaching influence operation OpenAI has come across, which led to a public response from a political figure. The Russian operatives impersonated a regional educational authority in Peru, tricking schools into organizing activities about Ukraine that featured Stepan Bandera, a controversial figure.
The impersonation led to a Peruvian outlet, Extra, reporting that one such event took place at a Peruvian school. A Polish news report about the school event prompted a far-right Polish member of the European Parliament to suggest banning the entry of people who professed support for Bandera. OpenAI noted that the Russian fake both fed on and fed into historical tensions between Ukrainians and Poles. According to Darren Linvill, a co-director of Clemson University's Watt Family Innovation Center Media Forensics Hub, Russians constantly engage in active measures, trying to create or spread propaganda that aligns with Russia's worldview.
The Russian internal memos created with ChatGPT claimed that the operatives ran a fake think tank in Latin America and recruited unwitting researchers to interview experts and compose reports. OpenAI said this effort was similar to a 2020 Russian operation posing as an independent news outlet linked to the late oligarch Yevgeny Prigozhin's Internet Research Agency, which was involved in efforts to influence the 2016 U.S. election. In other instances, the Russian operatives took credit for events they were not responsible for.
While the Russian operatives primarily used AI to update their superiors, Iranian operatives used AI to help them push out content in Persian and English. The Iranian campaign mainly focused on the U.S.-Iran war and, to a lesser degree, American politics. OpenAI said the operation's activity appeared consistent with a commercial actor running a for-hire influence campaign but was unable to identify who that was.
According to OpenAI, Iranian accounts created seven fake authors to pitch and polish long articles to small media outlets around the world, publishing almost 100 articles. The fake authors published articles in outlets including the progressive U.S. news site Daily Kos and the U.K.-based Middle East Monitor. The Middle East Monitor published at least 21 articles by four fake authors, which have since been taken down.
Experts noted that AI was valuable to the operatives, helping them create language that looked and felt authentic to the point where it was being published by legitimate outlets. Linvill said that a few years ago, an Iranian operation like this would have struggled to create such convincing content. OpenAI's discovery highlights the ongoing efforts of foreign actors to use AI tools for influence operations and the need for continued monitoring and mitigation of these threats.
Key points
- OpenAI caught and banned Russian and Iranian accounts using ChatGPT for influence operations.
- The Russian operations targeted Latin America to undermine support for Ukraine.
- Iranian operatives used AI to publish almost 100 articles in small media outlets.
