The growing concern about the potential dangers of advanced artificial intelligence may be distracting from the real and immediate harms it is causing, particularly in the form of exacerbating gender-based violence and misogyny. In South Africa, the issue has taken on a new level of urgency, with experts warning that AI is being used to automate and amplify abuse, driving women and people of diverse sexual orientations and gender identities out of public life.
The recent murders of nine women in Ekurhuleni, Gauteng, have prompted Amnesty International South Africa to call on the state to recognize femicide as a distinct crime. Research has confirmed that norms around gender equality play a significant role in driving intimate partner femicide, and that these norms are increasingly shaped on social media and in online spaces promoting misogynistic narratives.
AI is changing the scale of gendered harms, according to a new research report by the Association for Progressive Communications. The report notes that coordinated harassment campaigns or gender disinformation campaigns can now be run at volumes and with a persistence that was previously not possible. AI also lowers the technical threshold for abuse, with generative AI able to produce sexualized content from a single photo.
The infrastructure of AI itself is not neutral, with AI systems learning from training data that reproduces harmful stereotypes and racist ideas. Social media and AI are increasingly bound together in a closed loop, with content generated by AI, amplified by AI-driven algorithms, and moderated by AI. These engagement-optimized algorithms push sensational material, while automated tools fail to recognize gendered harm, especially in Global South contexts.
The lack of diversity in the AI industry is also a major concern, with only 22% of the world's AI professionals being women and hardly any women leading major social media companies. This means that standards are shaped by those who have historically defined what counts as harm, rather than by those who bear the brunt of abuse.
Double standards are also at play, with AI-generated, non-consensual images remaining online while platforms shadow-ban content by women and LGBTQIA+ people. This abuse drives women out of public life, with a UN Women study finding that 70% of women journalists, activists, and human rights defenders surveyed had experienced online violence.
Solutions do not require rejecting technology, but rather a redesign of power structures and accountability measures. AI offers real opportunities to support survivors and strengthen institutions meant to protect them. Regulation that focuses on content removal while leaving profitable revenue models untouched will fall short, and gender perspectives and harm analysis must be built into AI policy from the start.
Key points
- AI technology is exacerbating gender-based violence and misogyny in South Africa by automating and amplifying abuse.
- The lack of diversity in the AI industry and double standards in AI-generated content are major concerns.
- Solutions require a redesign of power structures and accountability measures, rather than rejecting technology.
