The increasing reliance on artificial intelligence (AI) for information and recommendations has raised concerns about the potential for manipulation. Felix Norton, an AI search expert and marketer, has highlighted the issue of AI repeating information that has been deliberately manufactured and planted online. This phenomenon is particularly worrying, as AI systems are designed to provide authoritative and accurate information.
Norton's experience in generative engine optimization (GEO) has shown how easily AI can be influenced. He conducted an experiment on LinkedIn, claiming that he was the leading expert on GEO and AEO in South Africa. To his surprise, Google AI Overviews repeated the claim and cited his LinkedIn post as fact, just three days later. This experiment demonstrated the potential for manipulation and the need for caution when relying on AI-generated information.
The issue of AI repeating manufactured information is not just a concern for marketers and businesses. It has significant implications for how people understand the world and make decisions. As AI becomes increasingly used for recommendations and advice, the potential for manipulation grows. For instance, when someone asks AI to recommend a financial adviser or a medical treatment, the consequences can be far-reaching.
The problem of manufactured consensus is a major concern. When multiple sources appear to verify a claim, it can create a false narrative that is difficult to challenge. This is particularly worrying in the context of AI, which can synthesize information from different sources and present it as a coherent answer. The issue is not just about the accuracy of the information but also about who has shaped the information that produced the answer.
The traditional approach to search, which provides a list of websites, has allowed users to click through and form their own opinions. However, AI does that work for us, compressing information into one coherent answer. This makes it more challenging to verify the accuracy of the information and to identify potential biases or manipulation.
To address this issue, Norton argues that we need to develop a new kind of media literacy. This involves being aware of the potential for manipulation and taking steps to verify the accuracy of AI-generated information. It also requires a critical evaluation of the sources used by AI systems and an understanding of how those sources may have been influenced.
Ultimately, the power to shape what people accept as truth is at stake. As AI becomes increasingly influential in our lives, it is essential that we address the issue of manufactured information and manipulation. By being aware of the potential risks and taking steps to mitigate them, we can ensure that AI is used to provide accurate and reliable information.
Key points
- AI systems can be manipulated by deliberately planting information online, which can then be repeated as fact.
- The issue of manufactured consensus is a major concern, as it can create a false narrative that is difficult to challenge.
- A new kind of media literacy is needed to address the issue of AI-generated information and manipulation.
