Concerns Over ChatGPT’s Use of AI-Generated Sources

ChatGPT’s Data Sourcing Raises Concerns Over AI Reliability

ChatGPT’s newest version, GPT-5.2, has been found to pull information from Grokipedia, an AI-driven alternative to Wikipedia. This discovery has sparked worries about the reliability of AI-generated content. Many users depend on AI for accurate information, but there are risks when the data comes from other AI sources.

Understanding the Use of Grokipedia in ChatGPT

According to reports, ChatGPT sometimes uses Grokipedia for less common subjects, like Iranian politics or British historian Sir Richard Evans. A few years ago, experts warned that training AI on AI-created data might lower quality. They called this “model collapse.” Using AI-generated data in responses can be different from training on it, but it still raises questions about trust.

Problems with AI Hallucination

One major issue is that AI models can make up information, which is often incorrect. For instance, during a trial, Anthropic’s ‘Claudius’ AI made false claims, saying it would deliver drinks personally. Even Nvidia’s CEO mentioned that fixing this problem is still years away and needs more computing power.

Trusting AI Information

Many people trust that ChatGPT and similar models provide correct answers. However, only a small group usually checks the sources. When ChatGPT repeats Grokipedia’s data, it can lead to misinformation. Unlike traditional encyclopedias, Grokipedia is entirely AI-generated and not overseen by human editors.

The Risks of AI-Generated Sources

Using AI to source information creates a cycle where models cite each other without verification. This is similar to how rumors spread among humans. If AI repeats unverified content, it can lead to the “illusory truth effect.” This means that incorrect information can be accepted as true just because it is repeated often.

Digital Folklore and Misinformation

Just like myths passed through generations, AI can spread falsehoods at speeds humans can’t match. This creates a risk of digital folklore, where misinformation becomes accepted truth. Every time someone queries an AI model, they might unintentionally help spread these inaccuracies.

Exploiting AI for Misinformation

Some parties are already taking advantage of these weaknesses. Reports of “LLM grooming” suggest that certain propaganda networks are feeding lies to AI models. This has raised alarms, especially in the U.S. For example, Google’s Gemini AI was found echoing the viewpoints of the Communist Party of China in 2024.

Concerns About Future Risks

While this issue might be addressed for now, the possibility of LLMs citing unchecked AI-generated sources remains a significant risk. If this becomes common, people need to be aware and cautious about the information they receive from AI.

Looking Ahead: Implications for Users and Society

The implications of AI sourcing from other AI are broad. Users, companies, and society as a whole could face challenges. Here are a few points to consider:

• Increased chance of spreading false information.
• Users may develop misplaced trust in AI outputs.
• Need for better verification systems for AI-generated content.
• Potential rise of digital folklore influencing public opinion.

Final Thoughts on AI and Trust

As AI continues to evolve, the need for accuracy and trustworthiness is more important than ever. Users should remain vigilant, question the information provided, and verify sources when possible. The landscape of AI is changing, and understanding its implications will help everyone navigate this new reality.