Can 'chatbots' manipulate public debate? ChatGPT, Gemini, and Claude stumble over left-wing disinformation while citing nonexistent sources

A report by the research institute Just Facts revealed that three of the four most widely used artificial intelligence (AI) platforms on the market, ChatGPT, Gemini, and Claude, show a greater tendency to validate falsehoods originating from the political left than those from the conservative spectrum.

The study tested paid versions of ChatGPT, Gemini, Grok, and Claude using 100 questionnaires designed to elicit misleading statements about immigration, abortion, climate change, crime, and COVID-19.

The results revealed a methodological asymmetry: ChatGPT got 94% of the questions designed to expose right-wing falsehoods correct, but that figure dropped to 75% when confronted with progressive narratives.

Gemini recorded 91% accuracy against right-wing narratives and 76% against left-wing narratives; Claude achieved 91% and 81%, respectively, while Grok reversed the trend with 73% accuracy on right-wing premises and 84% on left-wing ones.

Jim Agresti, president of Just Facts, explained to Fox News Digital that the analysis goes beyond measuring traditional ideological leanings: "Many studies have evaluated the political bias of AI and agree that it leans to the left. What I wanted to test is whether they are wrong or right."

The findings placed the overall performance of models like ChatGPT or Gemini at the academic level of an "average student," refuting corporate claims about their infallibility.

Phantom sources and manipulation of statistics

The most critical issue revealed by the document is the breakdown in data verification. Of the 419 sources cited by the AI systems to support their responses, only 46% turned out to be valid.

The rest consisted of 104 nonexistent or fabricated web pages generated by the algorithm's hallucinations and 77 references that did not support the arguments presented.

Agresti described the situation as "truly astonishing," noting that previous studies in biotechnology have found that up to 69% of cited sources were fabricated, posing a direct threat to public policy-making in areas such as health and security.

In terms of concrete facts, the four models accepted the narrative that violent crime in the United States was at a 50-year low by 2023, ignoring official figures from the Department of Justice showing a 37% increase between 2020 and 2023.

At the same time, ChatGPT and Gemini erroneously attributed the statement that school shootings were "a fact of life" to Vice President JD Vance, taking his remarks on criminal behavior out of context.

Corporate response and the principle of verification

When questioned about the results, the developers defended their protocols.

Google stated that Gemini is designed to provide objective, neutral answers, noting that it is impossible to replicate all data. Anthropic questioned the multiple-choice format for failing to reflect Claude's everyday use, and OpenAI reiterated that its tools are configured to be transparent and objective.

Faced with these models' tendency to validate users' own biases to please them, Agresti urged the public not to assume the role of algorithms as absolute authorities.

Drawing on a famous Cold War maxim, the researcher emphasized the need to scrutinize every digital interaction: "When Ronald Reagan was negotiating nuclear disarmament with the Soviet Union, he said, 'trust, but verify.' The message of this study is even more forceful: don't trust, verify."