AI Chatbots Rely on Sources With Clear Biases

Commentary

AI chatbots such as ChatGPT and Grok can be a big help in writing essays, conducting research, and exploring complex issues. But these tools bring risks, especially when they filter facts through a political lens. And the Trump administration is now stepping into the debate.

“We believe AI systems should operate free of ideological bias and avoid pushing socially engineered agendas,” David Sacks, the administration’s AI and crypto czar, said in a recent statement. “We’ve introduced several proposals to ensure AI stays truth-seeking and trustworthy.”

Over the weekend, I saw this bias unfold in real time.

On Aug. 1, a user on Elon Musk’s platform X asked Grok whether more guns make Americans safer.

Grok responded flatly: “No, evidence shows more guns correlate with higher firearm homicides and violent crime rates.”

The chatbot dismissed self-defense and deterrence, referring to my research—specifically my “more guns, less crime” theory—as something cited by “right-wing advocates.” Grok supported its claims by referencing Scientific American magazine and a RAND Corp. review, saying these sources show that guns don’t reduce crime and instead increase violence.

Those answers are misleading and wrong.

The Scientific American article had extensive biases. Grok ignored my published rebuttal in Scientific American. In it, I noted that more than two-thirds of peer-reviewed studies show that concealed carry laws do reduce crime. Melinda Wenner Moyer, a journalist affiliated with Michael Bloomberg’s The Trace, a well-known gun control advocacy outlet, wrote the article. I had provided Moyer with those studies while she prepared her piece, but she ignored them. She failed to acknowledge any of my post-1998 work and misrepresented the findings of the National Research Council’s major report on the topic.

Grok gave tremendous weight to RAND’s literature survey, claiming that RAND had surveyed 100-plus studies. Eventually, Grok conceded that the number of papers studying right-to-carry laws was actually 25, showing a range of mixed results. I pointed out that the California-based think tank was highly selective in the sources it included, ignoring dozens more papers showing that these laws lowered violent crime rates and surveys of academics who have published peer-reviewed empirical research.

Even then, Grok largely ignored my responses and focused on two papers claiming that right-to-carry laws increased violent crime. The first failed to control for any variables—such as changes in policing, poverty, or economic conditions—that affect crime trends after adopting right-to-carry laws. When I pointed that out, Grok mentioned another study that demonstrated a statistical technique that could account for such factors, but that study didn’t look at right-to-carry laws. Only after a prolonged exchange did Grok acknowledge the error.

The second paper Grok emphasized made a subtler mistake: It compared states that had recently adopted right-to-carry laws with states that had adopted such laws years earlier. The early adopters made it easier to obtain permits and saw much larger increases in concealed handgun permits during the period studied. Comparing later adopters—who saw smaller increases—with these early states skewed the results. If crime didn’t fall as much in the newer states, the flawed analysis made it look as if crime had risen. Again, only after I cited my own peer-reviewed studies from 2022 and 2024 did Grok acknowledge the issue.

When Grok argued that more guns lead to more firearm homicides, I asked it to name any country that banned all guns or handguns and saw homicide rates fall—or even stay the same.

Grok cited Australia, Great Britain, and Brazil, but none of those examples are accurate.

Australia never banned all guns or handguns. Firearm homicides had already been falling for 15 years before the 1997 buyback and fell more slowly afterward. Meanwhile, gun ownership actually increased and by 2010 had surpassed pre-buyback levels.

In Britain, handgun bans enacted in 1997 preceded a 50 percent surge in homicide rates over the next seven years. The rates didn’t decline until the government boosted the police force by 14 percent over two years. Even then, homicide rates took 14 years to return to pre-ban levels.

Brazil didn’t ban all guns or handguns either. While its 2003 gun control law included a boost in law enforcement resources, murder rates remained largely unchanged. Only after President Jair Bolsonaro took office in January 2019—liberalizing gun ownership and increasing legal gun ownership by 650 percent—did Brazil’s homicide rate drop by more than 30 percent.

Only after I laid out these facts did Grok concede, calling them “fair points” and then echoing the very arguments I had just made.

My experience with Grok is not unique. To study the chatbots’ political biases, the Crime Prevention Research Center, which I head, asked various AI programs questions on crime and gun control last year in March and again in August and ranked the answers on how progressive or conservative their responses were. The chatbots tilted to the left, claiming that things like higher arrest and conviction rates don’t deter crime and clearly supporting more gun control laws.

AI chatbots speak with certainty but often rely on sources with clear biases. They cite selective evidence, misrepresent or don’t understand complex findings, and ignore reputable research that challenges a politically convenient narrative. AI chatbots also hallucinate, meaning they sometimes completely make up facts.

Students, journalists, and everyday citizens increasingly rely on these tools. If they accept chatbot responses at face value, they risk walking away with a fundamentally distorted view of issues like gun policy.

From RealClearWire

Views expressed in this article are the opinions of the author and do not necessarily reflect the views of The Epoch Times.