Users trust AI and human fact-checkers equally, but for different reasons

UNIVERSITY PARK, Pa. — Users tend to trust artificial intelligence (AI)-powered fact-checkers as much as human fact-checkers, but for different reasons, according to a new study led by Penn State researchers. The researchers said there is no definitive “winner” when comparing the two fact-checking systems, because users see distinct strengths and weaknesses in each.

In their study published in Media Psychology, participants tended to trust AI more for large-scale scanning tasks, like identifying “red flags” in social media posts. They trusted humans for more nuanced fact-checking that requires piecing together evidence or interpreting complicated situations.

“There's a very clear distinction that emerges from the study that AI is considered good at low-level linguistic features, like identifying telltale signs that something is not credible,” said author S. Shyam Sundar, Evan Pugh University Professor and James P. Jimirro Professor of Media Effects at Penn State. “Humans are seen as being better at corroborating evidence from multiple sources.”

The research team first conducted a pretest to identify six news headlines that varied in credibility. Two hundred and ninety-one participants residing in the United States were then shown those headlines in simulated social media posts via an application created for this study called FactDeck. Some posts were labeled as fact-checked by an AI system and others by human fact-checkers. Participants saw one of three types of explanations:

• Evidence-based: The system labeled the post false with a reference to the information that contradicted the post.
• Feature-based: The system flagged suspicious wording or unusual phrasing.
• “Black box”: No explanation was given for why the post was marked false.

The researchers focused on “machine heuristics” — mental shortcuts people use when evaluating AI, based on stereotypes about machines. They found that while participants assumed AI systems were objective and accurate, they also distrusted AI for lacking human judgment. When it came to determining which system garnered the most trust, first author Mengqi Liao said the two opposite perspectives offset each other.

“Some studies only compare AI versus human fact-checkers, to find out which is trusted more,” said Liao, assistant professor at the University of Georgia, who completed her doctoral studies with Sundar at Penn State. “They get a lot of inconsistent results. That’s why we proposed a competing hypothesis that showed how positive and negative views of both can coexist and cancel each other out.”

Liao added that users preferred some explanation rather than none — the “black box” option.

“We want to provide enough explanation to users that helps them better understand how the system makes a specific decision. It may help them calibrate their trust,” Liao said. “They're not just relying on the system’s decision. They can also make a judgment based on how the system reached the decision.”

The findings suggest that effective fact-checking tools should not only provide accurate results but also explain how those results are reached. Liao said that fact-checking programs should help people recognize what AI systems are good at and what they're not, rather than relying on users’ own naïve, outdated notions of machine capabilities.

Sundar said this is increasingly important as AI fact-checking becomes more necessary. Human fact-checking can’t keep up with the volume of misinformation on social media today.

“The ideal situation would be a human-AI collaboration, but it's not always possible for humans to intervene and check for evidence from multiple sources,” Sundar said. “So, we are going to have to, at some level, completely automate this whole fact-checking business, and rely on AI, which can be much better at efficiently sifting through evidence from multiple sources than humans, despite what people think.”

Sian Lee, assistant professor at the University of Mississippi, who earned a doctorate in informatics from Penn State in 2024; Annie Dooley, a doctoral student at the Ohio State University, who earned a master’s degree in media studies at Penn State; and Aiping Xiong, assistant professor of privacy and cybersecurity informatics at Penn State, were authors on the paper as well.