Home Innovation Chatbots AI Chatbots Struggle with Accu...

AI Chatbots Struggle with Accurate News Summarization, Study Reveals

Chatbots

Business Honor
28 Febuary, 2025

Study shows AI chatbots often provide inaccurate or skewed news summaries, raising concerns about reliability.

The popular AI Chatbots like ChatGPT, Google’s Gemini are growing in popularity but they are major weakness of summarizing new stories. According to a new study, the AI tools other than Apple also have trouble in summarizing the news for the users.

Existing data is being used to train AI models, and news media companies have partnered with companies like OpenAI to train these AI Chatbots. Even still, the AI summaries are inaccurate and, in some concerning situations, skewed.

The results came from a BBC investigation in which ChatGPT, Copilot, and Gemini were asked to summarize over 100 news items. The organization hired journalists from related professions to review and score these summaries.

To put it slightly, the findings were surprising. According to the study, more than half of the AI summaries had some serious content problems. Not only this, but also about Apple's AI summary feature, which was eventually disabled with the iOS 18.3 update, and 19% of these AI news pieces contained factual mistakes.

One thing that experts have been saying for a long regarding AI tools is made clear by the study as to slow down the rollout of AI and even remove features if they make mistakes. Google's picture generation tool was accused of manipulating image attributes with racial bias, and the company has also experienced its fair share of AI mistakes. Although research like this demonstrates that the features of AI are fascinating, their accuracy raises serious concerns, thus companies like Google or OpenAI should exercise caution in how they package these tools before making them publicly available.