OpenAI launched the ChatGPT Search feature in October, promising a web-based information retrieval revolution. The feature merged AI-powered conversational abilities with web search tech technologies to provide accurate, real-time information and links to credible sources. However, a new analysis by Columbia University's Tow Center for Digital Journalism questions these claims, pointing out severe weaknesses in the system's accuracy and reliability.
Struggling with Attribution
The Tow Center's research included an assessment of 200 quotes from 20 sources, and ChatGPT was offered the task of properly sourcing them. The findings were mixed, and some of them were concerning since they revealed irregularities in the sourcing technique. Although the chatbot occasionally correctly quotes the source, it frequently misattributes, references non-existent sources, or fabricates the information.
Surprisingly, similar concerns occurred even among OpenAI's licensed partners, including The Wall Street Journal and Politico. Despite OpenAI's claims that these license agreements will enhance the precision of ChatGPT's citations, the results revealed minimal improvement over non-partnered publications. For example, the chatbot often misidentified the speakers of quotes from Axel Springer-owned Politico, undermining the intended credibility boost from such partnerships.
Problems with Web Crawlers
One of the main problems is that ChatGPT gets its information using web crawlers. Publications like the New York Times are blocking these crawlers, worsening the problem. Additionally, in several situations, actual data was missing, and ChatGPT instead fabricated sources or copied black material from other sources like DNyuz, a well-known content aggregation website. The study discovered that more than one-third of the answers in these circumstances were incorrect, indicating that AI was out of the blue when data wasn't accessible.
When given access, the AI could sometimes reference the sources, but not always. This indecisiveness gradually undermines AI's credibility, mainly when a feature is marketed as a tool for quick and accurate content fetching.
Implications for Journalism and AI
These findings have significant broad implications. As journalism shifts towards a click-based business model, the potential risks posed by ChatGPT and similar programs are alarming. If AI-based tools misattribute their material or drive viewers to unverified sources, they may not only jeopardize the financial stability of legitimate news outlets but also the very fabric of journalism as we know it.
Additionally, people's dependence on AI systems for news collection raises several questions about whether the information is accurate. Already dealing with misinformation in the media world, the generative AI's willingness to 'invent' facts will only deepen the trust crisis in the digital content industry, potentially leading to a loss of credibility for AI-generated content and the platforms that host it.
The Road Ahead
OpenAI has promised to increase ChatGPT's inline citations and publisher preference. Still, until these flaws are resolved, users should exercise caution when using the chatbot's products, double-check information, and verify sources on their own.
The concept of AI-enhanced search remains appealing, but without a commitment to precision and transparency, GPT can become a tool that misdirects the user through the information cloud rather than explaining the problem.