OpenAI has launched a new search engine based on ChatGPT, which the company says allows publishers to control the use of their content by specifying access parameters in a ‘robots.txt’ file on their websites. In contrast to the initial rollout of ChatGPT two years ago, when publishers learned that OpenAI had harvested their content without notice or consent to train its underlying models, this may seem like an improvement. However, Tow Centre's research has shown that the chatbot often misquotes sources, despite the company's claims of accuracy. This raises concerns about the reliability of the new search tool and its potential impact on publishers. Here's a list of the problems that Tow Centre found.

CONFIDENTLY WRONG
The researchers tested ChatGPT's ability to accurately identify the source of quotes. The results showed that the chatbot often provided incorrect answers even when it could not find the exact source. Unlike traditional search engines (e.g. Google or Bing), ChatGPT rarely admitted its inability to find an answer, replacing it with fictitious data. This can be misleading to users and damaging to publishers' reputations, as their content can be misattributed. This behaviour of the chatbot underscores the need to critically evaluate information received from artificial intelligence.

COPYCAT SOURCES
OpenAI claims to be able to find quality content, but the study showed that the chatbot often links to plagiarised or syndicated versions of articles, ignoring the original sources. For example, instead of quoting the original New York Times article, ChatGPT linked to a website that copied it. This raises concerns about the quality of the data sources used by ChatGPT and its ability to accurately attribute authorship. Such behaviour can lead to the dilution of publishers' brands, as their content may be presented without proper recognition, and encourage plagiarism.

UNPREDICTABLE (MIS)ATTRIBUTION
ChatGPT can provide different answers to the same query due to the model's configuration, which allows it to generate different text options. This means that the chatbot can answer correctly once and then make a mistake the next time. For example, when asked the same question about the source of a quote, ChatGPT first pointed to the wrong website and then found the article correctly. Such instability in responses can lead to inconsistent and inaccurate results, which undermines trust in the accuracy of the information provided by the chatbot.

THE ILLUSION OF CONTROL
The study showed that ChatGPT often inaccurately cited sources, even if publishers allowed access to their sites. Regardless of whether the publishers cooperated with OpenAI or not, their content was often presented incorrectly. This shows that the availability of the scanner does not guarantee the accuracy of citations. Even publications that have signed licensing agreements with OpenAI faced the problem of inaccurate citations. This underscores the need for further research to understand the causes of such errors and to develop more effective ways to ensure the accuracy of information generated by artificial intelligence. 

So, to summarise the above problems of the ChatGPT search tool, it should not be used for information search, especially for research. We don't want to underestimate the usefulness of AI in the life of a researcher. For example, ChatGPT can be used to structure your thoughts. Sometimes it can be difficult to formulate your thoughts into readable sentences, and AI can help with this. But remember that AI can help you to arrange your thoughts-word into beautiful and understandable sentences for the reader, and not write a publication for you. Don't forget to mention the fact that you used AI (whether it was for calculations, analysis, or image generation for research materials, or for text and literature design) in the research methods section.

Source: https://www.cjr.org/tow_center/how-chatgpt-misrepresents-publisher-content.php