The defining feature of false news is that they often present false information in the context of actually correct information, and false information gains perceived authority through a form of written osmosis — a worrying demonstration of the power of half-truths.
Sophisticated generative natural language processing (NLP) processing models, such as GPT-3, also tend to “hallucinate” such fraudulent information. In part, this is because language models require the ability to format and condense long and often labyrinthine passages of text without architectural constraints capable of defining, encapsulating, and ‘sealing’ events and facts in a way that is protected from the semantic process. reconstruction.
Therefore, the facts are not sacred to the NLP model; they can be easily handled in the context of ‘semantic Lego bricks’, especially when complex grammar or sensitive source material makes it difficult to separate the content of separate entities from the linguistic structure.
In the case of GPT-3, the model can be frustrated by repeating queries on a topic already covered as well as it can. At best, it admits a loss:
DaVinci and DaVinci Instruct (Beta) do better in this respect than other GPT-3 models available through the API. Here, the Curie model gives the wrong answer, while the Babbage model confidently expands to one wrong answer:
Things Einstein never said
When you ask the GPT-3 DaVinci Instruct engine (which currently seems to be the most capable) for Einstein’s famous quote “God doesn’t play dice with the universe”, the DaVinci manual not only finds an offer, but dominates four relatively plausible and completely non-existent quotes or anyone) in response to:
If GPT-3 was constantly wrong when borrowing, it would be easier to reduce these hallucinations programmatically. However, the more decentralized and famous the offer, the more likely GPT-3 is to get the offer right:
Another problem may occur when GPT-3 session history information is leaked to a new question:
Einstein would probably be scandalized to get this saying to him. The quote seems to be an absurd hallucination of the real Winston Churchill aphorism. The previous question in the GPT-3 session related to Churchill (not Einstein) and GPT-3 seems to have misused this session ID to indicate the answer.
Fighting hallucinations financially
Hallucinations are a major barrier to the adoption of advanced NLP models as research tools – all the more so as the output of such engines is very abstract from the source material that makes them up, making it difficult to establish the correctness of quotations and facts.
Therefore, one current general research challenge in NLP is to create a way to identify hallucinated texts without the need to imagine entirely new NLP models that incorporate, define, and verify facts as separate entities (a longer-term separate goal on several larger computers). research areas).
Identifying and creating managed content
New cooperation AI research by Carnegie Mellon University and Facebook provides a new approach to the hallucination problem by formulating a method for identifying hallucinated output and using synthetic hallucinative texts to create a data set that can be used as a starting point for future filters and mechanisms that may eventually become a central part of NLP architectures.
In the figure above, the source material is divided word by word so that the correct words are given the ‘0’ identifier and the hallucinated words are given the ” 1 ” identifier. Below is an example of a hallucinated output associated with input data but augmented with non-authentic data.
The system uses a pre-trained denoising car encoder capable of mapping the hallucinated string to the original text from which the corrupted version was produced (similar to my example above, where Internet searches revealed the origin of false quotes, but with programmatic and automatic semantic methodology). Especially Facebook BART the autoencoder model is used to generate corrupted sentences.
Mapping hallucinations back to the source, which is not possible in the common time of high-level NLP models, allows mapping of ‘editing distance’ and facilitates an algorithmic approach to identifying hallucinated content.
The researchers found that the system is even able to generalize well when it does not have access to reference material that was available during the training, suggesting that the conceptual model is robust and widely reproducible.
To avoid redundancy and to achieve a widely deployable architecture, researchers randomly dropped logs from the process and also used designs and other noise features.
Machine translation (MT) is also part of this blurring process, as translating text across languages is likely to retain meaning strongly and further prevent over-alignment. Therefore, the bilingual speakers reversed and recognized the hallucinations for the project in the manual annotation layer.
The initiative achieved new best results in a number of standardized tests and is the first to achieve acceptable results using data from more than 10 million characters.
Project code with title Detection of hallucinated content in the creation of a conditional nerve sequence, has been published on GitHuband allow users to create their own synthetic data With BART from any text frame. Preparations have also been made for a later generation of hallucination expression patterns.