The defining feature of false news is that they often present false information in the context of actually correct information, and false information gains perceived authority through a form of written osmosis — a worrying demonstration of the power of half-truths.

Sophisticated generative natural language processing (NLP) processing models, such as GPT-3, also tend to “hallucinate” such fraudulent information. In part, this is because language models require the ability to format and condense long and often labyrinthine passages of text without architectural constraints capable of defining, encapsulating, and ‘sealing’ events and facts in a way that is protected from the semantic process. reconstruction.

Therefore, the facts are not sacred to the NLP model; they can be easily handled in the context of ‘semantic Lego bricks’, especially when complex grammar or sensitive source material makes it difficult to separate the content of separate entities from the linguistic structure.

An observation of the way in which intricately crafted source material can confuse complex language models such as GPT-3. Source: Creating a paraphrase through deep reinforcement learning

This problem spreads from text-based machine learning part machine vision research, especially in areas that use semantic discrimination to identify or describe objects.

Hallucinations and inaccuracies

Hallucinations and inaccurate “cosmetic” new interpretations also affect computer vision research.

In the case of GPT-3, the model can be frustrated by repeating queries on a topic already covered as well as it can. At best, it admits a loss:

My recent experiment with a Davinc base engine in a GPT-3.  The model gets an answer right away on the first attempt, but is confused when asked a question a second time.  Because it retains the short-term memory of the previous answer and treats the repeated question as a rejection of the answer, it admits defeat.  Source: https://www.scalr.ai/post/business-applications-for-gpt-3

My recent experiment with a Davinci base engine in a GPT-3. The model gets an answer right away on the first attempt, but is confused when asked a question a second time. Because it retains the short-term memory of the previous answer and treats the repeated question as a rejection of the answer, it admits defeat. Source: https://www.scalr.ai/post/business-applications-for-gpt-3

DaVinci and DaVinci Instruct (Beta) do better in this respect than other GPT-3 models available through the API. Here, the Curie model gives the wrong answer, while the Babbage model confidently expands to one wrong answer:

Things Einstein never said

When you ask the GPT-3 DaVinci Instruct engine (which currently seems to be the most capable) for Einstein’s famous quote “God doesn’t play dice with the universe”, the DaVinci manual not only finds an offer, but dominates four relatively plausible and completely non-existent quotes or anyone) in response to:

GPT-3 produces four credible quotes from Einstein, none of which yield results at all in a full-text Internet search, although some trigger other (real) quotes from Einstein on the subject of “imagination”.

If GPT-3 was constantly wrong when borrowing, it would be easier to reduce these hallucinations programmatically. However, the more decentralized and famous the offer, the more likely GPT-3 is to get the offer right:

GPT-3 apparently finds the correct quotation marks when they are well represented in the accompanying data.

GPT-3 apparently finds the correct quotation marks when they are well represented in the accompanying data.

Another problem may occur when GPT-3 session history information is leaked to a new question:

Einstein would probably be scandalized to get this saying to him. The quote seems to be an absurd hallucination of the real Winston Churchill aphorism. The previous question in the GPT-3 session related to Churchill (not Einstein) and GPT-3 seems to have misused this session ID to indicate the answer.

Fighting hallucinations financially

Hallucinations are a major barrier to the adoption of advanced NLP models as research tools – all the more so as the output of such engines is very abstract from the source material that makes them up, making it difficult to establish the correctness of quotations and facts.

Therefore, one current general research challenge in NLP is to create a way to identify hallucinated texts without the need to imagine entirely new NLP models that incorporate, define, and verify facts as separate entities (a longer-term separate goal on several larger computers). research areas).

Identifying and creating managed content

New cooperation AI research by Carnegie Mellon University and Facebook provides a new approach to the hallucination problem by formulating a method for identifying hallucinated output and using synthetic hallucinative texts to create a data set that can be used as a starting point for future filters and mechanisms that may eventually become a central part of NLP architectures.

Source: https://arxiv.org/pdf/2011.02593.pdf

Source: https://arxiv.org/pdf/2011.02593.pdf

In the figure above, the source material is divided word by word so that the correct words are given the ‘0’ identifier and the hallucinated words are given the ” 1 ” identifier. Below is an example of a hallucinated output associated with input data but augmented with non-authentic data.

The system uses a pre-trained denoising car encoder capable of mapping the hallucinated string to the original text from which the corrupted version was produced (similar to my example above, where Internet searches revealed the origin of false quotes, but with programmatic and automatic semantic methodology). Especially Facebook BART the autoencoder model is used to generate corrupted sentences.

Defining a label.

Defining a label.

Mapping hallucinations back to the source, which is not possible in the common time of high-level NLP models, allows mapping of ‘editing distance’ and facilitates an algorithmic approach to identifying hallucinated content.

The researchers found that the system is even able to generalize well when it does not have access to reference material that was available during the training, suggesting that the conceptual model is robust and widely reproducible.

Rigging Overload

To avoid redundancy and to achieve a widely deployable architecture, researchers randomly dropped logs from the process and also used designs and other noise features.

Machine translation (MT) is also part of this blurring process, as translating text across languages ​​is likely to retain meaning strongly and further prevent over-alignment. Therefore, the bilingual speakers reversed and recognized the hallucinations for the project in the manual annotation layer.

The initiative achieved new best results in a number of standardized tests and is the first to achieve acceptable results using data from more than 10 million characters.

Project code with title Detection of hallucinated content in the creation of a conditional nerve sequence, has been published on GitHuband allow users to create their own synthetic data With BART from any text frame. Preparations have also been made for a later generation of hallucination expression patterns.

LEAVE A REPLY

Please enter your comment!
Please enter your name here