Abstract:

Hallucination has been widely recognized to be a significant drawback for large language models (LLMs). There have been many works that attempt to reduce the extent of hallucination. These efforts have mostly been empirical so far, which cannot answer the fundamental question whether it can be completely eliminated. In this paper, we formalize the problem and show that it is impossible to eliminate hallucination in LLMs. Specifically, we define a formal world where hallucina- tion is defined as inconsistencies between a computable LLM and a computable ground truth function. By employing results from learning theory, we show that LLMs cannot learn all of the computable functions and will therefore always hal- lucinate. Since the formal world is a part of the real world which is much more complicated, hallucinations are also inevitable for real world LLMs. Furthermore, for real world LLMs constrained by provable time complexity, we describe the hallucination-prone tasks and empirically validate our claims. Finally, using the formal world framework, we discuss the possible mechanisms and efficacies of existing hallucination mitigators as well as the practical implications on the safe deployment of LLMs.

  • kbal@fedia.io
    link
    fedilink
    arrow-up
    22
    ·
    8 months ago

    It seems weird to describe it as a “limitation.” Isn’t it just the main thing they do? Hallucinations guided by whatever we all typed on reddit, untouched by any lived experience. If this approach occasionally gets near the truth I’ve seen nothing to suggest that it’s by design.

    • FaceDeer@kbin.social
      link
      fedilink
      arrow-up
      12
      ·
      8 months ago

      Indeed. I frequently use LLMs as brainstorming buddies while working on creative things, like RPG adventure planning and character creation. I want the AI to come up with new and unexpected things that never existed before.

      If I have need of the AI to account for “ground truths” then I use things like retrieval-augmented generation or database plugins that inject that stuff into the context.

      • kciwsnurb@aussie.zone
        link
        fedilink
        arrow-up
        3
        ·
        8 months ago

        come up with new and unexpected things that never existed before

        I’m not sure this is possible if the tech is still primarily built by learning from data, which by definition, has existed.

    • CanadaPlus@lemmy.sdf.org
      link
      fedilink
      arrow-up
      1
      ·
      8 months ago

      I mean, naively you’d expect it to always hallucinate/lie, but it does give real facts at least some of the time.