Fact or Fiction? Exploring Explanations to Identify Factual Confabulations in RAG-Based LLM Systems
Journal
Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’25)
Type
conference paper
Date Issued
2025-04-26
Author(s)
Research Team
IWI6
Abstract
The adoption of generative artificial intelligence (GenAI) and large language models (LLMs) in society and business is growing rapidly. While these systems often generate convincing and coherent responses, they risk producing incorrect or non-factual information, known as confabulations or hallucinations. Consequently, users must critically assess the reliability of these outputs when interacting with LLM-based agents. Although advancements such as retrieval-augmented generation (RAG) have improved the technical performance of these systems, there is a lack of empirical models that explain how humans detect confabulations. Building on the explainable AI (XAI) literature, we examine the role of reasoning-based explanations in helping users identify confabulations in LLM systems. An online experiment (n = 97) reveals that analogical and factual explanations improve detection accuracy but require more time and cognitive effort than the no explanation baseline.
Language
English
Keywords
Generative AI
RAG
Large Language Models
Confabulations
Hallucinations
GenXAI
HSG Classification
contribution to scientific community
Refereed
Yes
Publisher place
Yokohama, Japan
Pages
13
Event Title
Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’25)
Event Location
Yokohama, Japan
Event Date
26.04.2025
Subject(s)
Division(s)