Designing for Conversational System Trustworthiness: The Impact of Model Transparency on Trust and Task Performance
Journal
European Conference on Information Systems (ECIS)
Type
conference paper
Date Issued
2022-06-24
Author(s)
Research Team
IWI6
Abstract
Designing for system trustworthiness promises to address challenges of opaqueness and uncertainty introduced through Machine Learning (ML)-based systems by allowing users to understand and interpret systems’ underlying working mechanisms. However, empirical exploration of trustworthiness measures and their effectiveness is scarce and inconclusive. We investigated how varying model confidence (70% versus 90%) and making confidence levels transparent to the user (explanatory statement versus no explanatory statement) may influence perceptions of trust and performance in an information retrieval task assisted by a conversational system. In a field experiment with 104 users, our findings indicate that neither model confidence nor transparency seem to impact trust in the conversational system. However, users’ task performance is positively influenced by both transparency and trust in the system. While this study considers the complex interplay of system trustworthiness, trust, and subsequent behavioral outcomes, our results call into question the relation between system trustworthiness and user trust.
Language
English
Keywords
Trust
Trustworthiness
Transparency
Machine Learning
Information Retrieval
Pedagogical Conversational Agents
HSG Classification
contribution to scientific community
Publisher place
Timișoara, Romania
Event Title
European Conference on Information Systems (ECIS)
Event Location
Timișoara, Romania
Event Date
18-24 Jun 2022
Subject(s)
Division(s)
Eprints ID
266811
File(s)![Thumbnail Image]()
open.access
Name
JML_874.pdf
Size
510.24 KB
Format
Adobe PDF
Checksum (MD5)
055f1c9895c45b962dc80b26fbd364de