Conversational agents in healthcare: expert-informed definition, classification, and conceptual framework
Journal
JMIR Preprints
Type
working paper
Date Issued
2023
Author(s)
Laura Martinengo
;
Xiaowen Lin
;
Ahmad Ishqi Jabir
;
;
Rifat Atun
;
Josip Car
;
Lorainne Tudor Car
Abstract
Background: Conversational agents (CAs), or chatbots, are computer programs that simulate conversations with humans. The use of CAs in healthcare settings is recent and rapidly increasing, which often translates to poor reporting of the CA development and evaluation processes, and unreliable research findings. We developed and published a conceptual framework for Designing, developing, evaluating, and Implementing a Smartphone-delivered, rule-based COnVERsational agent (DISCOVER) consisting of three iterative stages of CA’s design, development, evaluation, and implementation, complemented by two cross-cutting themes (user-centered design and data privacy and security).
Objective: This study aimed to perform in-depth, semi-structured interviews with multidisciplinary experts in healthcare CAs to share their views on the definition and classification of healthcare CAs and evaluate and validate the DISCOVER conceptual framework.
Methods:
We conducted one-on-one, semi-structured interviews via Zoom with 12 multidisciplinary CA experts using an interview guide based on our framework. The interviews were audio recorded, transcribed by the research team, and analyzed
using thematic analysis. Results: Following participants input, we defined CAs as digital interfaces that use natural language to engage in a synchronous dialogue using one or more communication modalities such as text, voice, images, or video. CAs were classified into 13 categories: response generation method, input and output modalities, CA purpose, deployment platform, CA development modality, appearance, length of interaction, type of CA-user interaction, dialogue initiation, communication style, CA personality, human support, and type of healthcare intervention. Experts considered that the conceptual framework could be adapted for AI-based CAs. However, despite recent advances in AI, including large language models, the technology is not able to ensure safety and reliability in healthcare settings. Finally, aligned with participants’ feedback, we present an updated iteration of the Conceptual framework for Healthcare conversational AgenTs (CHAT) with key considerations for CA design, development, evaluation, and implementation, complemented by three cross-cutting themes, ethics, user involvement and data privacy and security.
Conclusions:
We presented an expanded, validated Conceptual framework for Healthcare conversational AgenTs (CHAT) aimed to guide researchers from a variety of backgrounds and expertise in the design, development, evaluation, and
implementation of rule based CAs in healthcare settings.
Objective: This study aimed to perform in-depth, semi-structured interviews with multidisciplinary experts in healthcare CAs to share their views on the definition and classification of healthcare CAs and evaluate and validate the DISCOVER conceptual framework.
Methods:
We conducted one-on-one, semi-structured interviews via Zoom with 12 multidisciplinary CA experts using an interview guide based on our framework. The interviews were audio recorded, transcribed by the research team, and analyzed
using thematic analysis. Results: Following participants input, we defined CAs as digital interfaces that use natural language to engage in a synchronous dialogue using one or more communication modalities such as text, voice, images, or video. CAs were classified into 13 categories: response generation method, input and output modalities, CA purpose, deployment platform, CA development modality, appearance, length of interaction, type of CA-user interaction, dialogue initiation, communication style, CA personality, human support, and type of healthcare intervention. Experts considered that the conceptual framework could be adapted for AI-based CAs. However, despite recent advances in AI, including large language models, the technology is not able to ensure safety and reliability in healthcare settings. Finally, aligned with participants’ feedback, we present an updated iteration of the Conceptual framework for Healthcare conversational AgenTs (CHAT) with key considerations for CA design, development, evaluation, and implementation, complemented by three cross-cutting themes, ethics, user involvement and data privacy and security.
Conclusions:
We presented an expanded, validated Conceptual framework for Healthcare conversational AgenTs (CHAT) aimed to guide researchers from a variety of backgrounds and expertise in the design, development, evaluation, and
implementation of rule based CAs in healthcare settings.
Division(s)