Efficient Management of LLM-Based Coaching Agents' Reasoning While Maintaining Interaction Quality and Speed
Journal
Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’25)
Type
conference paper
Date Issued
2025
Author(s)
Research Team
IWI6
Abstract
LLM-based agents improve upon standalone LLMs, which are optimized for immediate intent-satisfaction, by allowing the pursuit of more extended objectives, such as helping users over the long term. To do so, LLM-based agents need to reason before responding. For complex tasks like personalized coaching, this reasoning can be informed by adding relevant information at key moments, shifting it in the desired direction. However, the pursuit of objectives beyond interaction quality may compromise this very quality. Moreover, as the depth and informativeness of reasoning increase, so do the number of tokens required, leading to higher latency and cost. This study investigates how an LLM-based coaching agent can adjust its reasoning depth using a discrepancy mechanism that signals how much reasoning effort to allocate based on how well the objective is being met. Our discrepancy-based mechanism constrains reasoning to better align with alternative objectives, reducing cost roughly tenfold while minimally impacting interaction quality.
Language
English
Keywords
Behavior Change
Education/Learning
Text/Speech/Language
Artifact or System
Quantitative Methods
HSG Classification
contribution to scientific community
Refereed
Yes
Publisher place
Yokohama, Japan
Pages
18
Event Title
Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’25)
Event Location
Yokohama, Japan
Event Date
01.05.2025
Subject(s)
Division(s)
File(s)![Thumbnail Image]()
Name
JML_1017.pdf
Size
1.89 MB
Format
Adobe PDF
Checksum (MD5)
80edd982808ac7bc33b021380b32a6ea