Chatbot Agents Displaying Non-factive Reasoning Enhance Expectation Confirmation

Current guidelines suggest setting low expectations for chatbots for continued engagement. That is of little use if only high expectations entice users. Such are raised by recent innovations, chatbot agents which generate final responses not directly via Large Language Models but after multiple intermediate generations of text as reasoning. This differentiates chatbot agents from other chatbots. Since these intermediate steps are generated, what impact has displaying them? This study aims to assess the impact of displaying these intermediate reasoning steps, conceptualizing them as enabling mindreading in the semi-literal sense, namely reading the internal reasoning generated by the agent. In 3 studies (N=280), we examine whether non-factive displays, which present reasoning as belief rather than knowledge, enhance user expectation confirmation. Results show that non-factive reasoning displays significantly improved confirmation. Therefore, displaying non-factive reasoning not only differentiates these agents in competitive markets but also improves user interaction.

Current guidelines suggest setting low expectations for chatbots for continued engagement. That is of little use if only high expectations entice users. Such are raised by recent innovations, chatbot agents which generate final responses not directly via Large Language Models but after multiple intermediate generations of text as reasoning. This differentiates chatbot agents from other chatbots. Since these intermediate steps are generated, what impact has displaying them? This study aims to assess the impact of displaying these intermediate reasoning steps, conceptualizing them as enabling mindreading in the semi-literal sense, namely reading the internal reasoning generated by the agent. In 3 studies (N=280), we examine whether non-factive displays, which present reasoning as belief rather than knowledge, enhance user expectation confirmation. Results show that non-factive reasoning displays significantly improved confirmation. Therefore, displaying non-factive reasoning not only differentiates these agents in competitive markets but also improves user interaction.