Using LLM for Conversational Assessment of Mental Health: Testing the Role of Message Contingency and User Agency

Doha Kim; Saejin Ju; Hui Min Lee; Migyeong Kang; Minseok Kim; Yoonvin Park; Hayeon Song; S. Shyam Sundar; Jinyoung Han

doi:10.1145/3772363.3799023

Using LLM for Conversational Assessment of Mental Health: Testing the Role of Message Contingency and User Agency

Doha Kim
, Saejin Ju
, Hui Min Lee
, Migyeong Kang
, Minseok Kim
, Yoonvin Park
, Hayeon Song
, S. Shyam Sundar
, Jinyoung Han

Film-Video and Media Studies

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Thanks to conversational AI, online mental health self-assessment tools enable users to describe symptoms in their own words, using natural language. However, prior work has focused mainly on performance quality of these tools, leaving user experience underexplored. In this work, we investigate how interaction design-specifically conversational contingency and user agency-shapes user experience of AI-based mental health assessment systems. We developed MIND, a mental health self-assessment chatbot based on a task-oriented dialogue system, and conducted a between-subjects experiment with 126 participants. Results show that conversational contingency significantly enhances social presence, trust, and perceived usefulness, while user agency primarily supports self-reflection and assessment credibility. Participants also valued empathetic responses and symptom-based dialogue, which enabled clearer self-expression and more positive engagement compared to fixed questionnaires. Overall, our findings reveal distinct roles of contingency and agency in shaping relational versus procedural trust.

Original language	English (US)
Title of host publication	CHI 2026 - Extended Abtracts of the 2026 CHI Conference on Human Factors in Computing Systems
Editors	Nuria Oliver, David A. Shamma, Heloisa Candello, Pablo Cesar, Pedro Lopes, Valentino Artizzu, Fiona Draxler, Gustavo Lopez, Anke V. Reinschluessel, Xin Tong, Phoebe O. Toups Dugas
Publisher	Association for Computing Machinery
ISBN (Electronic)	9798400722813
DOIs	https://doi.org/10.1145/3772363.3799023
State	Published - Apr 13 2026
Event	Extended Abtracts of the 2026 CHI Conference on Human Factors in Computing Systems, CHI 2026 - Barcelona, Spain Duration: Apr 13 2026 → Apr 17 2026

Publication series

Name	Conference on Human Factors in Computing Systems - Proceedings

Conference

Conference	Extended Abtracts of the 2026 CHI Conference on Human Factors in Computing Systems, CHI 2026
Country/Territory	Spain
City	Barcelona
Period	4/13/26 → 4/17/26

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

All Science Journal Classification (ASJC) codes

Human-Computer Interaction
Computer Graphics and Computer-Aided Design
Software

Access to Document

10.1145/3772363.3799023

Cite this

Kim, D., Ju, S., Lee, H. M., Kang, M., Kim, M., Park, Y., Song, H., Sundar, S. S., & Han, J. (2026). Using LLM for Conversational Assessment of Mental Health: Testing the Role of Message Contingency and User Agency. In N. Oliver, D. A. Shamma, H. Candello, P. Cesar, P. Lopes, V. Artizzu, F. Draxler, G. Lopez, A. V. Reinschluessel, X. Tong, & P. O. Toups Dugas (Eds.), CHI 2026 - Extended Abtracts of the 2026 CHI Conference on Human Factors in Computing Systems Article 714 (Conference on Human Factors in Computing Systems - Proceedings ). Association for Computing Machinery. https://doi.org/10.1145/3772363.3799023

Kim, Doha ; Ju, Saejin ; Lee, Hui Min et al. / Using LLM for Conversational Assessment of Mental Health : Testing the Role of Message Contingency and User Agency. CHI 2026 - Extended Abtracts of the 2026 CHI Conference on Human Factors in Computing Systems. editor / Nuria Oliver ; David A. Shamma ; Heloisa Candello ; Pablo Cesar ; Pedro Lopes ; Valentino Artizzu ; Fiona Draxler ; Gustavo Lopez ; Anke V. Reinschluessel ; Xin Tong ; Phoebe O. Toups Dugas. Association for Computing Machinery, 2026. (Conference on Human Factors in Computing Systems - Proceedings ).

@inproceedings{78720f53f5d44193a2646a15ee85a9d9,

title = "Using LLM for Conversational Assessment of Mental Health: Testing the Role of Message Contingency and User Agency",

abstract = "Thanks to conversational AI, online mental health self-assessment tools enable users to describe symptoms in their own words, using natural language. However, prior work has focused mainly on performance quality of these tools, leaving user experience underexplored. In this work, we investigate how interaction design-specifically conversational contingency and user agency-shapes user experience of AI-based mental health assessment systems. We developed MIND, a mental health self-assessment chatbot based on a task-oriented dialogue system, and conducted a between-subjects experiment with 126 participants. Results show that conversational contingency significantly enhances social presence, trust, and perceived usefulness, while user agency primarily supports self-reflection and assessment credibility. Participants also valued empathetic responses and symptom-based dialogue, which enabled clearer self-expression and more positive engagement compared to fixed questionnaires. Overall, our findings reveal distinct roles of contingency and agency in shaping relational versus procedural trust.",

author = "Doha Kim and Saejin Ju and Lee, \{Hui Min\} and Migyeong Kang and Minseok Kim and Yoonvin Park and Hayeon Song and Sundar, \{S. Shyam\} and Jinyoung Han",

note = "Publisher Copyright: {\textcopyright} 2026 Copyright held by the owner/author(s).; Extended Abtracts of the 2026 CHI Conference on Human Factors in Computing Systems, CHI 2026 ; Conference date: 13-04-2026 Through 17-04-2026",

year = "2026",

month = apr,

day = "13",

doi = "10.1145/3772363.3799023",

language = "English (US)",

series = "Conference on Human Factors in Computing Systems - Proceedings ",

publisher = "Association for Computing Machinery",

editor = "Nuria Oliver and Shamma, \{David A.\} and Heloisa Candello and Pablo Cesar and Pedro Lopes and Valentino Artizzu and Fiona Draxler and Gustavo Lopez and Reinschluessel, \{Anke V.\} and Xin Tong and \{Toups Dugas\}, \{Phoebe O.\}",

booktitle = "CHI 2026 - Extended Abtracts of the 2026 CHI Conference on Human Factors in Computing Systems",

}

Kim, D, Ju, S, Lee, HM, Kang, M, Kim, M, Park, Y, Song, H, Sundar, SS & Han, J 2026, Using LLM for Conversational Assessment of Mental Health: Testing the Role of Message Contingency and User Agency. in N Oliver, DA Shamma, H Candello, P Cesar, P Lopes, V Artizzu, F Draxler, G Lopez, AV Reinschluessel, X Tong & PO Toups Dugas (eds), CHI 2026 - Extended Abtracts of the 2026 CHI Conference on Human Factors in Computing Systems., 714, Conference on Human Factors in Computing Systems - Proceedings , Association for Computing Machinery, Extended Abtracts of the 2026 CHI Conference on Human Factors in Computing Systems, CHI 2026, Barcelona, Spain, 4/13/26. https://doi.org/10.1145/3772363.3799023

Using LLM for Conversational Assessment of Mental Health: Testing the Role of Message Contingency and User Agency. / Kim, Doha; Ju, Saejin; Lee, Hui Min et al.
CHI 2026 - Extended Abtracts of the 2026 CHI Conference on Human Factors in Computing Systems. ed. / Nuria Oliver; David A. Shamma; Heloisa Candello; Pablo Cesar; Pedro Lopes; Valentino Artizzu; Fiona Draxler; Gustavo Lopez; Anke V. Reinschluessel; Xin Tong; Phoebe O. Toups Dugas. Association for Computing Machinery, 2026. 714 (Conference on Human Factors in Computing Systems - Proceedings ).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Using LLM for Conversational Assessment of Mental Health

T2 - Extended Abtracts of the 2026 CHI Conference on Human Factors in Computing Systems, CHI 2026

AU - Kim, Doha

AU - Ju, Saejin

AU - Lee, Hui Min

AU - Kang, Migyeong

AU - Kim, Minseok

AU - Park, Yoonvin

AU - Song, Hayeon

AU - Sundar, S. Shyam

AU - Han, Jinyoung

PY - 2026/4/13

Y1 - 2026/4/13

N2 - Thanks to conversational AI, online mental health self-assessment tools enable users to describe symptoms in their own words, using natural language. However, prior work has focused mainly on performance quality of these tools, leaving user experience underexplored. In this work, we investigate how interaction design-specifically conversational contingency and user agency-shapes user experience of AI-based mental health assessment systems. We developed MIND, a mental health self-assessment chatbot based on a task-oriented dialogue system, and conducted a between-subjects experiment with 126 participants. Results show that conversational contingency significantly enhances social presence, trust, and perceived usefulness, while user agency primarily supports self-reflection and assessment credibility. Participants also valued empathetic responses and symptom-based dialogue, which enabled clearer self-expression and more positive engagement compared to fixed questionnaires. Overall, our findings reveal distinct roles of contingency and agency in shaping relational versus procedural trust.

AB - Thanks to conversational AI, online mental health self-assessment tools enable users to describe symptoms in their own words, using natural language. However, prior work has focused mainly on performance quality of these tools, leaving user experience underexplored. In this work, we investigate how interaction design-specifically conversational contingency and user agency-shapes user experience of AI-based mental health assessment systems. We developed MIND, a mental health self-assessment chatbot based on a task-oriented dialogue system, and conducted a between-subjects experiment with 126 participants. Results show that conversational contingency significantly enhances social presence, trust, and perceived usefulness, while user agency primarily supports self-reflection and assessment credibility. Participants also valued empathetic responses and symptom-based dialogue, which enabled clearer self-expression and more positive engagement compared to fixed questionnaires. Overall, our findings reveal distinct roles of contingency and agency in shaping relational versus procedural trust.

UR - https://www.scopus.com/pages/publications/105038109197

UR - https://www.scopus.com/pages/publications/105038109197#tab=citedBy

U2 - 10.1145/3772363.3799023

DO - 10.1145/3772363.3799023

M3 - Conference contribution

AN - SCOPUS:105038109197

T3 - Conference on Human Factors in Computing Systems - Proceedings

BT - CHI 2026 - Extended Abtracts of the 2026 CHI Conference on Human Factors in Computing Systems

A2 - Oliver, Nuria

A2 - Shamma, David A.

A2 - Candello, Heloisa

A2 - Cesar, Pablo

A2 - Lopes, Pedro

A2 - Artizzu, Valentino

A2 - Draxler, Fiona

A2 - Lopez, Gustavo

A2 - Reinschluessel, Anke V.

A2 - Tong, Xin

A2 - Toups Dugas, Phoebe O.

PB - Association for Computing Machinery

Y2 - 13 April 2026 through 17 April 2026

ER -

Kim D, Ju S, Lee HM, Kang M, Kim M, Park Y et al. Using LLM for Conversational Assessment of Mental Health: Testing the Role of Message Contingency and User Agency. In Oliver N, Shamma DA, Candello H, Cesar P, Lopes P, Artizzu V, Draxler F, Lopez G, Reinschluessel AV, Tong X, Toups Dugas PO, editors, CHI 2026 - Extended Abtracts of the 2026 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery. 2026. 714. (Conference on Human Factors in Computing Systems - Proceedings ). doi: 10.1145/3772363.3799023

Using LLM for Conversational Assessment of Mental Health: Testing the Role of Message Contingency and User Agency

Abstract

Publication series

Conference

UN SDGs

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this