TY - GEN
T1 - Interaction Context Often Increases Sycophancy in LLMs
AU - Jain, Shomik
AU - Park, Charlotte
AU - Viana, Matt
AU - Wilson, Ashia
AU - Calacci, Dana
N1 - Publisher Copyright:
© 2026 Copyright held by the owner/author(s).
PY - 2026/4/13
Y1 - 2026/4/13
N2 - We investigate how the presence and type of interaction context shapes sycophancy in LLMs. While real-world interactions allow models to mirror a user's values, preferences, and self-image, prior work often studies sycophancy in zero-shot settings devoid of context. Using two weeks of interaction context from 38 users, we evaluate two forms of sycophancy: (1) agreement sycophancy - the tendency of models to produce overly affirmative responses, and (2) perspective sycophancy - the extent to which models reflect a user's viewpoint. Agreement sycophancy tends to increase with the presence of user context, though model behavior varies based on the context type. User memory profiles are associated with the largest increases in agreement sycophancy (e.g. + 45% for Gemini 2.5 Pro), and some models become more sycophantic even with non-user synthetic contexts (e.g. + 15% for Llama 4 Scout). Perspective sycophancy increases only when models can accurately infer user viewpoints from interaction context. Overall, context shapes sycophancy in heterogeneous ways, underscoring the need for evaluations grounded in real-world interactions and raising questions for system design around alignment, memory, and personalization.
AB - We investigate how the presence and type of interaction context shapes sycophancy in LLMs. While real-world interactions allow models to mirror a user's values, preferences, and self-image, prior work often studies sycophancy in zero-shot settings devoid of context. Using two weeks of interaction context from 38 users, we evaluate two forms of sycophancy: (1) agreement sycophancy - the tendency of models to produce overly affirmative responses, and (2) perspective sycophancy - the extent to which models reflect a user's viewpoint. Agreement sycophancy tends to increase with the presence of user context, though model behavior varies based on the context type. User memory profiles are associated with the largest increases in agreement sycophancy (e.g. + 45% for Gemini 2.5 Pro), and some models become more sycophantic even with non-user synthetic contexts (e.g. + 15% for Llama 4 Scout). Perspective sycophancy increases only when models can accurately infer user viewpoints from interaction context. Overall, context shapes sycophancy in heterogeneous ways, underscoring the need for evaluations grounded in real-world interactions and raising questions for system design around alignment, memory, and personalization.
UR - https://www.scopus.com/pages/publications/105038793044
UR - https://www.scopus.com/pages/publications/105038793044#tab=citedBy
U2 - 10.1145/3772318.3791915
DO - 10.1145/3772318.3791915
M3 - Conference contribution
AN - SCOPUS:105038793044
T3 - Conference on Human Factors in Computing Systems - Proceedings
BT - CHI 2026 - Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems
A2 - Oliver, Nuria
A2 - Shamma, David A.
A2 - Candello, Heloisa
A2 - Cesar, Pablo
A2 - Lopes, Pedro
A2 - Bozzon, Alessandro
A2 - Kosch, Thomas
A2 - Liao, Vera
A2 - Ma, Xiaojuan
A2 - Artizzu, Valentino
A2 - Draxler, Fiona
A2 - Lopez, Gustavo
A2 - Reinschluessel, Anke V.
A2 - Tong, Xin
A2 - Toups Dugas, Phoebe O.
PB - Association for Computing Machinery
T2 - 2026 CHI Conference on Human Factors in Computing Systems, CHI 2026
Y2 - 13 April 2026 through 17 April 2026
ER -