”I thought having a robot would make my life easier mister”: How Users Hold LLM-Chatbots Accountable for Errors
Keywords
Loading...
Authors
Issue Date
2025-12-19
Language
en
Document type
Journal Title
Journal ISSN
Volume Title
Publisher
Title
ISSN
Volume
Issue
Startpage
Endpage
DOI
Abstract
Since the release of ChatGPT, millions interact daily with chatbots that are prone to present incorrect information as fact. Using Ethnomethodology and Conversation Analysis, this thesis analyses naturally occurring human-LLM interactions from the Wildchat-1M dataset where users identify and orient to chatbot errors in real time.
The analyses highlight four accountability practices: counterinformings (challenging false claims), known answer requests (testing competence), ascriptive B-event questions (seeking error explanations), and direct complaints.
Based on the analyses, two theoretical contributions are proposed: (1) LLM chatbots as a form of "socioinstrument", that users simulatanously orient to as both a malfunctioning tool and socially accountable agent; (2) "performative accountability"—how LLMs mirror users' accountability orientations without underlying understanding or memory, producing misleading responses.
More broadly, the findings demonstrate how the meanings of LLM errors emerge sequentially through the user's orientation to the error as a more or less accountable action and the ensuing system responses that tend to mirror the user's orientation.
Description
Citation
Supervisor
Faculty
Faculteit der Letteren