Early Action Ascription in Uni-Modal and Multi-Modal Communication

Abstract

The tight time constraints of the turn-taking system (Levinson, 2013) compel listeners to recognize the action conveyed by a turn and plan their response accordingly while the turn is still unfolding (Bögels, Magyari, & Levinson, 2015; Gisladottir, Chwilla, & Levinson, 2015). One fundamental task that listeners have to accomplish is to distinguish between questions, that is utterances that request information, and statements, which deliver information (Heritage, 2013). In order to ascribe meaning—including questionhood—to a turn, speakers can rely on grammar (Couper-Kuhlen, 2014), but also visual cues (e.g., Borràs-Comes, Kaland, Prieto, & Swerts, 2014). This study investigated the early recognition of questions and the role bodily information plays in this context. In a comprehension task, subjects had to classify spontaneously produced utterances from conversational corpora that were either questions or statements. These were presented auditorily or audio-visually in a gating paradigm (Grosjean, 1980), in order to gain insights about the time-course of action ascription. Stimulus presentation started with a 20-frame segment (equating 400 ms) and incrementally increased by another 20 frames at each pass. Subjects were able to correctly classify questions above chance level as early as the end of the first 20-frame-segment. The modality of presentation did not alter the results. The present findings suggest that listeners may rely overwhelmingly on lexico-syntactic information for the ascription of action, at least when distinguishing between question turns and statements. However, they can do so on the basis of partial information, before the end of the utterance. These findings support the early speech act recognition account (Gisladottir et al. 2015), making it possible for listeners to plan their turn while the incoming turn is still unfolding (Levinson, 2013).