The Effects of Iconic Gestures and Babble Language on Sentence Intelligibility
Keywords
No Thumbnail Available
Authors
Issue Date
2021-07-14
Language
en
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Previous studies on gestural enhancement have examined word intelligibility in noise vocoded
speech, or in one linguistic masker (e.g., Drijvers & Özyürek, 2017; Schubotz et al., 2020), and
speech-in-speech research has typically performed unimodal studies (e.g., Brouwer et al., 2012;
Van Engen, 2010). The current study combines these two fields to investigate multimodal
communication in a more ecologically valid manner. In particular, this thesis investigated to
what extent iconic co-speech gestures help sentence intelligibility in two different linguistic
maskers. Thirty-two native Dutch participants performed a Dutch sentence recognition task in
which they were presented with videos in which an actress uttered short Dutch sentences (e.g.,
Ze begint te openen ‘She starts to open’). Participants were presented with a total of six audiovisual conditions: two gesture conditions (gestures absent vs. gestures present) and three masker
conditions (clear vs. French 2-talker babble vs. Dutch 2-talker babble), and were asked to type
down what was said by the Dutch actress. The focus of the study was on the accurate
identification of the action verbs at the end of the target sentences. The results demonstrated
that performance on the task was better in the gesture compared to the non-gesture conditions.
In addition, sentence intelligibility was better in French than in Dutch babble. The results and
implications of this project may be valuable to everyone who engages in multimodal
communication, and especially to a public who often works in public places where competing
speech is present in the background.
Description
Citation
Supervisor
Faculty
Faculteit der Letteren