Abstract:
Previous studies on gestural enhancement have examined word intelligibility in noise vocoded
speech, or in one linguistic masker (e.g., Drijvers & Özyürek, 2017; Schubotz et al., 2020), and
speech-in-speech research has typically performed unimodal studies (e.g., Brouwer et al., 2012;
Van Engen, 2010). The current study combines these two fields to investigate multimodal
communication in a more ecologically valid manner. In particular, this thesis investigated to
what extent iconic co-speech gestures help sentence intelligibility in two different linguistic
maskers. Thirty-two native Dutch participants performed a Dutch sentence recognition task in
which they were presented with videos in which an actress uttered short Dutch sentences (e.g.,
Ze begint te openen ‘She starts to open’). Participants were presented with a total of six audiovisual conditions: two gesture conditions (gestures absent vs. gestures present) and three masker
conditions (clear vs. French 2-talker babble vs. Dutch 2-talker babble), and were asked to type
down what was said by the Dutch actress. The focus of the study was on the accurate
identification of the action verbs at the end of the target sentences. The results demonstrated
that performance on the task was better in the gesture compared to the non-gesture conditions.
In addition, sentence intelligibility was better in French than in Dutch babble. The results and
implications of this project may be valuable to everyone who engages in multimodal
communication, and especially to a public who often works in public places where competing
speech is present in the background.
