The Cognitive Interplay between the Screen and the Hands
Keywords
Loading...
Authors
Issue Date
2017-08-24
Language
en
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Lausberg and Kita (2003) found that hand choice was determined by the spatial location of geometrical objects on screen. The left hand displayed the object on the left side of the screen, and the object on the right was referred to with the right hand. This paper aimed to investigate the influence of the movements and locations of entities in a cartoon video on hand choice and its concurrent speech. The cartoon video was more complex than Lausberg and Kita’s (2003) experiment in that it revolved around character’s attributes, actions, and events. The methodology consisted of participants retelling the events of a cartoon video to a recipient. Hand choice was thoroughly examined through a qualitative analysis in relation to the following motion features of the video: location, trajectory, viewpoint and manner. The research questions under scrutiny were: how is spatio-motoric information of the stimulus video represented in gesture? Which hands are used to depict the video’s motion properties? And furthermore, why do the hands depict these spatial features in this way? The whyquestion linked the findings to Kita and Özyürek’s (2003) Interface Hypothesis. It was found that (1) character and dual viewpoint gestures were sometimes narrated from observer perspective, (2) the location and trajectory of the hands tended to be similar to the spatial location and path of the protagonist on screen, and (3) manner gestures from observer viewpoint sometimes had a trajectory that did not correspond to the trajectory in the video. In light of the Interface Hypothesis, the Action Generator might alter the trajectory of such manner gestures. This cognitive alteration facilitates the classification of event sequences from each other, and could therefore be considered a mnemonic benefit for speakers to cognitively organise their narrative for the purpose of the cartoon retelling task.
Description
Citation
Supervisor
Faculty
Faculteit der Letteren