Getting Tuned to Others' Intentions: a model for hierarchical spatio-temporal pattern recognition

Thumbnail Image
Issue Date
Journal Title
Journal ISSN
Volume Title
The human ability to understand other’s actions and attribute mental states to those actions plays a prominent role in social interaction between humans. An understanding of this mindreading ability may result in new techniques which for example can be applied to the detection of threatening or abusive behavior in surveillance applications, or can be used to take human-robot interaction to a higher level. It is believed that visual perception in the human makes use of hierarchical decomposition, an idea that has gained significant popularity since the findings of simple and complex cells in the visual cortex of cats (Hubel and Wiesel, 1959) and macaque monkeys (Hubel and Wiesel, 1968). In this thesis it is investigated whether a previous approach to mental state inference, the Mental State Inference (MSI) model by Oztop et al. (2005), can be improved by adding hierarchical structure to the model. A novel model is presented which is based on the detection of biological motion in order to infer the mental states of others. This is achieved by a series of so called complex ‘tuning forks’ each of is tuned resonate with a specific input signal. The behavior of these tuning forks is defined so that these forks ‘resonate’ during the time the pattern, to which the forks have been ‘tuned’ to, is present in the incoming signal. I.e., tuning forks behave as pattern recognizers, and their output is defined as the degree to which the input corresponds to or resonates with their intrinsic pattern. It is these pattern recognizers that form the basic building block of the presented model. The model is based on hierarchical decomposition since the output of the pattern recognizers within the model, referred to as responsibility signals, is further analyzed by another series of pattern recognizers. This additional level of analysis allows us to detect primitive biological motions and detect the order in which these occur, which in turn allows us to combine primitive behaviors to create a representation of more complex behavior, and more specifically allows us to detect hierarchical sequential ordered behavior. In this study it is hypothesized that the introduction of the above-mentioned hierarchical decomposition principle can be used to infer the intentions of others, given a sufficient amount of pattern recognizer layers as described above. In order to verify whether such an approach is fruitful three experiments have been set up. In the first experiment it is shown that the model is able to detect patterns embedded in a random time series. The second experiment shows that the responsibility values computed in the first experiment can be used to detect a specific sequence of patterns, and at the same time shows that the model is not limited to pattern recognition in one dimensional time series but can also be used to detect patterns in multidimensional time series. The third experiment shows that the model is not only able to detect patterns in synthetically generated signals but can also be used to detect patterns in empirical data. Accumulating evidence shows that ‘action understanding may precede, rather than follow from, action mirroring‘ (Csibra, 2007). In line with this argument, the model presented in this thesis demonstrates that mental state inference is possible without the need of mental simulation, and is therefore compatible with recent findings as part of the functional role of the mirror neuron system and action understanding in general.
Faculteit der Sociale Wetenschappen