Communicating Emotions: Multi-modal emotion recognition and the experience of emotional intensity

Simple item page
dc.contributor.advisorVries, G de
dc.contributor.advisorJanssen, J.
dc.contributor.advisorHaselager, W.F.G.
dc.contributor.authorTacken, P.E.P.
dc.date.issued2011-07-13
dc.description.abstractIn the past years various studies have focussed on automated emotion recognition using various modalities (video, audio, physiology) separately. The work we present comprises an investigation of the usefulness of each of these separate modalities and various modality combinations, for automated emotion recognition. Video, audio, and physiology signals were recorded from persons telling about an emotional event from their life, for five separate emotions: happy, relaxed, sad, angry and neutral. Features were extracted for each modality separately. The data from the three modalities was merged using feature-level fusion. Classification was done for each modality separately and for every modality combination, using a Support-Vector Machine (SVM) and a MultiLayer Perceptron (MLP). When looking at the classification performances for each modality separately, physiology showed promising results, especially for the SVM classifier. For both classifiers, there was an overall improvement in classification performance when modalities were merged instead of used separately. For automated emotion recognition, physiology appeared to be a quite influential modality. In order to investigate whether this could also be the case in human-human interaction, we conducted an experiment on the effect of a physiological signal on the judgment of the intensity of someone's emotional state. Participants were shown videos of persons telling about an emotional event (angry or neutral) from their life. Each video was accompanied by a heartbeat sound (slow or fast). Participants were told that the heartbeats were those of the persons in the videos at the moment they were telling the story. The participants were given the task to judge the emotional intensity of the persons the videos. Results showed that both faster heart rates and angry facial expressions caused higher emotional intensity ratings when compared to slower heart rates and neutral facial expressions. These results show the potential for a whole new way of communicating emotions.en_US
dc.embargo.lift10000-01-01
dc.identifier.urihttp://theses.ubn.ru.nl/handle/123456789/192
dc.language.isoenen_US
dc.thesis.facultyFaculteit der Sociale Wetenschappenen_US
dc.thesis.specialisationMaster Artificial Intelligenceen_US
dc.thesis.studyprogrammeArtificial Intelligenceen_US
dc.thesis.typeMasteren_US
dc.titleCommunicating Emotions: Multi-modal emotion recognition and the experience of emotional intensityen_US
Files
Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
Tacken_P_MA_Thesis_2011.pdf
Size:
704.24 KB
Format:
Adobe Portable Document Format
Description:
Scriptietekst
Download
Collections
Faculteit der Sociale Wetenschappen