Exploiting Feature Relations and External Knowledge in Multimedia Event Detection
Keywords
No Thumbnail Available
Authors
Issue Date
2018-12-14
Language
en
Document type
Journal Title
Journal ISSN
Volume Title
Publisher
Title
ISSN
Volume
Issue
Startpage
Endpage
DOI
Abstract
Detection of complex events in diverse Internet videos is a very important step towards
making sense of all the terabytes of video information uploaded every minute. The detection
of such high-level semantics in the video can help with tasks ranging from security (automated
surveillance system) to video search engines. We are testing the application of a fusion strategy
that takes into account feature and class relationships by applying a weight regularization.
As further extension we propose a weight initialization that adds prior expected values for the
covariance between layer weight matrices. Using this initialization we can enforce an initial
hierarchy/interdependency for the output labels and/or input features. The hierarchy of the
labels can be calculated using external knowledge from WordNet however in our case we used
the already calculated hierarchy of the corpus labels. We use a baseline of average fusion to
compare our results for different fusion strategies. Furthermore we use different error functions
to enforce the hierarchy of labels when the regularization isn’t applied. Finally we perform an
analysis and visualization of the feature correlations before and after the regularization. Which
shows that regularized features display a higher correlation than their un-regularized counterparts.
This suggests that shared learning is imposed by the layer weight matrix covariances
included in the regularization.
Description
Citation
Faculty
Faculteit der Sociale Wetenschappen