Exploiting Feature Relations and External Knowledge in Multimedia Event Detection

Keywords
No Thumbnail Available
Issue Date
2018-12-14
Language
en
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Detection of complex events in diverse Internet videos is a very important step towards making sense of all the terabytes of video information uploaded every minute. The detection of such high-level semantics in the video can help with tasks ranging from security (automated surveillance system) to video search engines. We are testing the application of a fusion strategy that takes into account feature and class relationships by applying a weight regularization. As further extension we propose a weight initialization that adds prior expected values for the covariance between layer weight matrices. Using this initialization we can enforce an initial hierarchy/interdependency for the output labels and/or input features. The hierarchy of the labels can be calculated using external knowledge from WordNet however in our case we used the already calculated hierarchy of the corpus labels. We use a baseline of average fusion to compare our results for different fusion strategies. Furthermore we use different error functions to enforce the hierarchy of labels when the regularization isn’t applied. Finally we perform an analysis and visualization of the feature correlations before and after the regularization. Which shows that regularized features display a higher correlation than their un-regularized counterparts. This suggests that shared learning is imposed by the layer weight matrix covariances included in the regularization.
Description
Citation
Faculty
Faculteit der Sociale Wetenschappen