Understanding and recognizing when a user has questions regarding an artificial agent’s behaviour

Thumbnail Image
Issue Date
Journal Title
Journal ISSN
Volume Title
The purpose of this thesis is to investigate if outlier detection is a suitable approach for detecting confusion in humans caused by an artificial agent’s behaviour. To answer this question, data has been collected in the form of video recordings which were used to train and analyse our outlier detection algorithm. As the data are video recordings they are data streams which provide unique challenges. To deal with these challenges, our outlier detection algorithm extracts the relevant information from each frame of the recordings and analyses them using auto-encoders. The auto-encoders learn from these incoming frames and attempt to recreate them. By measuring how well a frame can be recreated and computing the reconstruction cost, we can classify whether a frame is an inlier or an outlier. Since confusing events are rare, they fall in the outlier category and the algorithm is unable to learn how to recreate them resulting in higher reconstruction costs. The classification is based on the reconstruction cost, Chebyshev’s inequality and the exponentially weighted moving average model. The results of the experiments show that outlier detection does show promise for identifying confusing events as our algorithm outperforms random chance. However, further research should be done to optimize the algorithm and develop a better understanding of what information other than the recordings is required to be able to distinguish between outliers that are generated by confusing events and outliers which are caused by non-confusing events.
Faculteit der Sociale Wetenschappen