E cient Facial Expression Recognition in Everyday Photos

Weel, Wouter van der

E cient Facial Expression Recognition in Everyday Photos

Files

S4243773, WeelW 2018.pdf (13.46 MB)

Authors

Weel, Wouter van der

Issue Date

2019-03-01

Language

en

URI

https://theses.ubn.ru.nl/handle/123456789/10696

Abstract

Automatic facial expression recognition has been a much-researched topic over the past decade and even before. However, most studies have used simple, lab-controlled data. In practice, facial expressions can vary signi cantly from image to image, due to di erences in lighting, strength of the expression and pose, in addition to interpersonal di erences. Most studies that use more practical data also use very large neural networks that lead to ine cient image classi cation or allow the networks to get familiarize with the test data. This project aims to classify facial expressions as accurately as possible for use in an online application with user-uploaded data. As such, a much harder dataset than is usual needs to be used to emulate the data and there are strong constraints on image processing time, inference time, and network size. Four network structures (VGG16, MobileNet, Xception and a small, simple CNN) were used with the A ectNet dataset. A ectNet contains a large amount of human-labeled images of facial expressions found on the internet. To make classi cation easier, image pre-processing that aimed to make the data more uniform - keeping only expression-related information - has been applied. Each network was trained twice to compare the results of this pre-processing. Additionally, it is studied what the networks have learned about the data through optimal input generation. It is shown that facial expressions can be recognized quite robustly by most networks and there are only relatively small di erences in network accuracies. This is because the ambiguity in emotions creates much inconsistency in the labeled images, limiting any classi er's performance. The fact that the di erences in network speed and size are large means that a very small network is most e cient on this dataset. Pre-processing does not unambiguously increase network performance, while it does reduce processing speed.