Abstract:
Automatic facial expression recognition has been a much-researched topic over the past decade and even
before. However, most studies have used simple, lab-controlled data. In practice, facial expressions can
vary signi cantly from image to image, due to di erences in lighting, strength of the expression and pose,
in addition to interpersonal di erences. Most studies that use more practical data also use very large
neural networks that lead to ine cient image classi cation or allow the networks to get familiarize with
the test data. This project aims to classify facial expressions as accurately as possible for use in an online
application with user-uploaded data. As such, a much harder dataset than is usual needs to be used to
emulate the data and there are strong constraints on image processing time, inference time, and network
size. Four network structures (VGG16, MobileNet, Xception and a small, simple CNN) were used with
the A ectNet dataset. A ectNet contains a large amount of human-labeled images of facial expressions
found on the internet. To make classi cation easier, image pre-processing that aimed to make the data
more uniform - keeping only expression-related information - has been applied. Each network was trained
twice to compare the results of this pre-processing. Additionally, it is studied what the networks have
learned about the data through optimal input generation. It is shown that facial expressions can be
recognized quite robustly by most networks and there are only relatively small di erences in network
accuracies. This is because the ambiguity in emotions creates much inconsistency in the labeled images,
limiting any classi er's performance. The fact that the di erences in network speed and size are large
means that a very small network is most e cient on this dataset. Pre-processing does not unambiguously
increase network performance, while it does reduce processing speed.
