Improving Model-Based Reinforcement Learning by Disentangling the Latent Representation of the Environment
Keywords
Loading...
Authors
Issue Date
2019-07-12
Language
en
Document type
Journal Title
Journal ISSN
Volume Title
Publisher
Title
ISSN
Volume
Issue
Startpage
Endpage
DOI
Abstract
This thesis explores to what degree model-based reinforcement learning can benefit from recent
advances in representation learning. Specifically we measure the impact that the amount of featureentanglement
within the learned representation of the environment influences overall model
performance. We train a total of 45 (variational) autoencoders on a custom box-physics environment,
varying the relative influence of the Kullback-Leibler divergence term on the encoders loss. For each of
these models, we measure the amount of feature entanglement in their latent representations using
the measures proposed in Higgins et al. (2017) and Eastwood, C., & Williams, C. K. (2018). These
disentanglement scores will then be evaluated against the loss of a recurrent LSTM network that was
pre-trained on sequences of environment encodings, generated by the relevant autoencoder. -- We
find that less entangled representations of the environment significantly increase the accuracy of the
recurrent model and that this effect is even stronger for larger latent spaces.
Description
Citation
Supervisor
Faculty
Faculteit der Sociale Wetenschappen