THE EFFICIENCY OF ADVERSARIAL STATE EMBEDDINGS IN MODEL-BASED RL TASKS THE EFFECTS OF USING LATENT SPACES OF ADVERSARIAL NETWORKS TO ENCODE VISUAL INPUT FOR THE WORLD MODELS ARCHITECTURE
Keywords
Loading...
Authors
Issue Date
2020-02-12
Language
en
Document type
Journal Title
Journal ISSN
Volume Title
Publisher
Title
ISSN
Volume
Issue
Startpage
Endpage
DOI
Abstract
Unravelling all the information received from our surroundings is key to understand and interact with the
outside world. The outcome of this operation determines the performance of any other step in the reasoning
processes, thus making it a crucial element of any agent's learning experience. This thesis explored
how di erent representations of visual information a ected the ability of the World Models architecture
to nd an optimal policy using online, o ine and hybrid training procedures. To that end, we replaced its
original perception module with other alternatives that, with the same neural architecture, imposed different
biases and de ned di erent elements of interest. We attempted to promote disentanglement using
-VAE, and let the model de ne important high-level features in an adversarial fashion using VAE-GAN.
We proved that VAE-GAN can be an alternative to traditional autoencoders when encoding visual input
in a reinforcement learning setup. Not only that, but this technique improved the nal performance of
several of our con gurations. To the best of our knowledge, this was the rst piece of work that has ever
used an adversarial architecture to encode sensory input for a reinforcement learning task. Additionally, we
were able to test theWorld Models architecture on a new training procedure that alternated both training
in the real world and inside the model's imagination. Unfortunately, we could not directly encourage the
creation of disentangled latent spaces under the current con guration, but we still provided a qualitative
analysis of this characteristic for all our approaches.
Keywords Visual embeddings; knowledge representation; model-based reinforcement learning;
generative adversarial networks; World Models; VAE; VAE-GAN
Description
Citation
Supervisor
Faculty
Faculteit der Sociale Wetenschappen