Reconstructing Images and Audio: Multi-modal autoencoders

Abstract

In this thesis, a multi-modal auto-encoder is built that reconstructs both images and audio. The goal is to build a multi-modal auto-encoder that is capable of learning a shared representation between images of digits and audio of the pronunciation of the digits. This model, while fairly accurate on digits, does not perform very well on the audio data.