Learning Shared Representations in a Multimodal Deep Boltzmann Machine

Abstract

This Bachelor's thesis is about the ability of a multimodal Deep Boltzmann Ma- chine (DBM) to form and use shared representations across two modalities. Such shared representations are relevant for advanced robotics as well as for understand- ing the human brain, for instance that of synesthetes. To investigate the formation and usage of shared representations I implemented a multimodal DBM in Python and let it perform image reconstruction tasks under di erent experimental condi- tions. My results suggest that a multimodal DBM may not be well-suited to form and use shared representations.