Exposing the face manifold with generative modeling Reconstructing perceived faces from brain activations
Keywords
No Thumbnail Available
Authors
Issue Date
2020-02-01
Language
en
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Neural encoding and neural decoding can shed light on (aspects of) the functional
organization of visual processing in the brain. Here, we introduce a new
framework for HYperrealistic reconstruction of PERceived naturalistic stimuli from
brain recordings (HYPER). To this end, we embrace the use of generative adversarial
networks (GANs) at the earliest step of our neural decoding pipeline by acquiring
functional magnetic resonance imaging data as subjects perceived face images
created by the generator network of a GAN. Subsequently, we used a linear decoding
approach to predict the latent state of the GAN from brain data. Hence, latent
representations that are needed for stimulus (re-)generation are obtained, leading
to ground-breaking image reconstructions. Furthermore, functional brain regions
that encode for five visual features: gender, age, eyeglasses, pose, and smile, are
localized using a multivariate searchlight, thereby identifying distributed networks
of activity to be predictive of visual semantics of perceived stimuli. Altogether, we
have developed a highly promising approach for decoding neural representations
of real-world data, which may pave the way for systematically analyzing neural
information processing in the functional brain.
Description
Citation
Supervisor
Faculty
Faculteit der Sociale Wetenschappen