Hyperrealistic neural decoding: Linear reconstruction of face stimuli from fMRI measurements via the GAN latent space
Keywords
No Thumbnail Available
Authors
Issue Date
2019-07-01
Language
en
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
We introduce a new framework for hyperrealistic reconstruction of perceived
naturalistic stimuli from brain recordings. To this end, we embrace the use of
generative adversarial networks (GANs) at the earliest step of our neural decoding
pipeline by acquiring functional magnetic resonance imaging data as subjects
perceived face images created by the generator network of a GAN. Subsequently,
we used a linear decoding approach to predict the latent state of the GAN from brain
data. Hence, latent representations that are needed for stimulus (re-)generation
are obtained, leading to ground-breaking image reconstructions. Altogether, we
have developed a highly promising approach for decoding neural representations
of real-world data, which may pave the way for systematically analyzing neural
information processing in the functional brain.
Description
Citation
Faculty
Faculteit der Sociale Wetenschappen