Neural Decoding with Normalizing Flows
Keywords
Loading...
Authors
Issue Date
2021-05-01
Language
en
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
e pattern of light that reaches the retina is projected onto patterns of neural activity
via a cascade of re exive computations alongside six interconnected brain areas
in the occipital lobe called the visual ventral stream. e complex hierarchical
architecture of this visual streamallows for e cient processing of visual information
in the human brain. Machine learning has provided a powerful tool for encoding and
decoding between brain responses and perceptual content. is thesis probes the use
of generative ows as models of neural patterns from the perspective of decoding.
We introduce a Neural Flowmodel that can bijectivelymap between a topographical
interpretation of brain activity and the original input stimulus for a large-scale fMRI
dataset of the BBC series Doctor Who. is approach is unique, as it inherently
allows to combine neural encoding and decoding in a single neural network model.
Our results show that it is possible to use this model to reconstruct simulated brain
activity. For this, we applied the receptive eld estimators directly on the input
frames. is imitates a noise-free representation of brain data and mimics the
sparsity created within the encoding scheme. rough this, we show that the model
successfully achieves to extrapolate from missing spatial information. We further
nd through several ablation experiments on the simulation data that the model is
robust to sparse training data sizes and to sparse receptive eld information. Across
all ablation experiments, we nd strong positive correlations for the reconstructed
images. e reconstructions on actual brain data found in this thesis match the
previous benchmark results obtained on the same dataset. We achieve this, while at
the same time including activity from only single regions of interest of brain activity,
with leading performances for early visual areas VÕ and Vó. Overall, we nd that
our normalizing ow model succesfully allows to reconstruct brain activity, while
contributing a uni ed approach to neural encoding and decoding.
Description
Citation
Supervisor
Faculty
Faculteit der Sociale Wetenschappen