AFRT Neural Encoding of Visual Perception using Affine Transformations

Kimman, Nils

AFRT Neural Encoding of Visual Perception using Affine Transformations

Files

Kimman, N. s-1007368-MSc-MKI94-Thesis-2023.pdf (3.61 MB)

Authors

Kimman, Nils

Issue Date

2023-01-01

Language

en

Abstract

State of the art neural encoding models do not explicitly model neuron’s receptive fields, which is inefficient and increases the likelihood of overfitting. Even though this can be fixed using heavy regularisation, a more elegant solution is to explicitly learn the receptive fields of the biological neurons the model is trying to model. Enter AFRT, which takes a standard neural encoding architecture and adds an affine layer at the start. This layer spatially transforms the input by cropping and shifting the input image to fit a certain feature of the image that the neuron is sensitive to. Next, the feature model, AlexNet, extracts features from the selected part of the image, which are then converted into brain response space using a single linear layer. This simple architecture is used to investigate the learned receptive fields of neurons in various layers of AlexNet. Besides matching performance of typical neural encoding models, model parameters are far reduced and more interpretable. Finally, AFRT is used to reinforce the idea that feature complexity of neuron’s receptive fields increases down the visual pathway, and that this idea is mimicked in Deep Neural Networks.