Abstract:
In recent years, deep neural networks (DNNs) have come to dominate varied
domains in the field of machine learning. Particularly novel are accomplishments in
deep reinforcement learning and the successes of DNNs as feature models in neuroscientific
studies, e.g. in the case of functional magnetic resonance imaging (fMRI)
experiments investigating visual and auditory perception. This study represents
the juncture of these two branches and aims to locate mechanisms of perceptual
decision making by identifying the neural correlates of Deep Q Networks (DQNs).
In an fMRI experiment, 12 subjects played three conceptually different ATARI
video games for which DQNs have been shown to achieve human-level performance.
The Q-values and Hidden values of the DQN were used as feature regressors in a
representational similarity analysis, analyzing correlation with the blood oxygen
level dependant both at the scope of regions of interest and voxel searchlights.
The DQN generated features showed elevated correlations in occipital lobe. Actions
caused heightened correlations in both visual- and motor-related areas. Furthermore,
positive correlations were found in frontal lobe for Games. Albeit, statistical
significance could not be established for these correlations.
Qualitatively, neural correlates were identified for all regressors in line with current
neuroscientifc understanding. Potentially beneficial adjustments to the DQN
and the study design were recognized, which might allow to fully exploit this new
experimental paradigm in the future.
Key words: Deep Q Network, functional magnetic resonance imaging, Q-learning,
decision making, representational similarity analysis
