Robustness of rate-based recurrent neural networks to cell death
Keywords
Loading...
Authors
Issue Date
2022-07-07
Language
en
Document type
Journal Title
Journal ISSN
Volume Title
Publisher
Title
ISSN
Volume
Issue
Startpage
Endpage
DOI
Abstract
Rate-based recurrent neural networks are popular and successful in the field
of machine learning. However, they are not well-known for being robust to
cell silencing, and studies on this subject are scarce. In addition, recurrent
neural networks’ calculations are still difficult to understand and visualize.
Work has been done on spiking neural networks to visualize the system’s
behaviour, specifically its behaviour after cell silencing. The studies demonstrated
the conditions in which spiking networks are robust. However it
is not clear whether we can think alike for non spiking networks. In this
study, we will utilize ordinary differential equations so that we may compute
the continuous trajectories of the network’s behaviour, using differential
equations. We shall demonstrate the consequences of cell silencing on the
behaviour of the network using a revised equation for the state space of
the network. Which will allow us to visualize the neuronal contributions
as vectors in the phase plots. This approach yields new insights into what
happens when neurons are silenced in a network. In particular it shows
that silencing a neuron corresponds to the removal of that vector in the
phase plots and that recurrent neural networks become more robust as the
amount of neurons in the networks grows. This allows for greater insight
on what the consequences of cell death are to the robustness of rate-based
recurrent neural networks. Through visualization, this will also aid in the
comprehension of and prevention of overfitting in recurrent neural networks.
Description
Citation
Supervisor
Faculty
Faculteit der Sociale Wetenschappen