Robustness of rate-based recurrent neural networks to cell death

Thumbnail Image
Issue Date
Journal Title
Journal ISSN
Volume Title
Rate-based recurrent neural networks are popular and successful in the field of machine learning. However, they are not well-known for being robust to cell silencing, and studies on this subject are scarce. In addition, recurrent neural networks’ calculations are still difficult to understand and visualize. Work has been done on spiking neural networks to visualize the system’s behaviour, specifically its behaviour after cell silencing. The studies demonstrated the conditions in which spiking networks are robust. However it is not clear whether we can think alike for non spiking networks. In this study, we will utilize ordinary differential equations so that we may compute the continuous trajectories of the network’s behaviour, using differential equations. We shall demonstrate the consequences of cell silencing on the behaviour of the network using a revised equation for the state space of the network. Which will allow us to visualize the neuronal contributions as vectors in the phase plots. This approach yields new insights into what happens when neurons are silenced in a network. In particular it shows that silencing a neuron corresponds to the removal of that vector in the phase plots and that recurrent neural networks become more robust as the amount of neurons in the networks grows. This allows for greater insight on what the consequences of cell death are to the robustness of rate-based recurrent neural networks. Through visualization, this will also aid in the comprehension of and prevention of overfitting in recurrent neural networks.
Faculteit der Sociale Wetenschappen