Optimizing Convolutional Neural Networks for Fast Training on a Small Dataset

Abstract

Since a few years convolutional neural networks are state of the art for image recognition tasks. However training these networks is time and computationally intensive and there are a lot of optimizable parameters involved. There is a lot of literature describing the best network structures and parameters to get the best results on well known and large datasets after hours of training time. In contrast much less is known on how to get fast and good results on a small dataset. In this paper we present approaches to quickly train good performing convolutional neural networks on a small dataset so we can solve a problem with brain data (for which the amount of available data is limited in most cases). We found that network structure related parameters together with learning rate (and its decay) are the best choice of parameters to optimize your network with.