Hyperparameter specialization in the Hierarchical-Task Reservoir
Keywords
Loading...
Authors
Issue Date
2022-06-18
Language
en
Document type
Journal Title
Journal ISSN
Volume Title
Publisher
Title
ISSN
Volume
Issue
Startpage
Endpage
DOI
Abstract
The Reservoir Computing framework presents a more efficient approach
to training Recurrent Neural Networks. A classic implementation of this
framework is the Echo State Network. In addition to being easier to train,
Echo State Networks can display a variety of dynamics, which are controlled
by the network hyperparameters. When working with complex multiscale
sequential data the capabilities of singular Echo State Networks can prove
insufficient. In those cases, Hierarchical Echo State Networks are the better
choice. Hierarchical Echo State Networks display richer dynamics by having
different timescales on each layer of the model. Recently, a novel Hierarchical
Echo State Network, called the Hierarchical-Task Reservoir, has been
introduced. Its key feature is abstraction. The model’s task is divided into
sub-tasks. Every layer in the Hierarchical-Task Reservoir performs a different
sub-task as opposed to ”vanilla” Hierarchical Echo State Networks,
where each layer has the same task. The analysis of the Hierarchical-Task
Reservoir’s performance showed that the model has higher accuracy with
respect to Hierarchical Echo State Networks with no task abstraction. This
thesis focuses on investigating how the Hierarchical-Task Reservoir is able
to achieve better performance. The novel model is analyzed from the perspective
of its hyperparameters. The goal is to find out how the parameters
control the model accuracy and which parameters have the most impact.
Additionally, the effect of the abstraction feature on the parameter values
is evaluated, as well as the weight of the feature’s contribution to the model
accuracy, in comparison to the contribution of hyperparameters. It is concluded
that the model performs best with optimal parameter values and
that the abstraction feature has most effect on the model dynamics.
Description
Citation
Supervisor
Faculty
Faculteit der Sociale Wetenschappen