The GRU and Transformer explaining Human Behavioural Data represented by N400.
Keywords
Loading...
Authors
Issue Date
2019-06-07
Language
en
Document type
Journal Title
Journal ISSN
Volume Title
Publisher
Title
ISSN
Volume
Issue
Startpage
Endpage
DOI
Abstract
Recurrent Neural Networks (RNN) are a popular type of neural network which are effective at processing language. The Gated Recurrent Unit (GRU) is a well known network that often outperforms other RNNs. Recently, a new neural network architecture has been introduced; the Transformer. In this investigation, the GRU and the Transformer are compared in their ability in predicting human sentence processing. The human language processing data is provided by Electroencephalography (EEG) measuring brain activity. The language models compute surprisal values on a corpus of English sentences. These surprisal values are compared to the human data given by the EEG experiment on the same corpus. The findings show that the GRU and Transformer differ significantly in predicting human language processing data; the Transformer shows higher goodness-of-fit scores for the vast majority of the training. This implies that the Transformer outperforms the GRU as cognitive model.
Description
Citation
Supervisor
Faculty
Faculteit der Letteren