Pretrained Harry vs Fine-tuned Potter: The influence of fine-tuning a pretrained Twitter-based Sentiment Analysis BERT model on relationship classification in fictional novels

Thumbnail Image
Issue Date
Journal Title
Journal ISSN
Volume Title
Characters and the relationships between characters contribute a lot to the storylines of fictional novels. Classifying these relationships automatically requires SA, which is a subtask of NLP. The latest research on classifying the relationships between characters in fictional novels was done with a fine-tuned BERT model by Zehe et al. (2020). Their model proved to outperform models from earlier studies. Because of this, we proposed to use and fine-tune a SA BERT model in this research. We chose to classify the relationship between characters, based on (inter-) actions between two characters in sentences, as being positive, negative or neutral. Because pretraining a BERT model is resource intensive and because domain specific models are not always available, we made the unconventional choice use a Twitter-based SA BERT model as a baseline. We fine-tuned this Twitter-based SA BERT model on the first Harry Potter novel. The performance of the baseline Twitter-based SA BERT model and our fine-tuned version of this model was measured based on their evaluation scores. In addition to this we measured the performance of both models in relationship classification. This involved using both models on all sentences that contain (inter-) actions between two characters in the novel. The proposed model outperformed the Twitter-based SA BERT model in both evaluation scores and relationship classification results.
Faculteit der Sociale Wetenschappen