Finding the Topics of Case Law: Latent Dirichlet Allocation on Supreme Court Decisions

Thumbnail Image
Issue Date
Journal Title
Journal ISSN
Volume Title
The law produces a large amount case law, which is still mostly processed by hand. The Case Law Analytics project aims to develop a technology that assists the legal community in analyzing case law. As a part of this project, this thesis explores the possibilities of finding accurate and useful legal topics with LDA and whether or not legal experts and people with a non-legal background agree in their judgments about this. To this end I investigated possible methods suited for evaluation of the model's results. I evaluated the topics as well as their assignment to the documents using human evaluation. I found that the topics evaluated to cohere most, are easy to label. Human subjects were also mostly able to differentiate between topics assigned to a document with high probability and topics that do not belong to this document. However less than half the topics were evaluated as coherent by the subjects and according to the subjects the main topic of a document was not found by the model for most of the documents. I also found that domain experts and non domain experts might evaluate topics differently. I argue that the usability of the results depends on the intended application and and introduce some complications specific to the legal domain, which should be taken into account as well.
Faculteit der Sociale Wetenschappen