Game Theoretic Efficient Learning Systems

Rood, Thomas

Game Theoretic Efficient Learning Systems

Files

Rood, T. s-1005156-MSc-MKI92-Thesis-2022.pdf (3.46 MB)

Authors

Rood, Thomas

Issue Date

2022-07-01

Language

en

URI

https://theses.ubn.ru.nl/handle/123456789/16353

Abstract

The aim of this thesis is the development of new pruning methods based on power indices such as the Shapley value and the Banzhaf index. As a proof of concept, a preliminary study was conducted on the application of these concepts to decision tree pruning, which showed that these power indices constitute viable pruning metrics, with the additional benefit that they also provide interesting insight into feature importance. The main work of this thesis continued this train of thought, applying the pruning to image classification using small and medium-sized models. To improve scaling of our methods to larger models, as well as the accuracy of the pruned models, new techniques were introduced. An extension of Monte Carlo dropout was applied as a method for estimating optimal network size. This network size was subsequently used to bias the sampling in the Banzhaf index to obtain a new power index: Biased Banzhaf. To cut down further on computation time, layer-wise power index sampling was introduced. This allowed for pruning the AlexNet model with reasonable computational resources, while outperforming common pruning baselines. As a second topic during the thesis, experimental results were contributed to the work “Deep Regression Ensembles”. This work introduced a new model that uses ensembles of random features followed by ridge regression to learn complex problems without the need for gradient descent. The contributions involved the implementation of the proposed model, additions to help scale the model, and the application of the model to image classification on MNIST and TinyImageNet. Finally, the thesis involved contributions to the work “Neural Payoff Machines: Predicting Fair and Stable Payoff Allocations Among Team Members”, mainly in the form of scientific discussion about experiment design and interpretation.