Human-in-the-Loop: Active Learning Approach to Image Classification using CLIP
Keywords
No Thumbnail Available
Authors
Issue Date
2023-07-11
Language
en
Document type
Journal Title
Journal ISSN
Volume Title
Publisher
Title
ISSN
Volume
Issue
Startpage
Endpage
DOI
Abstract
Image classification can be a labor-intensive job, because of manually labeling the data. Previous research
within Squadra Machine Learning Company has focused on the capabilities of large multimodal models
for image classification and feature extraction. It was found that the zero-shot capabilities of these models
do not always suffice and fine-tuning is necessary. This paper investigates the possibilities of fine-tuning
an image model for few-shot classification. For this, a human-in-the-loop Active Learning approach
was used to train OpenAI’s CLIP model without a highly labor-intensive labeling task. The research
question that will be tried to answer is: “What is the impact of different uncertainty measures on the
performance of Active Learning in few-shot classification tasks, and how does this approach compare to
traditional machine learning?” The report discusses the four main uncertainty measures and compares
the performance of Active Learning to traditional machine learning. Results show that entropy sampling
and margin outperform traditional machine learning and are potential approaches to use at Squadra
Machine Learning Company
Description
Citation
Supervisor
Faculty
Faculteit der Sociale Wetenschappen