Prospect theory in exploration
Keywords
Loading...
Authors
Issue Date
2023-01-27
Language
en
Document type
Journal Title
Journal ISSN
Volume Title
Publisher
Title
ISSN
Volume
Issue
Startpage
Endpage
DOI
Abstract
Exploration problems are situations where an agent should explore an environment without any prior knowledge. The standard approach to this problem is by using reinforcement learning. This is a method where the agent learns a sequence of actions that lead to the best cumulative reward. Problems with this approach are the learning inefficiency and the inapplicability to real-world problems. A possible solution is to use cheaper and less complex approaches. One possibility is expected utility theory. In this theory the agent always chooses the option with the highest expected utility. This utility is calculated by taking the weighted sum of all possible outcomes. In 1979 Kahneman and Tverksy offered critique on this theory as a model of human behaviour when risks are involved. They presented a better alternative called prospect theory, which encapsulates the risk averse and risk seeking behaviour of humans. These three theories are compared in the setting of an exploration problem. This leads to the research question: How does prospect theory compare to expected utility theory and reinforcement learning in exploring an environment? To answer this question a general framework was created. This framework contained many necessary functions to implement the before mentioned theories and algorithms. The final implementations were tested in several environments with increasing complexity. Ultimately, it seems that reinforcement learning is superior but the results are inconclusive due to several limitations. Unfortunately this means that the original problem with reinforcement learning has not been solved and it is uncertain if either prospect theory or expected utility can be used as an alternative.
Description
Citation
Supervisor
Faculty
Faculteit der Sociale Wetenschappen