Sample Efficiency in Model-Based Reinforcement Learning for Ship Seakeeping & Manoeuvring

Greep, Justin

Sample Efficiency in Model-Based Reinforcement Learning for Ship Seakeeping & Manoeuvring

Files

Greep, J. s-4820177-MSc-MKI94-Thesis-2023.pdf (1.81 MB)

Authors

Greep, Justin

Issue Date

2023-04-01

Language

en

URI

https://theses.ubn.ru.nl/handle/123456789/16406

Abstract

Ships travel in stochastic environments. The stochasticity does not only come from the waves but also from the wind, current, bathymetry changes and presence of other objects. Maintaining the course of the ship in such an environment is challenging as it involves dynamic interaction of the ship and its appendages with this stochastic environment. For course keeping of ships, the potential of reinforcement learning (RL) is mainly unexplored as classical feedback controllers are typically used. The training of RL models, however, requires a large number of interactions with the environment, which makes the real world application of these models infeasible. This research project investigates a model-based approach to increase the sample-efficiency in RL. Model-based RL learns a dynamics model alongside the policy to estimate the state transitions in the environment. In this project, a model-based RL agent is trained to maintain the course of a ship in waves, and is compared to a linear-quadratic regulator (LQR) and a model-free RL agent. By employing the model-based agent, the required number of agent-environment interactions to learn a policy was significantly reduced compared to the model-free agent. Furthermore, the performance of the agent was improved in the sense that the rudder usage was reduced considerably while maintaining the course as well as the model-free RL agent and the LQR.