A Multi-Agent Approach to the Train Unit Shunting Problem

No Thumbnail Available
Journal Title
Journal ISSN
Volume Title
The Nederlandse Spoorwegen (NS) operates a fleet of a considerable amount of trains. During the night and off-peak hours, the trains that are not in use need to be parked; this is done on shunting yards. These are not only used for parking, but also to clean the trains and to perform maintenance on them. Making a schedule for all the movements and the performance of service tasks, also referred to as the Train Unit Shunting Problem, is currently done by hand. However, as the shunting yards need to expand their capacity without the addition of more tracks, this becomes less feasible. To solve this, an automated solution is being sought. While several programs have been developed for this purpose, none have succeeded completely in creating both correct and consistent schedules. Following several projects that used neural networks to ensure consistency, this thesis tries to model the problem with agents that decide the actions of individual train units, as opposed to one planner that decides all the actions. This program is the first one to both use a neural network and include recombination in the problem description. The problem is structured as a Markov Decision Process which the agents try to solve using Value Iteration, for which they use a neural network to compute the state values. While the program performed worse than the currently best-performing program, it was able to solve problems with up to 6 train units 65% of the time after 126.000 episodes. Later in the training process the state values diverged, and the performance of the program dropped to 0%. Finally, it did show interesting results for regarding the recombination strategies before the state values diverged; contrary to both human strategies and strategies imposed on other programs, it performed both split and combine actions early on in the timeline of the problem.
Faculteit der Sociale Wetenschappen