A Multi-Agent Approach to the Train Unit Shunting Problem
Keywords
No Thumbnail Available
Authors
Issue Date
2020-01-29
Language
en
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
The Nederlandse Spoorwegen (NS) operates a fleet of a considerable amount of trains. During the
night and off-peak hours, the trains that are not in use need to be parked; this is done on shunting
yards. These are not only used for parking, but also to clean the trains and to perform maintenance
on them. Making a schedule for all the movements and the performance of service tasks, also
referred to as the Train Unit Shunting Problem, is currently done by hand. However, as the shunting
yards need to expand their capacity without the addition of more tracks, this becomes less feasible.
To solve this, an automated solution is being sought. While several programs have been developed
for this purpose, none have succeeded completely in creating both correct and consistent schedules.
Following several projects that used neural networks to ensure consistency, this thesis tries to model
the problem with agents that decide the actions of individual train units, as opposed to one planner
that decides all the actions. This program is the first one to both use a neural network and include
recombination in the problem description. The problem is structured as a Markov Decision Process
which the agents try to solve using Value Iteration, for which they use a neural network to compute
the state values. While the program performed worse than the currently best-performing program, it
was able to solve problems with up to 6 train units 65% of the time after 126.000 episodes. Later in
the training process the state values diverged, and the performance of the program dropped to 0%.
Finally, it did show interesting results for regarding the recombination strategies before the state
values diverged; contrary to both human strategies and strategies imposed on other programs, it
performed both split and combine actions early on in the timeline of the problem.
Description
Citation
Faculty
Faculteit der Sociale Wetenschappen