- Title
- Modelling railway traffic management through multi-agent systems and reinforcement learning
- Creator
- Bretas, A.; Mendes, A.; Chalup, S.; Jackson, M.; Clement, R.; Sanhueza, C.
- Relation
- 23rd International Congress on Modelling and Simulation (MODSIM2019). Proceedings of 23rd International Congress on Modelling and Simulation (MODSIM2019) (Canberra, Australia 01-06 December, 2019) p. 291-297
- Publisher
- Modelling and Simulation Society of Australia and New Zealand
- Resource Type
- conference paper
- Date
- 2019
- Description
- Australia plays a significant role in the world’s coal supply. The world’s largest coal operation is located in the state of New South Wales, where more than 87% of the transport is done through railways. One of the strategies to increase throughput is to use sophisticated computational techniques for train scheduling optimisation and this study applies artificial intelligence techniques to the railway traffic management problem in the context of the Hunter Valley Coal Chain Coordinator (HVCCC). This problem has been studied mostly through centralised decision-making models, applying linear integer programming, heuristics and hybrid approaches. However, recent publications indicate a lack of practical applications (Lamorgese et al [2018]), pointing out that low computational requirements, scalability, decentralisation and real-world commitment are key features required for deployment-ready applications. Towards that, one option is to model system actors (trains, stations, dispatchers, operators, and more) as autonomous intelligent agents that interact, learn and act independently to reach their own objectives – thus constituting a multi-agent system (MAS). This way, the railway traffic system will be capable of making rapid, distributed decisions. Few studies have modelled railway traffic management as a MAS and they lack many of the important decisions, constraints and actors present in real-world scenarios (Lamorgese et al. [2018]). This paper describes a discrete event simulation model of a small, closed railway, and implements a de- centralised and heterogeneous MAS for train dispatching. The model was built using the Arena Simulation Software1. It includes several train agents and a single dispatcher agent that applies different decision methods (First-in-First-Out rule, random walk and reinforcement learning) to regulate railway traffic decisions. The paper describes how experiments were designed, computational results, calibration of the reinforcement learning (RL) algorithm, performance tests for various levels of congestion, and tests for transfer learning between different instance configurations. The RL performance outperforms the FIFO standard dispatch rule by 10.3% for the high-congestion network configuration. In addition, transfer learning tests illustrate the generalisation capability of the RL method, where knowledge gained during the training using an instance reduces the time required for the training of additional instances. This represents an initial step towards the application of the approach in the HVCCC network traffic management problem.
- Subject
- railway traffic management; multi-agent systems; reinforcement learning; transfer learning; discrete event simulation
- Identifier
- http://hdl.handle.net/1959.13/1460093
- Identifier
- uon:45860
- Identifier
- ISBN:9780975840092
- Language
- eng
- Reviewed
- Hits: 1896
- Visitors: 1868
- Downloads: 0
| Thumbnail | File | Description | Size | Format |
|---|