Self-learning systems have attracted increasing attention in the ramp metering domain in recent years. These systems are based on reinforcement learning (RL) and can learn to control motorway traffic adaptively. However, RL-based ramp metering systems are still in their early stages and have shown limitations regarding their design and evaluation.
This research aims to develop a new RL-based system (known as RAS) for ramp metering to overcome these limitations.
A general framework for designing a RL-based system is proposed in this research. It contains the definition of three RL elements in a ramp metering scenario and a system structure which brings together all modules to accomplish the reinforcement learning process. Under this framework, two control algorithms for both single- and multi-objective problems are developed. In addition, to evaluate the proposed system, a software platform combining the new system and a traffic flow model is developed in the research. Based on the platform developed, a systematic evaluation is carried out through a series of simulation-based experiments.
By comparing with a widely used control strategy, ALINEA, the proposed system, RAS, has shown its effectiveness in learning the optimal control actions for different control objectives in both hypothetical and real motorway networks. It is found that RAS outperforms ALINEA on improving traffic
efficiency in the situation with severe congestion and on maintaining user equity when multiple on-ramps are included in the motorway network. Moreover, this research has been extended to use indirect learning technology to deal with incident-induced congestion. Tests for this extension to the work are carried out based on the platform developed and a commercial software package, AIMSUN, which have shown the potential of the extended system in tackling incident-induced congestion.
