A Distributed Deep Q-Network Based Freeway Traffic Control Scheme Actuated by Clusters of CAVs

Connected and Automated Vehicles (CAVs) are a technological advancement that can transform the current mobility system. These vehicles, in addition to being more efficient than traditional ones, can be used to implement vehicle-based control strategies, as it is proposed in this work. Specifically, this paper presents a freeway control strategy where the speed to be maintained along the traffic stretch is actuated by groups of CAVs, called clusters. Differently from previous approaches, this study proposes a Distributed Deep Q-Network (D-DQN) based on Multi-Agent Reinforcement Learning (MARL) theory to determine the speed that CAV clusters should maintain in a certain portion of the freeway to mitigate congestion. In this regard, an extended version of the Cell Transmission Model (CTM) is leveraged to emulate the traffic dynamics with the presence of CAV clusters. A numerical analysis, considering a stretch of the A20 freeway in the Netherlands, compares the proposed approach with the centralized version of the control scheme, demonstrating the effectiveness and the advantages of the proposed strategy.