Deep Q-Learning-based Traffic Control with Clusters of CAVs

Connected and Automated Vehicles (CAVs) represent a technological advancement that can effectively reshape the mobility system as we know it today. In fact, besides being themselves more efficient than traditional vehicles, these vehicles can be used to implement vehicle-based control strategies, as is the purpose of this work. More in detail, we present a freeway control strategy in which a variable speed limit control is actuated by means of groups of vehicles, here denoted clusters, which are used as control actuators to enforce a certain speed to surrounding Traffic. Different from other approaches already existing in the literature, this study investigates the application of Deep Q-learning networks (DQN) to define the speed that the clusters of CAVs must maintain to decongest a freeway stretch. The proposed method employs an enriched version of the Cell Transmission Model (CTM) to simulate the Traffic dynamics in the presence of groups of CAVs, and uses a DQN-based controller to determine optimal variable speed limits for CAV clusters. Numerical validations, performed on the real stretch of the A20 freeway in the Netherlands, demonstrate significant reductions in Total Travel Time (an improvement of about 27% compared to the uncontrolled case), showing the effectiveness of vehicle-based control strategies using reinforcement learning.