Exploring Decision Transformer for Highway Automated Driving

The evolution of Automated Driving Functions (ADFs) is contingent upon the effective implementation of Decision-Making (DM), context perception, and predictive vehicle control. Conventional Deep Reinforcement Learning (DRL) methodologies frequently prove inadequate in dynamic settings, largely due to their inherent limitations in addressing real-time DM and assigning long-term credit. DRL via sequence modeling represents a promising avenue for addressing these challenges by combining the strengths of Attention-based architectures, such Transformer, and DRL. The integration of self-attention mechanisms with offline DRL enables long-term credit assignment, fine-tuning and prevent continuous interaction with the environment, mitigating risks related to real-world simulations and trial-and-error approaches. This paper examines the potential of Decision Transformer (DT) within the AD domain. A DT model was implemented and trained within the highway-env simulation environment. To do so, an offline RL dataset was constructed using a pre-trained Deep Q-Network (DQN) agent. The model was evaluated by comparing its performance against that of the pre-trained DQN and a random agent. Results demonstrated that the DT model exhibited superior DM capabilities, with higher average returns and longer episode durations than DQN. These findings highlight the potential of Transformer-based DRL in AD.