Reinforcement learning for autonomous navigation of articulated vehicles

Abstract

This research delves into the development of a reinforcement learning based model for the lateral and longitudinal control of tractor‐trailer vehicles through a roundabout in‐ tersection. With such vehicles being a crucial part of the logistics sector, which provides basic needs to our society, the development of autonomous driving models for such ve‐ hicles has the potential to reduce the costs of the logistics sector through greater fuel and time efficiency and reduce accidents caused by human fatigue. Despite this, so far, limited research that covers the motion control of articulated heavy goods vehicles has been carried out. Due to their physical properties, tractor‐ trailer vehicles require specially crafted models which obtain a suitable representation of their movement. Such movement differs from light passenger vehicles due to the articulated joint, length, width, height, and weight of tractor‐trailer vehicles. Previous researchers developing rule‐based approaches note that the mathematical models used to define the dynamics models of the tractor‐trailer vehicle, which are the basis of their work, are simplified for the benefit of computational complexity. This limits the applica‐ bility of rule‐based models in different environments without specific fine‐tuning. Reinforcement learning allows the development of models which learn through trial and error, enabling them to obtain superior performance to rule‐based models in unknown environments. Firstly, due to the lack of publicly available models, this work proposes a model of a tractor‐trailer vehicle in the high‐fidelity CAR Learning to Act (CARLA) simulator. Five different roundabout scenarios were also developed to perform the driving trials. Such intersections have varying physical properties, allowing for the discovery of each model’s advantages and pitfalls. Being the first of its kind, this work proposes a reinforcement learning environment which can be utilised for such vehicles. The 69 continuous observations provide information about the vehicle and the route. We compare the PPO, dueling double DQN, and SAC algorithms, with the on‐ policy, policy optimisation algorithm, PPO, obtaining superior results with a 0.77 suc‐ cess rate on testing scenarios while also maintaining the least distance to the centre of the lane, mimicking human‐like behaviour. Aiming to discover the full potential of this framework, the proportional‐integral controller maintaining a constant velocity was eliminated with the reinforcement learning algorithm being able to accurately control the tractor‐trailer vehicle’s longitudinal and lateral movements, obtaining a 0.68 testing success rate using the PPO algorithm.