Published August 3, 2023 | Version v1
Journal article Open

Multimodal Manoeuvre and Trajectory Prediction for Automated Driving on Highways Using Transformer Networks

  • 1. ROR icon University of Warwick
  • 2. ROR icon Queen's University Belfast


Predicting the behaviour (i.e., manoeuvre/trajectory) of other road users, including vehicles, is critical for the safe and efficient operation of autonomous vehicles (AVs), a.k.a., automated driving systems (ADSs). Due to the uncertain future behaviour of vehicles, multiple future behaviour modes are often plausible for a vehicle in a given driving scene. Therefore, multimodal prediction can provide richer information than single-mode prediction, enabling AVs to perform a better risk assessment. To this end, we propose a novel multimodal prediction framework that can predict multiple plausible behaviour modes and their likelihoods. The proposed framework includes a bespoke problem formulation for manoeuvre prediction, a novel transformer-based prediction model, and a tailored training method for multimodal manoeuvre and trajectory prediction. The performance of the framework is evaluated using three public highway driving datasets, namely NGSIM, highD, and exiD. The results show that our framework outperforms the state-of-the-art multimodal methods in terms of prediction error and is capable of predicting plausible manoeuvre and trajectory modes.



Files (970.0 kB)

Name Size Download all
970.0 kB Preview Download

Additional details


Hi-Drive – Addressing challenges toward the deployment of higher automation 101006664
European Commission