AdvDO: Realistic Adversarial Attacks for Trajectory Prediction

Yulong Cao1,2
Chaowei Xiao2
Anima Anandkumar2,3
Danfei Xu2
Marco Pavone2,4
1 Umich
2 NVIDIA
3 Caltech
4 Stanford


code Code

Abstract

Trajectory prediction is essential for autonomous vehicles (AVs) to plan correct and safe driving behaviors. While many prior works aim to achieve higher prediction accuracy, few study the adversarial robustness of their methods. To bridge this gap, we propose to study the adversarial robustness of data-driven trajectory prediction systems. We devise an optimization-based adversarial attack framework that leverages a carefully-designed differentiable dynamic model to generate realistic adversarial trajectories. Empirically, we benchmark the adversarial robustness of state-of-the-art prediction models and show that our attack increases the prediction error for both general metrics and planning-aware metrics by more than 50% and 37%. We also show that our attack can lead an AV to drive off road or collide into other vehicles in simulation. Finally, we demonstrate how to mitigate the adversarial attacks using an adversarial training scheme.

We consider two essential properties of modern prediction models: (1) motion property, which captures the influence of past agent states over future states; and (2) social property, which captures how the state of each agent affects others. As illustrated in the Figure, by only manipulating the history trajectory of the adversarial agent, we are able to mislead the predicted future trajectory for the adversarial agent (i.e. incorrect prediction for left turning future trajectory of red car). Furthermore, we are able to mislead the prediction for other agent’s behavior (i.e. turning right to turning left for the yellow car).

AdvDO Overview

In this work, we propose AdvDO (Adversarial Dynamic Optimization) for generating effective and realistic adversarial trajectories against trajectory prediction models.

Attack Demo

We demonstrate that with the generated adversarial trajectories, down stream tasks like planning will be significantly affected. Down below we demonstrate the collision consequences with Carla simulation.

Realistic Trajectory Generation

Down below, we show that the adversarial trajectory generated from search has either behavior change or unrealistic steering rates while trajectory generated from AdvDO is more realistic.

Benign Example

AdvDO

Search

Paper

AdvDO: Realistic Adversarial Attacks for Trajectory Prediction

Yulong Cao, Chaowei Xiao, Anima Anankuda, Danfei Xu and Marco Pavone

description arXiv version
insert_comment BibTeX
integration_instructions Code

Citation

inproceedings{cao2022advdo,

          title={AdvDO: Realistic Adversarial Attacks for Trajectory Prediction},
        
          author={Yulong Cao, Chaowei Xiao, Anima Anankuda, Danfei Xu and Marco Pavone},
        
          booktitle={European conference on computer vision (ECCV)},
        
          year={2022},
        
          organization={Springer}
        
        }
}