Abstract
BATON compiles 136.6 hours of naturalistic driving from 127 drivers across 380 routes, synchronizing front-view video, in-cabin video, decoded CAN signals, radar-based lead-vehicle interaction, and GPS route context to capture bidirectional control transitions between human and driving-automation systems. It defines three benchmark tasks: driving-action understanding (7-class), handover prediction (human→DAS), and takeover prediction (DAS→human), with baselines (GRU, TCN, XGBoost) and zero-shot vision-language-model evaluations. We find that visual input alone is insufficient for reliable transition prediction; adding vehicle dynamics and route context substantially improves accuracy; and that takeovers unfold gradually — benefiting from longer prediction windows — while handovers hinge on immediate context.
Dataset overview: synchronized multimodal streams per drive.
The three benchmark tasks and their evaluation protocol.
Distribution of driving actions and transition events.
In-cabin view at the moment of a take-over.
Synchronized Multimodal Views
Front camera, cabin camera, and sensor streams aligned in time.
BibTeX
@article{wang2026baton,
title = {BATON: A Multimodal Benchmark for Bidirectional Automation Transition Observation in Naturalistic Driving},
author = {Wang, Yuhang and Xu, Yiyao and Yang, Chaoyun and Li, Lingyao and Sun, Jingran and Zhou, Hao},
journal = {arXiv preprint arXiv:2604.07263},
year = {2026}
}