Skip to content

Dataset · human–ADAS transitions · IEEE ITSC 2026

ADAS-TOA Large-Scale Multimodal Naturalistic Dataset and Empirical Characterization of Human Takeovers during ADAS Engagement

The first large-scale naturalistic dataset dedicated to ADAS-to-manual transitions: 15,659 twenty-second clips aligned at the moment control comes back to the driver, from 327 drivers and 22 brands.

Every clip pairs the front camera with the vehicle's logs around one takeover. Trigger, driver intent, kinematics and vision-language cues are labeled, so the question "what did the driver see, and how early?" can be asked at scale.

IEEE ITSC 2026 · accepted · arXiv 2603.06986front video + logs · 10 / 100 Hzrelease v2 on Hugging Face · 16,446 clips
Open the anatomy arXiv Paper Dataset Code

Yuhang Wang1·Yiyao Xu1·Jingran Sun1·Hao Zhou1

1University of South Florida · MOTIF-Lab

● REC
Front-camera frame at the moment of a brake takeover
live · release v2 clips · 10 Hz80 traces · 4 triggers
15,659takeover clips (paper) 87.0 hof video around takeovers (paper) 327drivers (paper) 22vehicle brands · 163 models (paper) 285safety-critical cases (paper)

Release v2 on Hugging Face (2026): 16,446 clips · 364 drivers · 179 models · 2,585 routes. Paper numbers are from the ITSC camera-ready; the release grew after the camera-ready.

01 · OVERVIEW

The handover, observed in the wild

The abstract, verbatim, beside what you need to know at a glance.

ADAS-TO characterizes transitions from Advanced Driver Assistance Systems (ADAS) back to manual control. It comprises 15,659 twenty-second clips from 327 drivers across 22 vehicle brands, each synchronizing front-camera video with vehicle logs. Takeovers are categorized by trigger — brake, steering, accelerator, combinations, and system disengagement — and by whether they were driver-initiated or forced. Our analysis identifies 285 safety-critical cases and, by combining vehicle kinematics with vision-language-model annotations, finds that actionable visual cues emerge at least three seconds before takeover in roughly 60% of critical events. ADAS-TO provides a rich, multimodal foundation for studying human–automation handover and building predictive take-over warning systems.

02 · ANATOMY OF A TAKEOVER

Ten seconds before, ten seconds after

One clip plays while its own signals draw in white over quantile bands of thousands of takeovers of the same kind. Switch the trigger and the bands morph. The vertical rule at t = 0 is the moment control came back to the driver.

● REC
Kinematic signatures of takeovers by trigger, from the paper
Static view. Kinematic signatures by trigger (paper figure); the live anatomy needs JavaScript.

What you are looking at. Three release-v2 sample clips (brake, steering and gas takeovers; two pseudonymous drivers) with openpilot's logs at 10 Hz. White is this clip; the coloured bands are the population. Shading marks when the assistance system was engaged. Lane deviation and lane confidence come from the lane model; TTC and THW from the radar lead.

03 · EVIDENCE

What the numbers rest on

285 critical = 173 low-TTC + 127 low-THW − 15 both

The long tail: most takeovers are calm; a few hundred are not.

Each point is a cover-takeover clip from release v2 with a valid lead: minimum TTC against minimum THW inside the 20 s (log scales, capped at 30 s and 10 s). Dashed thresholds at TTC 3.0 s and THW 0.8 s define "critical" in the paper. Colour by criticality or by trigger.

Scatter of minimum TTC against minimum THW per clip, from the paper
TTC–THW scatter (paper figure).

Two definitions, both shown: 285 kinematic-threshold cases on the paper's 15,659 clips (paper); 749 manually reviewed safety-critical clips in release v2 (16,446 clips) (release). Release-v2 lead-valid population: 4,786 clips — 77 below both thresholds, 739 low-TTC only, 432 low-THW only.

59.3 % of critical cases

Visual cues at least three seconds before the takeover.

Vision-language annotations of the 285 critical cases: 63.7 % traffic dynamics, 47.4 % infrastructure degradation, 37.5 % adverse environment — categories overlap. In the paper's words, cues emerge ≥ 3 s ahead in 59.3 % ("roughly 60 %").

Early-warning advantage: when visual cues appear relative to the takeover
Lead time of visual cues before the kinematic takeover (paper figure).
39.6 % brake-first takeovers

How drivers take back control.

First action in the paper's 15,659 clips: brake 39.6 %, steering 25.3 %, system disengagement 13.7 %, gas 13.5 %, mixed 7.9 %. A lead vehicle was present in 49.8 % of clips; mean speed 54.9 ± 30.8 km/h.

First actionShare
Brake39.6 %
Steering25.3 %
System disengagement13.7 %
Gas13.5 %
Mixed7.9 %
16,446 clips in release v2

Labelled by what the clip actually is.

Release-v2 annotation table: cover (brake / steer / gas takeover), lane change, stop, turn, other. 364 pseudonymous drivers, 179 models, 2,585 routes; 9,992 clips with 10 Hz logs and 6,454 with 100 Hz logs.

Label counts in the release table.
84.0 % agreement · 500 clips

The ego / non-ego partition was audited by four experts.

Agreement 84.0 % (420 of 500 clips); ego precision / recall 90.2 / 81.5 %, non-ego 77.1 / 87.5 %. The paper calls the partition and hazard labelling "intentionally lightweight" — we quote it rather than oversell it.

Semantic clustering of takeover contexts from vision-language annotations
Semantic clusters of takeover contexts (paper figure).

Journal extension (under review). A longitudinal-control follow-up on release v2 (16,446 clips, 364 drivers) finds 6,329 confirmed manoeuvre-filtered events; 65.6 % of first actions precede the logged disengagement, and the next five seconds carry a 78.5 % lower minimum TTC. Numbers will be pinned here once that paper is public.

04 · METHOD

From a disengagement to a labelled clip

Every openpilot disengagement in the source logs becomes a 20-second window centred on the takeover; the front video and the logs are cut together, then labelled by trigger, driver action and visual context.

Dataset overview: sources, clip construction, labels and statistics
Overview. Sources, clip construction, labelling and headline statistics (paper figure).
Clip structure: twenty seconds of video and logs aligned at the takeover
Clip structure. −10…+10 s around the takeover, video and logs on one clock.
Geographic distribution of the recordings
Where. Recordings span U.S. states and several countries; coordinates are not released with the clips.

Takeover scenarios

Close vehicle ahead
Truck cut-in
High-severity case
Cut-in from the right
Rapid closing
TTC drop
Sequence of driver actions around the takeover
Action sequence. Which pedal or the wheel moves first, and when (paper figure).
05 · RESULTS

Tables from the paper

Table II · ITSC 2026 (paper) and release v2 (Hugging Face, 2026)

PaperRelease v2
Clips15,65916,446
Hours of video87.0—
Drivers327364
Models / brands163 / 22179 / —
Routes—2,585
Log rate61.7 % at 10 Hz · 38.3 % at 100 Hz9,992 at 10 Hz · 6,454 at 100 Hz
Speed54.9 ± 30.8 km/h53.8 km/h mean (10 s before)
Lead vehicle present49.8 %48.8 %
Sources & notes
    06 · DATA & ACCESS

    Get the clips

    On request

    ADAS-TO · full release

    16,446 clips, ~41 GB, 15 files per clip; CC BY-NC 4.0. Gated with manual approval — share your contact information on Hugging Face.

    as of 2026-10-02 · gated: manual · CC BY-NC 4.0

    Request access
    On request

    Samples

    ADAS-TO-Sample (1,342 clips, 6 drivers) and ADAS-TO-Critical (491 clips from the reviewed set) — same layout, smaller downloads.

    as of 2026-10-02 · gated: manual

    SampleCritical
    Open

    Code & longitudinal tables

    Clip construction, labelling and analysis code on GitHub; the longitudinal-control tables as a separate MIT-licensed dataset.

    as of 2026-10-02 · gated: manual · MIT

    GitHubLongitudinal
    “Driver and route identifiers are anonymized.” · “Clip signal files contain no GPS coordinates.”Hugging Face dataset card · fetched 2026-10-02

    Terms. Non-commercial research use; no re-identification attempts; cite the ITSC paper; redistribute derived tables under the same terms, not the raw video.

    07 · CITE

    Read the paper, cite the data

    BibTeX
    @inproceedings{wang2026adasto,
  title     = {ADAS-TO: A Large-Scale Multimodal Naturalistic Dataset and Empirical Characterization of Human Takeovers during ADAS Engagement},
  author    = {Wang, Yuhang and Xu, Yiyao and Sun, Jingran and Zhou, Hao},
  booktitle = {IEEE International Conference on Intelligent Transportation Systems (ITSC)},
  year      = {2026},
  note      = {arXiv:2603.06986}
}

    Related work from the lab

    Dataset · ITSC 2025

    OpenLKA

    Lane keeping on real roads — the same recordings, before the takeover.
    Benchmark · arXiv 2026

    BATON

    Both directions of the control transition, with the cabin camera.
    arXiv 2026

    TriDrive

    Forecasting driver, vehicle and road together — in the car at 5 Hz.