TriDrive: Joint Driver, Vehicle, and Road Modeling for Human-Centered Driving

Predicting how drivers, vehicles, and road scenes interact and evolve together is central to human-centered driving. Prior work models in-cabin activity or traffic-conditioned driver motion in isolation; a framework that models all three jointly and runs in real time on vehicle hardware had not been shown. TriDrive jointly models the driver, the vehicle, and the road: to our knowledge it is the first to forecast driver kinematics, vehicle dynamics, and road demands together from an automation-conditioned transition model, with forecasting and warning as its two applications. Modality-specific encoders — an anchored kinematic representation of the driver, causal CAN-bus dynamics, and frozen V-JEPA 2 road latents with structured road margins — are connected by directed residual connections through which driver and road context refine vehicle forecasts. On the public AIDE benchmark the kinematic encoder recipe sets a new full-set state of the art among published baselines (48.05 vs. 71.47 All-MPJPE). On 197.2 hours of naturalistic BATON driving, directed connections and road margins raise assistance-engaged PR-AUC by 0.084 for steering onset and 0.286 for time-to-collision drops. For real-time use we distill the road encoders and run TriDrive on a comma four with an external 8 GB GPU, where a lightweight current-state warning probe updates at 5 Hz with 177 ms p95 latency while the forecasting model runs concurrently. The probe is above an openpilot-based baseline on human-labeled manual-driving warnings (AUROC 0.725 vs. 0.563), and in a paired on-road study 14 drivers rate its warnings as more appropriate (+1.79) and timely (+2.67) than those of openpilot's driver-monitoring system.