Presentation Material
Abstract
The Adversarial Scenario Fuzzer is an automated testing framework that evaluates autonomous vehicle resilience against potentially harmful teleoperation commands. While teleoperation can help resolve complex driving situations, incorrect or malicious commands pose safety risks.
The fuzzer systematically generates challenging scenarios through simulation, including:
- Malicious trajectory suggestions
- Conflicting guidance signals
- Environmental perturbations
Using iterative optimization, the fuzzer creates increasingly impactful test cases while evaluating the vehicle’s ability to reject unsafe commands. This approach helps validate the robustness of autonomous decision-making systems and ensures safety mechanisms can effectively handle adversarial inputs.
AI Generated Summary
The talk presents a security-focused validation framework for teleoperation systems in autonomous vehicles, where a remote human operator provides guidance to the AI when it encounters uncertainty. The core research applies fuzzing—a technique from software security—to the autonomous driving stack in simulation. Instead of fuzzing code, the system fuzzes driving scenarios by mutating teleoperation commands (e.g., suggested waypoints) derived from real-world driving data.
The framework extracts key scenario features—vehicle state, operator command, and surrounding traffic behavior—from millions of miles of logged data to create reproducible base scenarios. These are mutated using “teleoperation parameters” that apply offsets to paths, speeds, and timing. Each mutated scenario is executed in a high-fidelity simulator, and the vehicle’s behavior is monitored for unsafe outcomes like collisions, near-misses, or improper rejections of commands.
Running this process generated over 50,000 scenario variants. The fuzzer discovered subtle edge cases where malicious or erroneous operator commands could cause collisions, even when the AI’s perception seemed correct. Examples include a “valley collision” where a slightly mutated path bypassed the AI’s safety checks, and a reverse command at an intersection that triggered a side collision due to a timing bug in the prediction module. These cases revealed hidden vulnerabilities in decision logic that manual or nominal testing missed.
The findings drove root-cause analysis and retraining of affected AI modules, improving resilience against both accidental and intentional misuse. The approach is being extended to fuzz other stack components: perception (by distorting sensor inputs) and planning (by injecting rare, high-severity maneuvers). The work argues that safety and security at scale require proactive, automated stress testing that mimics adversarial conditions, moving beyond checking nominal behavior to hardening the system against worst-case scenarios. This process is presented as essential for building public trust in autonomous vehicles.