Testing Traffic Scenario Generation Pipeline in ADAS

The challenge

Research on generating driving scenarios becomes useful only when it can support an end-to-end testing workflow. The pipeline must accept how people naturally describe a scenario, produce an executable artifact, run it in simulation, and make the model calls observable enough to improve quality and control cost.

From research to implementation

This project applies ideas related to OSG-Q3 in a practical product pipeline. Instead of stopping at generation experiments, it connects external LLMs, command-line workflows, scenario execution, tracing, and real user data.

What I worked on

• Integrated external LLMs through OpenRouter and ChatGPT into a CLI-based scenario-generation workflow.
• Connected generated traffic scenarios to CARLA Simulator for execution and ADAS testing.
• Deployed Langfuse to trace the generation pipeline, inspect model behavior, and guide cost optimization.
• Built a private dataset of real-world scenario descriptions written by everyday users and practitioners who directly author ADAS test cases.

System shape

1. A user describes a traffic situation in natural language.
2. The CLI orchestrates an external LLM to turn the description into a structured scenario.
3. CARLA executes the scenario for practical ADAS testing.
4. Langfuse captures traces that support debugging, iteration, and cost review.
5. Private user-authored examples feed ongoing evaluation and improvement.

Outcome

The pipeline brings scenario-generation research closer to real testing work by linking natural-language input to simulation and adding the observability needed to operate and improve the system.

Content to add: supported scenario formats, evaluation criteria, dataset size, representative traces, cost changes, and a shareable CARLA demo.

What I learned

Moving from a research prototype to a working AI product changes the optimization target: generation quality still matters, but traceability, cost, realistic data, and simulator integration determine whether the system can be used repeatedly.