Qwen releases a research model for driving perception and planning

A shared model for three driving tasks

Alibaba's Qwen team and Huazhong University of Science and Technology have released Qwen-Drive-1.0, a research system that connects a general vision-language model to driving perception and planning. The project includes code, downloadable weights and a technical report. Its central question is whether one visual representation can support both an explanation of a road scene and a proposed path through it.

The researchers retain the pretrained language-model architecture instead of replacing it with a driving-only network. A bird's-eye-view perception head extracts object, occupancy and map information. A separate planning expert predicts the vehicle's future trajectory. The language decoder remains available for questions about images, including questions unrelated to driving.

How the training is organized

The paper describes staged training that mixes driving supervision with general vision-language examples. This is intended to add specialized capability without discarding the base model's broader visual understanding. The data pipeline also reconciles different perception labels, revises driving question-and-answer annotations for consistency and expresses trajectories in a common waypoint format.

These are three different learning problems. Object perception describes what surrounds the vehicle; question answering explains a scene in words; planning proposes how the vehicle should move. Sharing the underlying representation makes their interaction a subject of the experiment, rather than treating a fluent verbal explanation as proof of a good driving decision.

What the published numbers show

The repository identifies Qwen3.5-4B as the shared backbone and distributes two planning heads: one trained with supervised imitation and another subsequently optimized with rewards. The ordinary NAVSIM v1.1 planning score rises from 88.2 to 90.7 with the latter. The corresponding best-of-six results are 89.3 and 91.4, a separate sampling condition that should not be mixed with single-output scores.

The gains are not uniform. On the NVIDIA PhysicalAI open-loop measure, three-second minimum average displacement error is 0.34 meters for the supervised version and 0.38 meters for the reinforcement-trained version; lower is better. This illustrates why the release cannot be reduced to one headline percentage. Reward optimization improved the reported NAVSIM result while this particular trajectory-error measure moved in the opposite direction.

The separate evaluation guide clarifies that best-of-six selection uses the ground truth to choose the strongest sampled trajectory. It is therefore an upper bound, not a selection method available to a real vehicle that does not know the future. The guide also distinguishes Waymo's human-preference score from displacement error: a reasonable alternative path can receive a good preference rating even when it differs from the recorded drive.

A modular package rather than one opaque checkpoint

The repository separates a roughly 9.1 GB shared vision-language model from two approximately 2.1 GB planning heads and a 0.5 GB perception head. Researchers attach the head required for the task. The reward-trained planner is documented for reasoning-conditioned planning, whereas the imitation-trained head supports both direct and reasoning modes. Running both heads under an unsupported common mode would therefore not reproduce the intended comparison. The package layout makes that dependency visible before a researcher starts a driving experiment.

What is available to reproduce

Qwen publishes the project under Apache 2.0 and recommends a GPU with at least 24 GB of memory. The package separates the shared model from the planning and perception heads. Bundled demonstrations cover an intersection at night, left and right turns, and slowing near a parked truck.

Those examples let researchers examine a proposed trajectory alongside its generated reasoning. The release is an experimental model and evaluation package. It does not announce a production vehicle deployment or establish that the same results hold in unrestricted road traffic.