NVIDIA LPR Introduces VFlash: A Generalist Decoding Framework for Physical AI Visual Planning

• NVIDIA LPR released VFlash, a System-1 VLM visual planning framework that replaces autoregressive decoding with generalist decoding of visual primitives (tracks, boxes, points) for Physical AI.
• VFlash reduces average 3D displacement error (ADE) by 42.6% across three robot datasets, with specific improvements of 59% on InternA1 (15.3cm to 6.2cm) and 61% on AgiBot (10.5cm to 4.1cm).
• Latency is significantly reduced on edge hardware: Jetson Thor sees 5x faster robot inference and 3x faster driving inference, while DGX Spark achieves 12x and 22-49x speedups respectively.
• Integrated with WAM (Cosmos-Predict2.5 based), VFlash achieves a 70.2% average success rate on MolmoSpaces, outperforming the 62.6% baseline and 61.0% autoregressive plan.

Technical Mechanism

• A lightweight drafter (5-layer non-causal transformer) predicts complete visual primitives in a single parallel pass using frozen VLM context.
• The target VLM updates context and extracts features for downstream action models, with alignment ensuring decoding blocks match primitive boundaries.

Performance Benchmarks

• On DAVIS, VFlash reduces average displacement error by 27% compared to MolmoMotion-AR and generates single rollouts 25x faster on H100.
• For object localization on COCO and Dense200, box generation rate increases by 2.6-2.8x with similar F1 scores to autoregressive targets.
• Driving performance on NAVSIM and Waymo shows VFlash-RL achieving a PDMS of 90.7 and ADE3s of 1.22m, maintaining competitiveness with other RL-based planners.

Deployment and Optimization

• Optimized for TensorRT-Edge-LLM and TensorRT engines on Jetson Thor, DGX Spark, and H100.
• Uses FP16 VLM engines with KV cache (no quantization) and specific attention kernels (FMHA for vision/prefill, XQA for decoding/verification).
• Action models utilize BF16 TensorRT engines with head-dimension-128 CuTe attention on Thor and Spark.