NVIDIA VFlash Reduces Robot Trajectory Prediction Latency by 5.1x

Details

Release of the VFlash technical report, which transitions the generation of robot and autonomous driving trajectory coordinates by Vision-Language Models (VLMs) from sequential autoregressive methods to parallel decoding
In the Jetson Thor environment, total latency for the MolmoSpaces task was reduced by approximately 5.1x, from 3552.7ms to 703.3ms, with a 12-42x reduction in the decoding stage alone
3D trajectory prediction accuracy (ADE) improved by 18-61% compared to the autoregressive method, achieving an average success rate of 70.2% in MolmoSpaces simulations, which is 7.6%p higher than the baseline model
For bounding box (COCO, Dense200) tasks, the F1 score drops sharply with format checking alone, but recovers to 61.78% with the addition of alternative generation paths while improving speed by 2.6x
Training and evaluation code released under the Apache 2.0 license in the Eagle repository, with model checkpoints to be released later

Key point