NVIDIA's LocateAnything for Faster Grounding

Existing VLMs (Vision-Language Models) used an autoregressive approach that generates coordinates token by token when performing visual grounding and detection. This approach breaks down box geometry into individual tokens for processing, causing a bottleneck of slow inference speed and poor consistency between coordinates.

To address this, NVIDIA introduced LocateAnything, a unified framework based on Parallel Box Decoding (PBD). PBD treats bounding boxes or points as a single Atomic Unit, predicting the entire coordinate set in parallel in a single step. This maintains geometric consistency within boxes while dramatically improving inference throughput.

LocateAnything offers three flexible inference modes:

• Fast Mode (MTP): Predicts boxes in parallel for maximum throughput, suitable for robotics or on-device environments.
• Slow Mode (NTP): Decodes coordinates autoregressively for high stability, suitable for precise data labeling.
• Hybrid Mode: Uses Fast Mode by default, but switches to Slow Mode when format errors or spatial ambiguity are detected to ensure accuracy.

The model is built on a Moon-ViT vision encoder and a Qwen2.5 language decoder, and was trained using a large-scale dataset called LocateAnything-Data, which was constructed to include 138 million samples and 785 million boxes.