Picksam3: Track from text to video across 270,000 concepts

Detect and segment every object instance in images and videos with a single short text prompt. Unlike existing models, it finds every instance of a specific concept without omission, offering scalability to handle 270,000 unique concepts.

It combines visual prompts such as points, boxes, and masks with text to improve precision. By inputting positive and negative examples together, it accurately isolates only the desired targets. A single model handles everything from static image segmentation to dynamic video tracking.

It achieves human-level performance of 75–80% on the new SA-CO benchmark. It handles over 50 times more concepts than existing benchmarks, significantly expanding the limits of open-vocabulary prompting. It is compatible with the Transformers library, enabling pipeline-based inference.