SenseNova-Vision Released as Unified Model for Multiple Vision Tasks

SenseNova-Vision has released a framework that handles various computer vision tasks, such as object detection, segmentation, and depth estimation, with a single generative model without separate output heads. Based on BAGEL-7B-MoT, this model utilizes existing paths for generating text and images to simultaneously learn structured text records and spatially aligned visual maps.

Unified Approach

Unlike existing models, it does not add task-specific dedicated modules; instead, it integrates text answers via next-token prediction and visual answers via rectified-flow learning in the VAE latent space. This allows a single model to generate outputs in various forms, including object detection, OCR, depth maps, and segmentation masks.

Performance and Limitations

In benchmark results, SenseNova-Vision outperformed the specialized model X-SAM on ReasonSeg and Interactive Box, but lagged behind on RefCOCO and Interactive Point. This is interpreted as an approach integrating multiple output forms rather than replacing all specialized models. Additionally, prompt sensitivity and limitations in geometric prediction accuracy are noted, requiring verification of output stability for real-world applications.

Execution Environment and License

Model weights are available for download on Hugging Face, and inference requires a PyTorch 2.5.1+ and CUDA 12.4 environment. It has been verified on a single A800 80GB GPU using BF16, while 8x80GB GPUs or more are recommended for full benchmarks. The code follows the Apache-2.0 license, and model weights follow the CC BY-NC 4.0 license.