AI BriefingOFA: Towards Building a One-For-All Model
Key point
Unveiled **OFA**, a multimodal pretrained model that unifies tasks, modalities, and architecture into one.
Details
OFA aims for unification across three dimensions: task, modality, and architecture. The model pursues Task agnostic design, accommodating new tasks without architectural modification; Modality agnostic design, handling diverse inputs without complex preprocessing; and Task comprehensiveness, enabling transfer to unseen tasks by combining existing abilities.
For modality unification, images and bounding boxes are discretized. Images are represented as VQ tokens, and bounding boxes as bins, converting them into a form the Transformer can process. The architecture adopts a general-purpose Transformer encoder-decoder similar to T5, introducing Normformer for training stability and utilizing ResNet blocks for image input.
The core innovation is multitask learning. The model is pretrained through the following 8 tasks:
- Vision-language: visual grounding, grounded captioning, VQA, image-text matching, image captioning
- Vision: detection, image infilling
- Language: text infilling
A text-form instruction is inserted to distinguish each task, designed to enable the model to perform zero-shot generation even on new tasks it was not trained on.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Privatemode Benchmarks GLM-5.3-Flash as Vision-Capable Decision Model with Parity to Jev but Higher Cost
Xiaomi Releases Open-Weight Multimodal Model MiMo-V2.6-Pro… Higher Intelligence and Competitive Pricing Compared to Similar-Sized Models
LLM Mathematical Excellence Driven by Pre-training Data Quality, Not Verifiability