Google releases next-generation VLM PaliGemma 2

PaliGemma 2 is a next-generation vision language model (VLM) that combines a SigLIP image encoder with a Gemma 2 text decoder.

Key features are as follows:

• Various model sizes: It is offered in 3B, 10B, 28B parameter models, allowing users to choose the balance between performance and efficiency according to their needs.
• Flexible resolution support: It supports three input resolutions: 224x224, 448x448, 896x896.
• Optimized for fine-tuning: It is designed to be easily adapted to downstream tasks, and 3B and 10B models fine-tuned on the DOCCI dataset are also released together, providing sophisticated captioning performance.

The model follows the Gemma License, so commercial use, redistribution, and fine-tuning are possible. Model repositories, Transformers integration, fine-tuning scripts, and a VQAv2 demo are immediately available via Hugging Face.