Kakao Unveils Multimodal Language Model Kanana-v, Outperforming GPT-4o in Korean OCR

Kakao's Kanana ⍺ organization revealed the performance and development process of its self-developed multimodal language model Kanana-v. Based on the research outcomes of Honeybee, a CVPR 2024 highlight paper, the model demonstrates competitive performance in both Korean and English, despite currently being in the preview stage.

Korean Performance: Overwhelming Lead

In evaluations of Korean-specific benchmarks (OCR, document understanding, problem-solving, and Korean knowledge), Kanana-v recorded an average score of 75.1, significantly outperforming GPT-4o (61.9) and GPT-4V (39.8). Notably, in the Korean OCR domain, it achieved 84.4, demonstrating overwhelming reading capability compared to GPT-4o (49.0). In the Korean knowledge domain, it recorded the second-highest score among commercial models, indicating a need for improvement through additional data acquisition.

English Performance: Global Competitiveness Secured

The average score on English benchmarks was 75.9, comparable to GPT-4o (76.8) and slightly exceeding Qwen2-VL (75.8). It scored 94.5 on DocVQA, higher than GPT-4o (92.8), and surpassed GPT-4o (73.6) on OCRBench with a score of 81.8.

Technical Features and Training Strategy

Kanana-v adopts the Native resolution approach, which maintains the original resolution without fragmenting images to handle high-resolution image processing. This principled approach preserves full context but requires expansion work on the pre-trained vision encoder. To connect the vision encoder and language model, it uses the C-Abstractor projector, which dynamically adjusts the number of tokens while preserving spatial information.

Training proceeded in four stages:

1. Projector pre-training: Pre-training the projector with image caption data
2. Continued pre-training: Training the entire model with high-quality captions, OCR, and document data
3. Supervised fine-tuning: Learning question-answering capabilities with VQA format data
4. Human preference alignment: Aligning with user preferences and safety using RLHF techniques

Additionally, to prevent catastrophic forgetting, the learning rate was adjusted while tracking language benchmarks, and high-quality data including Chain-of-Thought (CoT) reasoning evidence was constructed to enhance logical explanation capabilities.

Future Plans

Kakao is pushing to enhance Kanana-v's multi-turn conversation capabilities and model lightweighting for on-device serving. It is augmenting Korean person data to improve person identification accuracy and is concurrently developing Kanana-o, an integrated multimodal model supporting audio and video inputs.