Upstage Unveils Open Ko-LLM Leaderboard

Upstage has unveiled the Open Ko-LLM Leaderboard to systematically evaluate the performance of Korean LLMs and foster a transparent research environment.

This leaderboard is notable for building a private test set, unlike existing public benchmarks, to address the issue of data contamination and enhance the fairness of evaluation.

Evaluation is conducted through 5 core tasks that reflect the unique characteristics and culture of the Korean language:

• Ko-ARC: Scientific reasoning and problem-solving ability
• Ko-HellaSwag: Situational understanding and prediction ability
• Ko-MMLU: Language comprehension and knowledge level across various fields
• Ko-Truthful QA: Accuracy and truthfulness of factual relationships
• Ko-CommonGEN V2: Understanding of Korean common sense and cultural context

To date, over 1,000 models have participated, establishing it as a key indicator for gauging the performance of Korean LLMs.