Upstage and NIA Jointly Build Korean Open LLM Leaderboard

Upstage is teaming up with NIA to jointly build a Korean-style Open LLM leaderboard. The two organizations plan to use Korean-language training data from AI Hub to establish the Open-Ko LLM Leaderboard within this month.

This leaderboard is an evaluation platform that expands Hugging Face's Open LLM Leaderboard to fit the Korean environment. In addition to the existing 4 metrics of reasoning, common sense, language understanding, and hallucination prevention, it will also evaluate and rank models based on additional criteria such as ethics and expert knowledge, drawing on reliable data from NIA and academia.

Upstage ranked 1st with its own generative AI model on the Hugging Face Open LLM Leaderboard last August. Through this collaboration, the company aims not only to develop high-quality LLMs that reflect Korean culture and sentiment, but also to build an ecosystem for domestic LLM independence.

• NIA will also join the 1T Club, providing high-quality data.
• Partners will be able to use Upstage's LLM API at discounted prices and will also share in the revenue from the API business.
• Data will be used only for training purposes, with enhanced security and privacy protections to prevent extraction of the original text.

NIA collects and processes data across various fields through AI Hub and provides it free of charge to developers. It aims to build over 1,300 types of data by 2025, and through this agreement, it will help expand Korea's domestic AI ecosystem and strengthen the competitiveness of Korean-style LLMs.