Upstage Ranks 2nd on HuggingFace Open LLM Leaderboard

Upstage achieved 2nd place on the HuggingFace Open LLM Leaderboard with its self-developed model. With an average score of 64.7, it ranked at the top following Llama 2 70B, and achieved the best performance among 30B-class models.

In particular, it scored 56.5 on the hallucination-prevention metric, surpassing Llama 2's 52.8. In an environment competing against models from global AI companies such as Meta, MS, Stability AI, and Databricks, Upstage explained that it scored about 10% higher on average than Falcon and MosaicML-based models, which have recently held top rankings.

Upstage built this model in-house and submitted it to the leaderboard, achieving these results about 2 months after starting development. It also emphasized its experience building KLUE, the prompt engineering and fine-tuning know-how accumulated while operating AskUp, and the participation of personnel proven at international competitions and conferences as part of the TF.

Small-scale LLMs can be installed and operated directly on internal servers, making them highly useful in the private AI market. Upstage plans to further train the model on additional Korean data to boost Korean-language performance, and anticipates this could expand the trend of companies building and using generative AI on their own without concerns over information leakage.