Developing Large Language Models (LLMs) with Cross-Cultural Considerations - LG AI Research Blog

As NLP technology advances rapidly, model size and data volume are exploding. However, since human thought and behavior are deeply connected to the culture one belongs to, cultural factors must not be overlooked when models imitate humans.

LG AI Research is one of the few organizations in Korea capable of training and deploying LLMs from scratch, and recently released EXAONE 3.0 7.8B, a model that shows excellent performance on Korean and English benchmarks. This post covers cultural alignment research discussed at the C3NLP workshop at ACL 2024.

Benchmarks for Measuring Cultural Alignment

• CDEval: Based on Hofstede's cultural dimensions theory, it measures 6 cultural dimensions including power distance, individualism/collectivism, and uncertainty avoidance.
• KoBBQ: Designed to address the fact that the existing BBQ benchmark fails to reflect the specificities of Korean culture, it measures social biases specialized for Korean culture.

Modeling Research Beyond simply measuring performance, how a model achieves cultural alignment also matters. Since existing alignment datasets mainly reflect the preferences of English-speaking users, simply translating them cannot fully capture the cultural preferences of speakers of the target language. The research analyzes how cultural alignment affects a model's judgments, using Commonsense Morality as a criterion.