AI BriefingLG AI Research 352
Key point
We look at the types of training data and the multifaceted evaluation metrics that determine the performance of LLMs, the core of generative AI.
Details
Amid the rapid spread of generative AI technology, LLM (Large Language Models) have established themselves as a core technology that boosts productivity across industries. The performance of an LLM is determined not only by the model's parameter size but also by the scale and quality of the data used for training.
Training data mainly draws on the following sources.
- CommonCrawl: Data of diverse topics and styles based on web-crawled data
- Wikipedia: Refined, high-quality encyclopedia data
- Code: Programming code and Q&A based on Github(TheStack) and StackExchange
- Others: Academic papers (Arxiv), books (Bookscorpus), medicine (PubMedCentral), and more
Evaluation data is classified from three perspectives to verify the model's capabilities from multiple angles.
- Language generation ability: Sentence completion (LAMBADA) and summarization (XSum) performance
- Knowledge utilization ability: Common-sense-based Closed-book QA and Open-book QA that leverages external knowledge
- Reasoning ability: General common-sense reasoning (PIQA, HellaSwag) and mathematical logic (GSM8K) tests
In addition, knowledge across various domains is evaluated through MMLU, BIG-bench, and others, and datasets optimized for specific purposes (code, bio, etc.) are used to verify the model's accuracy and robustness.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.