[ICLR 2024] Latest Research Trends in the Foundation Model Field - LG AI Research BLOG

Recently, AI models are being evaluated for their core value of generalization ability—the capacity to reason reasonably even on data beyond the training data. Accordingly, the utilization of Foundation Models (FM) pretrained on massive data is rapidly increasing.

At this year's ICLR 2024, the development direction of Foundation Models was addressed from three perspectives.

1. Interpretability: Research on why models have high generalization performance. In particular, research that proved, from a Causal Inference perspective, that models are learning the underlying system of data generation drew attention.

2. Efficiency: Research on how to build Foundation Models more cheaply and efficiently.

3. Safety: Research on how to ensure the reliability and safety of ultra-large models.

In particular, LG AI Research is overcoming technical limitations by focusing on the fact that, unlike statistical models, Causal Models are robust to changes in data distribution, through research such as revealing that AI models understand the world causally.