LG AI Research: ICLR 2024 Research Summary

Recently, Generalization—the ability to perform reasonable reasoning even on data not seen during training in order to solve real-world problems—has become a core aspect of AI technology. To this end, Foundation Models pretrained on massive amounts of data are being widely used.

At ICLR 2024, LG AI Research presented the direction of foundation model development from three perspectives.

1. Interpretability: Research that theoretically and experimentally understands why a model shows high generalization performance. In particular, research examining how models learn causal relationships from a Causal Inference perspective—thereby achieving robust performance even under changes in data distribution—is drawing attention.
2. Sustainable AI: Research on methods to make foundation models more cost-efficient.
3. Safe AI: Research on methods to ensure the safety of large-scale models.

In particular, unlike statistical models that assume a fixed data distribution, causal models have the characteristic of being more Robust to distribution shifts by learning the Causal System hidden behind the data.