LG AI Research 381

At NeurIPS 2023, a wide range of topics were covered beyond improving the performance of LLMs (Large Language Models), including reliable evaluation, Alignment with humans, training and inference optimization, and Privacy/Bias.

The major research trends are as follows.

• LLM Performance and Safety: Pointed out that the Emergent Abilities of LLMs may be a property of the evaluation Metric rather than an actual capability, and analyzed why Safety Training fails in certain situations.
• Tools: Introduced Toolformer, which enables a model to teach itself to use external APIs, and ToolkenGPT, which tokenizes tools for efficient use.
• Efficient Learning: Presented QLoRA, which fine-tunes parameters in a 4-bit quantized state, and research analyzing the impact of Multi-epoch training in data-constrained settings.
• Trustworthiness: Warned of the potential risks of high-performance models such as GPT-4 through the DecodingTrust benchmark, which comprehensively evaluates toxicity, bias, robustness, and more.
• RL (Reinforcement Learning): DPO (Direct Preference Optimization), which directly optimizes preferences without building a separate reward model, drew attention.