[NeurIPS 2023] Latest LLM Research Trends - LG AI Research Blog

At NeurIPS 2023, various topics were covered beyond just improving the performance of LLM(Large Language Models), including reliable evaluation methods, alignment with humans, training and inference optimization, privacy protection, and bias.

The major research trends are as follows.

• LLM Performance and Safety: Research was presented offering a critical perspective that the 'Emergent Abilities' appearing as LLMs grow in scale may be an illusion caused by specific evaluation metrics, along with research into why Safety Training fails in certain situations.
• NLP and Tool Use: Tool-use capabilities were highlighted, such as Toolformer, where a model learns to use external APIs on its own, and ToolkenGPT, which efficiently extends a fixed language model by adding tool embeddings.
• Efficient Training: Optimization methodologies drew attention, such as QLoRA, one of the PEFT(Parameter-Efficient Fine-Tuning) techniques for efficiently fine-tuning a model's parameters.