LG AI Research 477

Recently, LLMs (Large Language Models) have shown outstanding performance across various areas such as natural language understanding and generation, but as model size increases, they face challenges including massive computational demands, high costs, and degraded inference speed.

To address these issues, recent research has focused on building sustainable LLMs through more efficient algorithms and architectures. This article examines methodologies for optimizing model performance and resource usage, centered on key research presented at ACL 2024.

The LayerSkip (Meta) study was designed to increase the inference speed of LLMs. Based on the insight that a model's Early Layers can already produce accurate predictions, it proposes the following 3-stage approach.

• Layer Dropout and Early Exit Loss Training: Improves training efficiency so that inference can be terminated early at specific layers of the model.
• Inference via Early Exit: Reduces computational cost by skipping computation after predefined layers.
• Self-speculative Decoding: Generates initial predictions (Drafting), then verifies and corrects them through the remaining layers, preventing accuracy loss while reducing memory usage.

Experimental results showed that LayerSkip overwhelmingly outperformed the existing Llama2 7B model on code generation tasks, achieving approximately 1.83x faster speed compared to the baseline model on both CNN-DM and code generation tasks.