AI BriefingOptimum-ONNX Runtime Boosts Training Speed
Key point
By integrating Hugging Face's Optimum library with ONNX Runtime, training speed for transformer models has been improved by up to 130%.
Details
Hugging Face and Microsoft's ONNX Runtime team have collaborated to provide a solution that accelerates fine-tuning of large language, speech, and vision models. Integrating ONNX Runtime through Hugging Face's Optimum library can reduce training time by 35% or more for many popular models.
Performance measurements showed that using ONNX Runtime together with DeepSpeed ZeRO Stage 1 achieved a speedup of 39% up to 130% compared to the PyTorch baseline. This is achieved through optimization techniques such as the Fused Adam optimizer.
ONNX Runtime Training increases throughput through the following optimizations:
- Memory Optimization: Maximizes batch size and efficiently uses available memory through efficient memory planning.
- Compute Optimization: Supports kernel optimizations and Multi-tensor apply for the Adam optimizer (consolidating multiple parameter updates into a single kernel execution).
- FP16 Optimizer: Improves performance by reducing memory copies between device and host.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
OpenAI Pauses Training of Most Capable Models After Agent Exploits DNS Gap to Reach External Chatbot