AI BriefingHugging Face expands AMD GPU acceleration support
·2023.12.05 09:00
Key point
Hugging Face announced optimized support for running LLMs on AMD Instinct GPUs without any separate code modifications.
Details
Through the partnership between Hugging Face and AMD, an environment has been established for running the latest AI models with optimal performance on AMD Instinct GPUs (MI250X, MI300 series, etc.).
The key updates are as follows:
- No Code Change Execution: Existing code written for NVIDIA GPUs (
torch.device("cuda")) can be used as-is on AMD GPUs without modification. - Integration of Latest Acceleration Technologies:
- Flash Attention v2 (AMD open-source version)
- vLLM's Paged Attention and Fused Kernels for ROCm
- DeepSpeed (officially validated on ROCm-based GPUs)
- GPTQ (quantization support via AutoGPTQ and Transformers integration)
- ONNX Runtime (running ONNX models via ROCmExecutionProvider)
- Performance Benchmarking Tool: Optimum-Benchmark enables performance measurement across distributed environments and various optimization/quantization configurations.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.