Hugging Face Supports AMD MI300

Through collaboration between Hugging Face and AMD, AMD Instinct MI300 GPUs have been integrated into the Hugging Face platform as first-class citizens. Users can now prototype and deploy models on AMD hardware using the transformers and text-generation-inference(TGI) libraries without any code changes.

Key updates include the following:

• Infrastructure and CI/CD: Leveraging Azure's ND MI300x V5 VMs, a continuous integration and deployment (CI/CD) environment has been built. This ensures that the transformers and TGI libraries are regularly tested on MI250 and MI300 to guarantee stability.

• Inference Performance Optimization: In collaboration with AMD engineers, Flash Attention v2, Paged Attention, GPTQ/AWQ quantization, ROCm TunableOp, and optimized fused kernels have been integrated to maximize the inference performance of models such as the Llama family.

Through this integration, developers can now efficiently run and operate large-scale AI models using AMD accelerators, similar to the NVIDIA environment.