AI BriefingRunning CUDA+ROCm simultaneously
·2026.05.01 07:46
Key point
A build method for using CUDA and ROCm simultaneously with llama.cpp on Windows was shared.
Details
A build and run procedure was shared for connecting llama.cpp to both CUDA and ROCm simultaneously on Windows, loading models without Vulkan.
- In CMake,
GGML_HIP=ON,GGML_CUDA=ON, andGGML_BACKEND_DL=ONwere all enabled together. CMAKE_C_COMPILER,CMAKE_CXX_COMPILER, andCMAKE_HIP_COMPILERwere set to ROCm 6.4'sclang-cl.exe, whilenvcc.exewas used as the CUDA compiler.GGML_CPU_ALL_VARIANTS=ONcaused compile errors, requiring thealderlakeCPU variant to be commented out inggml/src/CMakeLists.txt.- When running
MiniMax-M2.7-UD-Q4_K_Son a Ryzen 5950X setup, 63/63 layers were offloaded to GPU, with the CUDA buffer showing 83,650.42 MiB and the ROCm buffer showing 40,314.35 MiB.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
