AMD Publishes Paper on Numerical Behavior Modeling of GPU Matrix Cores

Matrix multipliers in the latest GPUs do not fully comply with the IEEE 754 floating-point standard, making it difficult to guarantee reproducibility of computation results across devices through software control alone. To address this, a paper was published presenting accurate numerical behavior models for AMD GPUs' CDNA 1, 2, and 3 architectures.

Research Background and Challenges

Matrix computation units in GPUs vary by architecture and vendor in terms of accumulator width, rounding behavior, normalization timing, intermediate overflow handling, and subnormal processing methods. Because these implementation details were not documented, it was difficult to identify the root cause when computation results were inconsistent.

Modeling Methodology and Verification

The research team conducted studies on three architectures: MI100 (CDNA 1), MI210/250 (CDNA 2), and MI300A/300X (CDNA 3).

• Test Vector Design: Designed test vectors targeting the numerical characteristics of all supported input formats.
• Iterative Refinement: Built a MATLAB-based software model that matches the hardware at the bit level by repeating a test refinement loop after random testing.
• Verification: Verified reproducibility against the hardware using a test suite of 10 million random input sets.

Applications and Implications

The developed models were used to perform a proof of concept (PoC) quantifying application-level accuracy differences between AMD matrix cores and NVIDIA tensor cores. This provides a critical benchmark for ensuring numerical consistency across hardware and debugging during AI model training and inference.