[AI Infrastructure] Network Design Considerations for RoCEv2-based AI GPU Clusters

kt cloud is currently conducting technical verification for a transition from the proprietary structure of InfiniBand to open Ethernet-based RoCEv2. For CSP operators, the key challenge is finding the 'sweet spot' that maximizes network fabric utilization while minimizing Tail Latency.

InfiniBand supports lossless communication at the hardware level, but RoCEv2 has the advantages of the openness of the Ethernet ecosystem and low TCO. However, to build a lossless environment via RoCEv2, precise configuration of PFC (Priority Flow Control), ECN (Explicit Congestion Notification), and the DCQCN mechanism that integrates and manages them is essential.

As next-generation technologies, TIMELY and SWIFT, which perform latency-based congestion control, are drawing attention. In particular, SWIFT prevents the 'victim flows' phenomenon through delay decomposition and guarantees very tight tail latency.

When designing network architecture, it is recommended to use High-Radix Shallow Buffer switches, which can minimize latency, for the backend AI fabric used for GPU-to-GPU communication. On the other hand, Deep Buffer switches are more suitable for the Spine fabric, which needs to handle unpredictable traffic bursts.