Introduction to AWS Interconnect Technology from a Distributed Training Perspective - GPU-to-GPU Communication Using NCCL in AWS Environments

EFA is AWS's high-speed network infrastructure for distributed training, and NCCL is the library that coordinates GPU-to-GPU communication on top of it. This article summarizes how NCCL performs collective communication such as AllReduce, AllGather, ReduceScatter in the AWS environment, and why it is central to large-scale training performance.

Unlike CPU-centric MPI, NCCL is designed for GPU-to-GPU communication, and on AWS it uses libfabric and EFA via the aws-ofi-nccl plugin. MPI implementations, on the other hand, use libfabric directly, and in particular Intel MPI's built-in libfabric lacks an EFA provider, so without separate configuration it can fall back to TCP — requiring settings such as I_MPI_OFI_LIBRARY_INTERNAL=0 and FI_PROVIDER=efa.

Collective communication is a method that reduces one-to-one connections and enables all GPUs to cooperate together. For example, AllReduce sums each GPU's gradients and distributes the same result to all GPUs, AllGather combines fragmented data and delivers the whole to everyone, and ReduceScatter splits and stores the summed result to reduce memory usage.

NCCL also automatically reads the hardware topology. It identifies NVLink, NVSwitch, PCIe, and NUMA structures to group nearby GPUs first, and selects the NIC path closest to the EFA card to reduce communication latency. After that, it automatically selects Ring and Tree algorithms depending on message size, and splits data across multiple channels and chunks for pipelining to maximize bandwidth.

From an operational standpoint, most tuning is automated in the latest stack, so it's better to use default values rather than excessively adjusting environment variables. The article recommends FI_EFA_USE_HUGE_PAGE=0 as a minimal setting, and suggests using NCCL_DEBUG=INFO only when troubleshooting is needed, to check whether EFA is recognized and which protocol is selected.**