ResBM Cuts Pipeline Communication by 128x

Macrocosmos released ResBM (Residual Bottleneck Models).

It is designed to insert a residual encoder-decoder bottleneck between pipeline boundaries to reduce inter-stage communication volume, while preserving a low-rank identity path.

The paper reports activation compression 128×, and presents that the convergence loss compared to the uncompressed baseline was not significant.

Under compression conditions, results using Muon were the strongest, and the authors interpret this as a direction for decentralized / internet-grade pipeline-parallel training.