#training-efficiency

The latest AI and developer news about #training-efficiency, with the original source and a short summary.

CoWA and MALA Attention Mechanisms Offer Significant Speedups for Long-Context Models

CoWA and MALA achieve up to 8.6x attention-operator speedups at 128K tokens while maintaining comparable model capabilities to FullAttn.

·1·2026.09.29 14:00

JiT-DDT Architecture Improves Text-Image Model Training Speed by 3.6x Compared to Linum v2

The JiT-DDT architecture, adopting a pixel-space encoder-decoder structure, achieved faster convergence and improved image quality while reducing GPU usage by 3.6x compared to Linum v2.

·1·2026.09.17 01:00

A 3-Stage Layer Management Technique for Saving GPU Memory

A new training technique is proposed that interleaves the management of 3 layer states to improve GPU memory efficiency.

·2·2026.06.30 07:00

CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs

CODA, a GPU kernel abstraction that solves the memory bottleneck in Transformer training, has been released

·1·2026.05.22 13:00

Token AI Announces New Optimizer STAM

Token AI has announced STAM, a new optimizer that enhances training stability and maximizes memory efficiency.

·1·2026.05.08 14:00