AI Briefing#memory-efficiency
The latest AI and developer news about #memory-efficiency, with the original source and a short summary.
The latest AI and developer news about #memory-efficiency, with the original source and a short summary.
PACE uses periodic rounding and channel-aligned scaling to maintain accuracy at four bits, matching 8-bit performance on models up to 35B parameters.
A drop-in component for transformers that reduces peak memory by 1.5-3.4x while maintaining accuracy.
The method stores precomputed n-gram counts on host memory to improve validation loss and enable domain adaptation without training.
After upgrading to Valkey 9.1, Imweb cut overall CPU usage by 86.6%.
A new optimizer called SkewAdam reduces optimizer state memory for MoE training from 50.6GB to 1.29GB, enabling 6.78B MoE training on a single 40GB GPU.
ax-grep reproduces the browser's accessibility tree to improve agent web search token usage by 3x and reduce memory usage by 15x.
It proposes a new method that dramatically reduces memory usage by dividing the network into blocks and training them independently.
With GaLore technology, even 7B-scale LLMs can be efficiently trained on consumer GPUs.
Reformer uses techniques like LSH attention to minimize memory usage while processing long sequences of up to 500,000 tokens.