#zeroth-order-optimization
The latest AI and developer news about #zeroth-order-optimization, with the original source and a short summary.
The latest AI and developer news about #zeroth-order-optimization, with the original source and a short summary.
Dust introduces a zeroth-order optimization method for transformer pretraining that perturbs activations per token, achieving competitive loss with backpropagation and significantly outperforming weight-space Evolution Strategies like EGGROLL.