AI BriefingDeDLOC: Internet-Based Collaborative Distributed Training
Key point
The DeDLOC algorithm has been released, which trains language models by leveraging the resources of multiple users even in slow internet network environments.
Details
Training large language models (LLMs) requires massive computing resources, which are difficult for individuals or small organizations to afford.
Existing distributed training methods presuppose ultra-fast networks, so collaborative training over the internet has had low effectiveness due to data transfer bottlenecks.
HuggingFace proposes a new algorithm that can adapt to network and hardware constraints, called DeDLOC (Distributed Deep Learning in Open Collaborations).
Using this technology, they successfully pre-trained sahajBERT, a Bengali-language model, together with 40 volunteers, and this achieved performance comparable to existing models trained using large-scale accelerators.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.