AI BriefingByteDance releases Cola-DLM
·2026.05.15 20:19
Key point
ByteDance has released checkpoints for Cola-DLM, a continuous latent diffusion language model.
Details
Cola DLM is a hierarchical continuous latent diffusion language model that combines a Text VAE, which converts text into a continuous latent space, with a block-causal Diffusion Transformer (DiT) prior.
This repository publishes Hugging Face-format checkpoints for the paper Continuous Latent Diffusion Language Model.
- The VAE encodes text into a continuous latent sequence and reconstructs it back into tokens.
- The DiT performs latent prior transport via Flow Matching.
- The released weights correspond to the 2000 EFLOPs checkpoint on the RQ4 scaling curve from the paper.
- The tokenizer is the OLMo 2 tokenizer, with a vocabulary size of 100,278.
- Special tokens are provided as
pad_token_id=100277,eos_token_id=100257, andim_end_token_id=100265. - The implementation assumes PyTorch 2.1+ and Transformers 4.40+, and is licensed under Apache 2.0.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.