Carbon Open DNA Model Released

Hugging Face has released Carbon, an open DNA foundation model family. The flagship model Carbon-3B reportedly matches the performance of Evo2-7B, described as the current SOTA, while being 275x faster.

• Since DNA differs from natural language and has a lot of noise and redundancy, they didn't just reuse LLM-style recipes as is.
• Instead of nucleotides, the tokenizer uses deterministic 6-mers, making sequences 6x shorter and lowering attention costs.
• To reduce loss spikes late in training, they switched midway from cross-entropy to a factorized loss (FNS).
• The data progressively mixed in functional DNA and mRNA, with ratios tuned via ablation.
• A technical report and a beginner-friendly biology demo were also released.