Borealis Releases Audio LLM Recipe

Borealis has released a 5B audio-language model for Russian and English along with a reproducible training recipe.

The composition consists of a Whisper3-large audio encoder, a Qwen 4B LLM backbone, and a middle adapter combination. Total parameters are about 5B, but the actual trainable portion is about 500M, tuned using the LoRA + adapter approach.

The scope of the release is broad.

• Data pipeline
• training code and config
• eval results and research notes
• vLLM serving patch and demo

In experiments, the best run recorded an average WER of 19.17%. A 10% text mix showed slight improvement, but 25% text actually worsened performance, and native data proved more advantageous than mixing in English. Segments with heavy noise, echo, and multiple speakers, like webinars, remained the most difficult case.