Mistral 3.5 MLX 4-bit Released

A conversion of Mistral Medium 3.5 128B quantized to MLX 4-bit for Apple Silicon has been posted on LM Studio and Hugging Face. It supports Text, Vision, Thinking, Tool Calling, and is stated to require a minimum of ~72GB RAM.

• The original is FP8 static 133.6GB, and the converted version is about 70GB at 4-bit affine, group_size=64.
• The vision encoder remains in BF16, and projector and lm_head are also left unquantized, with vision understanding described as identical to the original FP8.
• The architecture is 127.7B dense, an 88-layer text model, a 48-layer Pixtral vision encoder, 262K context, and a 131K token vocab.

Using reasoning_effort="high" outputs [THINK]...[/THINK] thinking blocks, and native tool calling is also supported.

However, repetition loops can occur after 500~1000 tokens across all runtimes and quantizations, so repeat_penalty 1.1~1.15, top_k 20, and a max_tokens limit are recommended.

It also notes that a local patch was applied to fix a sanitize bug in mlx_vlm's mistral3, addressing key-handling issues for model.vision_tower.* and model.multi_modal_projector.*.

The license is Modified MIT, allowing commercial use up to $20 million in monthly revenue.