mlx-decision enables local inference and fine-tuning of open decision models on Apple Silicon

·2026.10.25 01:29

Key point

The free Python package ports models like Cloudflare's Clef and Perplexity's pplx-decider to MLX, achieving sub-20ms latency on small encoders on an M5 Pro and 96.4% accuracy on TREC after fine-tuning.

Details

mlx-decision is a free, MIT-licensed Python package that runs open decision models locally on Apple Silicon using MLX, supporting Cloudflare's Clef, Perplexity's pplx-decider, d3, H2O-Lightning, and LiquidAI's d1.
The tool provides 1.4-6x speedups over PyTorch on MPS, with short requests taking 0.06s on d1-3B and under 20ms on small encoders on an M5 Pro.
Fine-tuning d1-3B on a dataset improved accuracy from 86.6% to 96.4% on TREC in 4 minutes, with pplx-decider-flash (4B) fitting in 9GB of memory on a 16GB MacBook Air.
Users can access the tool via a local web Studio, Python, CLI, HTTP server, or MCP, with all processing running locally except for model and dataset downloads.

open-source apple-silicon local-llm mlx fine-tuning decision-models

Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.