Inco AI Releases Splash, an Inference Engine Dedicated to Apple Silicon

Inco AI has released Splash, an inference engine dedicated to Apple Silicon that sacrifices generality to optimize model-specific kernels, Draft Models, and memory planning. This engine is designed to overcome the limitations of multi-request and long-context processing required in coding agent environments.

Key Technologies and Performance

Splash uses the DFlash 2 speculative decoding technique as its primary path, adopting a method where the entire block is predicted in parallel and then verified once by the target model. On an M5 Pro (48GB) device, the Qwen3.8-27B model achieved output speeds of 74 tokens per second for short prompts and 54 tokens for 32K prompts. Notably, the Time to First Token (TTFT) upon a 32K context cache hit is 282ms, which is 6.6 to 7.3 times faster than competing engines.

Benchmarks and Comparison

According to SPEED-Bench results, Splash improved decoding speed by 2.0x for the 27B model and 1.7x for the 35B model compared to existing engines. When processing 4 concurrent requests, the aggregate throughput is 2.0 to 3.9 times higher than the runner-up engine, and it can handle all 16 concurrent requests for the 27B model on a 48GB device. This contrasts with general-purpose engines, which handle only 9.

Installation and Requirements

• Requirements: M3 chip or higher, macOS 26.4 or higher, 36GB or more unified memory (48GB recommended)
• Installation: brew install incoai/tap/splash
• Execution: splash serve --model incoai/Qwen3.8-27B-Splash (serves at 127.0.0.1:8000)
• License: Apache 2.0 (model weights are subject to a separate license)

It supports OpenAI and Anthropic API compatibility and has been integrated into LM Studio Bionic as a first-class inference engine. Currently, two packages, Qwen3.8-27B and Qwen3.6-35B-A3B, are provided with 4-bit quantization and DFlash 2 draft models included.