Cactus Compute Releases Ultra-Compact 'Needle 3' Model (29MB), Claims DeepSeek V4 Flash-Level Performance

Cactus Compute has released 'Needle 3', an ultra-compact foundation model targeting edge devices and on-device AI.

Core Architecture and Performance

• Intelligence Laddering: Adopts a subnetwork structure within a single model where capacity increases from 2 layers (2L) to 20 layers (20L).
• Size and Efficiency: The total scale is 121M parameters, deployable at sizes of 29MB to 121MB when applying CQ2 quantization. It features high MFLOPs efficiency per token.
• Performance Claims: The 4-layer subnetwork (29M parameters) surpassed or achieved performance levels equivalent to DeepSeek V4 Flash's cloud API during fine-tuning. Specifically, it outperforms models 10x larger in Mobile Actions and shows performance similar to models 2-3x larger in Extraction tasks.
• Speed: Runnable on devices such as Raspberry Pi, providing high inference speeds through its lightweight structure.

Fine-Tuning and Deployment

• Fine-Tuning: LoRA is applied to a frozen base model (20 layers), after which subnetworks with specific layer counts are built and deployed as 4-bit .cact files. When fine-tuned with the DroidCall dataset, performance across all subnetworks improved by 18 to 36 points.
• Supported Platforms: Supports various platforms including macOS, Linux, Windows, Android, iOS, tvOS, watchOS, and browsers (WASM).