Needle 2 - A 14MB Agentic LLM for Smartphones, Wearables, Smart Home, and Robots

Cactus has released Needle 2, an on-device LLM with 45M parameters and a 14MB size. It targets low-cost edge devices that use only GPUs/NPUs and tens of MBs of RAM, such as smartphones, wearables, smart home devices, and small robots.

Needle 2 focuses on tool calling and structured data extraction rather than general conversation.

• Converts natural language into functions with typed parameters for device capabilities
• Generates structured outputs in enum, array, and object formats based on schemas
• Prevents invalid JSON and malformed structures through grammar compilation
• Supports multi-step tasks involving sequential calls to multiple tools
• Rejects irrelevant requests with empty calls and provides a confidence score

The model was trained assuming 2-bit quantization, with weights, activations, and KV cache jointly optimized for deployment environments. The provided binary is distributed as a dependency-free C++ single executable that includes the model, tokenizer, and grammar compiler, supporting everything from Cortex-M to x86 and WebAssembly.

According to public materials, Needle 2 was measured using the deployment binary on the Mobile-Actions benchmark, delivering performance of over 800 tokens/sec prefill and over 500 tokens/sec decode on a Raspberry Pi 5. Fine-tuning for custom tool vocabularies is supported on Mac and PC, and it is currently used for local voice command processing in the Index 01 app by wearable company Pebble.