Rapid-MLX - Ultra-Fast Local AI Engine Exclusively for Apple Silicon

It's a local AI inference engine optimized for Apple Silicon Mac environments by leveraging Apple's MLX framework and Metal compute kernels.

Key Performance and Features:

• Overwhelming Inference Speed: Provides up to 4.2x faster speed compared to Ollama (180 tok/s based on Phi-4 Mini 14B).
• Low Latency: Shows fast responsiveness with TTFT (Time To First Token) of 0.08~0.3 seconds in cached state.
• Tool Calling Optimization: Automatically recovers corrupted output from quantized models into structured format through 17 built-in parsers.
• Hardware-Tailored Mapping: Automatically suggests optimal model configurations by RAM capacity, from 16GB MacBook Air to 256GB Mac Studio.

Technical Differentiators:

• Reasoning Separation: Supports separating the reasoning process of Chain-of-Thought models into a separate field.
• Cache Optimization: Improved TTFT for multi-turn conversations by 2~5x through KV cache trimming and DeltaNet state snapshot technology.
• Smart Cloud Routing: Supports automatic switching to cloud LLMs (GPT-5, Claude, etc.) for large-scale context requests.
• High Compatibility: Compatible with OpenAI API, allowing immediate integration with existing tools like Cursor, Claude Code, LangChain, and Open WebUI.

Other Support:

• Supports multimodal features including Vision, Audio, and Embeddings, and is provided under the Apache 2.0 license.