AI Briefing#apple-silicon
The latest AI and developer news about #apple-silicon, with the original source and a short summary.
The latest AI and developer news about #apple-silicon, with the original source and a short summary.
The open-source library supports exact k-NN, k-means, NN-Descent, and CAGRA algorithms, outperforming CPU-based hnswlib in both build time and search speed.
Leaker Aaron Perris revealed evidence of Apple's 'Harbor' app, which reportedly turns Macs into private AI inference servers and pools memory across multiple devices via Thunderbolt 5.
The model uses a 255-way readout head to output probabilities over candidate answers in a single forward pass, avoiding token generation entirely.
uzu, a Rust inference engine for Apple Silicon, recorded a speed of 142.3 tok/s on the M5 Max.
The packages wrap Apple's new SpeechAnalyzer API with NVIDIA and Alibaba models, achieving 1.55% WER on LibriSpeech test-clean, and enable typed decision outputs via six backends including MLX and llama.cpp.
The free Python package ports models like Cloudflare's Clef and Perplexity's pplx-decider to MLX, achieving sub-20ms latency on small encoders on an M5 Pro and 96.4% accuracy on TREC after fine-tuning.
Open-source engine Kapitsa matches Recursive's native performance on short contexts and leads by 1.30x at 128K context on Apple Silicon.
PyTorch 2.15 improves Mac GPU MoE decoding by 5-6.5x and supports FP8.
The update introduces native FP8 support and optimizes Metal kernels, reducing driver memory peaks from 270.135 GiB to 8.108 GiB for video decoding.
The Apache 2.0 licensed tool uses custom Metal kernels and speculative decoding to outperform standard MLX engines on local coding models.
The tool transcribes one hour of audio in approximately 13 seconds on a base M3 MacBook Air using the Neural Engine.
The system aggregates idle GPUs from macOS devices to execute large-scale machine learning inference without new hardware procurement.
Larmor is a free, MIT-licensed tool that enables two-way voice interaction with agents like Claude Code and Codex using local models on Apple Silicon.
rGPU is an open-source tool that enables PyTorch applications to execute operations on remote NVIDIA GPUs while keeping the client application local, supporting users on systems like Macs without CUDA.
The open-source tool runs AI coding agents in isolated Linux VMs on macOS 27+, keeping API keys on the host via a Swift proxy.
The release includes Kev-27B and an updated Kev-9B, offering Apache-2.0 weights compatible with TypeSafe's Jev API for local and cloud deployment.
Hugging Face Transformers has released a feature that allows direct inference of GGUF models in a compressed state without dequantization on Apple Silicon Macs, utilizing llama.cpp's ggml Metal kernels.
Liquid AI has released LFM2.5-VL-3B-DSpark, a draft model for speculative decoding of its VLM LFM2.5-VL-3B, which improves decoding speed by up to 3.13x on the Apple M5 Max.
Underdog has released 'Husky', a model-specific inference engine that is faster than MLX across all measured metrics, achieving up to 4.5x performance gains particularly in editing tasks.
Jared Palmer released Kev, a lightweight decision-making model based on Qwen3.5, offering simultaneous processing of Yes/No, multiple-choice, and rating tasks with Apple Silicon support.
Hugging Face Transformers now directly supports the GGUF format from llama.cpp, simplifying local inference on Apple Silicon.
Hugging Face has hired oMLX developer Jun Kim to strengthen support for the Apple Silicon-based MLX ecosystem and local AI infrastructure.
Inco AI has released Splash, an Apple Silicon inference engine that applies model-specific kernels and draft models.
Edge0 released an inference framework that runs 35B MoE models on 3GiB of memory using SSD expert streaming technology.