AI BriefingLaunch HN: General Instinct (YC P26) – Frontier models for edge devices
Key point
They unveiled **InstinctRazor**, a technology optimized to run MoE models on edge devices.
Details
General Instinct has open-sourced InstinctRazor, a technology for efficiently running large MoE (Mixture-of-Experts) models on low-spec edge hardware.
Through this technology, they successfully compressed the Qwen3.5-122B-A10B model, roughly 245GB (BF16) in size, into a 48GiB GGUF format. Despite being smaller than Gemma-4-26B-A4B, the compressed model recorded higher performance on benchmarks such as MMLU-Pro and GPQA-D.
The key technical features are as follows:
- Selective quantization: Always-active components such as routers, norms, and Gated-DeltaNet/SSM layers are preserved, while only the routed experts portion is aggressively quantized.
- On-policy distillation: Used to recover performance lost during the quantization process.
- Memory efficiency: By streaming experts from system RAM, the model can run even on a 'small GPU' setup. Peak VRAM usage is around 7.6-8GB for an 8k context window.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Apple Silicon-only inference engine 'Lily' improves decoding speed by 35% over MLX-LMNVIDIA Releases NVFP4 Quantized Model of DeepSeek-V4-Pro-0813