Edge0-145B-NVFP4: 145B MoE model, running locally with NVFP4 quantization

Edge0-145B-NVFP4 is a text generation model featuring a Mixture of Experts (MoE) architecture with 145 billion parameters. Its core focus is lowering the barrier to local inference for large models by applying NVFP4 4-bit quantization. The total file size is approximately 6GB, significantly reducing storage and memory overhead compared to existing 145B-class models.

It is optimized for coding, agent tasks, and interactive interactions. It is compatible with major inference frameworks such as llama.cpp, vLLM, and Ollama, and can be easily integrated into existing codebases via an OpenAI-compatible API. Using Docker Model Runner allows for immediate execution even in containerized environments.

This model targets use cases on edge devices or in resource-constrained environments, as indicated by the edge-inference and latent-moe tags. It is suitable for developers who want to run high-performance LLMs on personal developer machines or small servers rather than large-scale data center GPUs. It is released under the community license (edge0-community-model-1.0).