AI BriefingNVIDIA Parakeet Ported to ggml
·2026.06.01 05:35
Key point
NVIDIA's Parakeet STT model has been ported to run in a C++/ggml environment without Python, greatly improving performance and efficiency.
Details
NVIDIA's Parakeet speech recognition (STT) model has been ported to the ggml engine (the foundation of llama.cpp and whisper.cpp), enabling it to run on CPU and GPU (CUDA, Metal, etc.) without Python or PyTorch.
Key performance and features:
- Accuracy: Guarantees byte-identical output to the NeMo model (WER 0 for f32/f16).
- Speed: Up to 5x faster than NeMo on GPU for large models, and about 1.86x faster with CPU quantization.
- Efficiency: Reduces memory usage by about 2x, processing 1 hour of audio in about 6 seconds on GPU (600x real-time).
- Quantization: Supports various GGUF quantization formats, from f16 to q4_k.
- Integration and convenience: Uses a self-contained GGUF format that includes the tokenizer, and has been integrated into the LocalAI backend to provide an OpenAI-compatible API.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.