llmfit Released: Pre-estimate Local LLM Speed

The terminal tool llmfit, developed in Rust, has released a feature that estimates hardware compatibility and expected generation speed (tok/s) before running local LLM models. The tool automatically detects CPU, RAM, GPU/VRAM, and acceleration backends to analyze hundreds of models from its built-in catalog.

Key Features and Estimation Principles

To prevent trial and error due to out-of-memory or insufficient speed before model downloads, llmfit scores models based on four criteria: quality, speed, compatibility, and context. Speed estimation is based on VRAM bandwidth, and when the GPU model name is identified, it references an actual bandwidth table (approximately 80 types including NVIDIA, AMD, Apple Silicon, etc.). For MoE models, calculations are based on active parameters, applying a default efficiency coefficient of 0.55.

Confidence Levels and Integration

Estimation results are displayed with confidence levels such as measured_local, measured_community, calibrated, and estimated. It automatically detects major local runtimes including Ollama, llama.cpp, MLX, and LM Studio, and allows integration with cluster schedulers or dashboards by running a REST API server via the llmfit serve command. On Windows, signed binaries are provided, but there are detection limitations in some AMD/Intel GPU environments.