whichllm - Finding the Local LLM That Actually Runs and Performs Best on Your Hardware

whichllm is a CLI tool that recommends the local LLM optimized for your hardware based on actual measured benchmarks rather than relying on parameter count.

Key Features and Functions

• Automatic Hardware Detection and Support: Supports NVIDIA, AMD, Apple Silicon, and CPU environments, automatically detecting GPU, CPU, and RAM specs.
• Unified Multi-Benchmark Ranking: Integrates various metrics such as LiveBench, Chatbot Arena ELO, and Open LLM Leaderboard to prioritize the models with the best actual performance.
• Intelligent Model Verification: Features a Recency-aware function that prioritizes the latest models, along with a trust-discount system that tracks model lineage to block false scores.
• Precise Resource and Speed Estimation: Predicts inference speed based on architecture, accounting for VRAM usage (including KV cache and activations) and memory bandwidth.
• One-Stop Workflow: With the whichllm run command, you can go from model download to interactive chat instantly, with support for isolated environments via uv and various quantization formats (GGUF, AWQ, etc.).
• Hardware Simulation: Provides the ability to virtually configure a specific GPU to check performance before purchasing, or reverse-lookup the hardware required to run a specific model.