Open-source tool inside-llm visualizes token-level divergence between quantized LLMs

·2026.11.16 06:46

Key point

The tool uses llama.cpp to render probability streams, sampler gates, and logit lenses for comparing two GGUF models in a browser.

Open-source tool inside-llm loads one or two GGUF models via llama.cpp to visualize internal LLM operations in a browser.
Displays token probability streams with brightness indicating likelihood and magenta marks highlighting where two models diverge.
Visualizes the full sampler chain (penalties, top-k, top-p, min-p, temperature, GBNF) to show candidate survival at each stage.
Includes a logit lens feature to map predictions per layer as 2D maps or 3D reliefs.
Built as a single C++17 process with a Svelte 5 frontend, supporting CUDA and Vulkan on Linux and Windows.
Requires both models to share the same tokenizer and fit simultaneously in VRAM; runs locally on 127.0.0.1 with no telemetry.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.