Show HN: Ctrlb-decompose: A Tool That Strips Noise from Logs Before Sending Them to an LLM

ctrlb-decompose has been released, a tool that compresses millions of lines of noisy logs into a small number of actionable patterns using statistics, anomaly detection, and correlation. This tool optimizes log data before passing it to an LLM, reducing token costs while conveying the core information.

Key working principles:

• CLP (Compressed Log Processor) encoding: Converts variable tokens into typed placeholders, consolidating structurally identical logs into a single log type.
• Drain3 clustering: Patterns logs through tree-based similarity clustering, processable in a single streaming pass.
• Variable classification and statistics: Classifies extracted variables into semantic types such as IPv4, UUID, and Duration, and calculates quantile statistics like p50/p99 via DDSketch.
• Anomaly detection and scoring: Detects frequency spikes, error cascades, bimodal distributions, and more, and scores severity based on keywords.

Output formats: Users can output results in ANSI terminal (for humans), Compact Markdown (for LLMs), or JSON format depending on purpose. It can run as a Rust library, CLI, or in a browser environment via WASM.