AI BriefingLocal LLM Observability Tool 'Torrix' Released
Key point
Torrix, a self-hosted observability tool that lets you monitor requests, responses, latency, and more for local LLMs, has been launched.
Details
Running entirely in the local environment without sending data to external servers, Torrix is an HTTP proxy-based observability tool for users of local LLMs such as Ollama.
Unlike existing cloud-based tools, it preserves privacy while still enabling detailed analysis of model behavior. It can be applied immediately without any code changes by simply changing the base_url in your existing application.
Key Features:
- Detailed logging: Stores model name, token count, latency, and full prompt and response text
- Performance analysis: Provides p50/p95/p99 latency statistics and cost estimates
- Tracing and grouping: Supports session grouping and agent traces for multi-step LLM chains
- Quality evaluation: Automatically scores quality, accuracy, and reasoning depth via LLM Judge
- Data utilization: Filtered data can be exported to CSV for use in fine-tuning
Installation and Usage: It can be easily installed with a single command via a Docker container. The Community Edition is provided free of charge, with limitations such as a 100-run history and 7-day data retention.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Foremerge v0.5.0 Released: Rust-Based Open-Source Tool Detects Intent Conflicts Between Parallel AI Coding Agents