Show HN: Reame – a CPU inference server that gets faster the more it runs

Reame is a lightweight LLM inference server built on top of llama.cpp, designed for efficient inference even in low-spec CPU and ARM environments.

The key technical features are as follows:

• Persistent shared-prefix KV cache: Saves prompt prefixes as disk snapshots and reuses them, reducing computation costs for repeated system prompts and similar content.
• Palimpsest: Stores generated outputs as an n-gram archive, allowing instant drafting for similar requests without model computation.
• Self-regulating speculative decoding: Predicts tokens using a small draft model or n-gram lookup, self-measuring the effectiveness of this feature based on hardware performance to decide whether to enable it.
• The Conclave: Generates N candidate answers in parallel for the same prompt, then selects the best answer by majority vote to compensate for model variability.
• OpenAI-compatible API: Supports OpenAI API specifications such as /v1/chat/completions, enabling immediate integration with existing clients.

This server is optimized not for general-purpose conversational AI but for repetitive, domain-specific workloads such as document extraction, classification, batch pipelines, and implementing AI features within SaaS products.