LLM-integrated C library

llm is a header-only C library that supports OpenAI, Anthropic, Groq, Ollama, Together AI, Mistral, Cohere, Gemini, DeepSeek, OpenRouter, Perplexity, Fireworks, vLLM, and custom.

Anthropic and Cohere use their native request formats, while the rest are unified under the OpenAI-compatible chat completions format.

• It supports streaming, async fire-and-forget, and up to 8 parallel batches.
• It provides tool calling, JSON mode, seed, stop sequence, proxy, SSL verification, and OpenAI org/project ID configuration.
• Automatic retries apply to 429, 5xx, and timeouts, with adjustable backoff and max delay.

Responses include latency_ms, retries, time_to_first_token_ms, stream_chunks, token usage, error codes, and HTTP status. llm_complete is thread-safe, and llm_complete_batch and llm_complete_async manage their own threads.

The accompanying llm.c CLI supports environment variable API keys, file attachments, streaming, stats output, and custom endpoints, and building requires linking libcurl, pthread, and lm.