AI BriefingThe 13-Minute Wait Trap
Key point
The terminal is fast, but a local LLM trap emerged where hooking it up to an app took as long as 13 minutes.
Details
Typing directly into the Ollama terminal was fast, but the moment it was hooked up to an app, response time stretched to as much as 13 minutes.
The core cause lay less in raw model performance than in the hidden system prompt, MCP/tool overhead, context size, and whether the GPU was being used.
- Tools like Claude Code/Codex can carry system prompts of around 20K~21K tokens, which already makes the very first request heavy.
- On laptop setups, when running on an integrated GPU (iGPU) or CPU/shared memory, local LLMs slow down dramatically.
- Commenters advised checking GPU usage with
ollama ps, and either using a smaller model that fits VRAM or trying acceleration paths like ROCm/Vulkan. - There were many accounts of real-world experience, and some gave up on local LLMs altogether and switched back to an API (e.g., DeepSeek).
Ultimately, this case shows that it's not that "local LLMs are slow," but that the app integration method and hardware determine the speed.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Google Includes Premium Colab Benefits in AI Subscription Tiers
Ollaya Launches Local Runtime for Open-Source Decision Models with TypeSafe SDK Compatibility