AI BriefingReal-time voice booking bot built without a generative LLM, achieving 300–500ms latency using jev
·2026.10.29 11:39
Key point
Introduces a real-time voice booking bot demo that combines the non-generative model jev with code-based logic to address the latency and hallucination issues of generative LLMs.
Details
- Proposes an architecture combining the non-generative discriminative model jev with code-based business logic to resolve the 400–900ms variable latency and hallucination issues of existing 10B-class lightweight LLMs
- Separates roles by assigning STT error correction and date extraction to the preprocessing layer (Kiwi morphological analysis), allowing jev to focus solely on intent determination based on extracted options
- Utilizes ColBERT instead of an LLM for FAQ handling, blocking hallucinations through token-level matching and extracting static answers within tens of milliseconds
- Splits utterance units into segments such as repetitions and query results for asynchronous transmission, maintaining conversation flow through importance-based state control upon user intervention (Barge-in)
- The demo is based on a restaurant reservation scenario, allowing users to verify remaining seat information and complex utterance processing in real-time audio
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Offloading Bounded Decisions to Jev Cuts Agent LLM Calls by 29.6% on SOP-BenchPerplexity releases pplx-decider-v1.1-27b, outperforming Jev on Decision Index