AI BriefingThe Foundation of Enterprise AI: Grounding
Key point
AI21 presented grounding as the key challenge for enterprise AI.
Details
In enterprise AI, the key is grounding. Because LLMs only have general knowledge fixed at training time and don't know an organization's internal data and latest context, plausible-but-wrong hallucination remains the biggest risk. Answers need to be tied to external evidence and sources need to be presented, and the most widely used solution is RAG.
RAG retrieves relevant information from trusted sources like CRM, wikis, and DBs and feeds it into the model along with the query. This connects answers to documents, and users can instantly verify the evidence, boosting both auditability and reliability.
Evaluation and operations are trickier. FACTS Grounding from Google DeepMind and Google Research handles up to 32,000 tokens, examining helpfulness and factual accuracy separately. Real-world deployment requires balancing the following:
- precision vs recall: search broadly for relevant information while reducing noise.
- creativity vs abstaining: write good answers, but be able to refuse to answer when there's no grounding.
- latest information vs parametric memory: the latest evidence must win out over the model's outdated memory.
When grounding is done well, adoption and ROI change. Alphabet saw $100 billion wiped off its market cap after Bard's factual error, and a McKinsey survey found that 50% of US employees were concerned about the accuracy of generative AI. On the other hand, LinkedIn reduced its median issue resolution time by 28.6% in about 6 months with a RAG-based customer support system. The effect is significant in tasks that require both accuracy and speed simultaneously, such as internal AI assistants, contract review, and technical manual QA.
AI21 addresses this need with its Jamba family. The latest Jamba 1.7 has improved grounding on the FACTS benchmark, and its 256K-token context window handles larger document sets at once. Its Transformer + SSM hybrid architecture combined with Mamba aims for both accuracy and efficiency, with VPC, on-premises, and air-gapped deployment also in mind.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.