Is GenAI Living Up to the Hype? Insights from 3 Years in the Field

Three years of field experience have made it clear that GenAI is not a cure-all. In enterprise deployments, infrastructure costs ran 40–60% higher than expected, integration timelines stretched 2–3x longer, and data preparation alone accounted for 60% of total project effort in some cases.

AI21 Labs addresses this reality with its hybrid model Jamba. Jamba stands for Joint Attention and Mamba, a structure that combines Mamba's state-space layer with Transformer attention. Each block consists of 7 Mamba layers and 1 attention layer. The company argues that traditional Transformers are memory-intensive and slow on large inputs, making it difficult in practice to exceed 100k tokens. The NVIDIA RULER benchmark, meanwhile, tops out at 128k tokens, but internal testing reportedly handles up to 256k tokens. As an example, they cite that combining 10 years of Google, Meta, Amazon, and Microsoft 10-Ks amounts to roughly 16,000 pages.

The combinations that actually deliver results are distinct:

• RAG: Injecting relevant information improves accuracy, but comes with high demands on data quality, cost, and integration complexity.
• Systematic workflows: Early static chains improved accuracy, but become fragile when made too rigid.
• Prompt engineering: few-shot examples and tailored prompts can boost performance, but model-to-model variance and maintenance burden are significant.
• Long-context processing: Longer context improves accuracy but causes cost and memory usage to spike.

A real-world example given is automated generation of financial term sheets. Jamba Mini is used to extract and verify key points from the input, RAG selects example documents to draft a first version, and a separate model, Claude, critiques the result — a multi-stage validation flow. Because it's open-source-based, on-premise, cloud, and hybrid deployment are all possible, and the conversational chat interface is seen as having significantly lowered the barrier to entry for GenAI.

In conclusion, GenAI still has limitations around full autonomy, error recovery, multi-tool integration, and cost management. However, combining long context with verification-based workflows makes the enterprise value proposition clear. The next stage of agentic workflows is expected to move beyond simple automation toward systems that can detect failures and change course.