The Evolutionary Specialization of AI

The AI inference market is the fastest-growing market while also fragmenting rapidly.

NVIDIA's data center revenue jumped from $3.6 billion in Q4 2022 to $62.3 billion in Q4 2025 since the launch of ChatGPT, growing 17x in three years. Just as databases split into relational, document, key-value, graph, time series, and vector, inference is also fragmenting due to differences in workloads.

The core of this fragmentation is that constraints differ by task.

• Images and video require heavy computation.
• Long context consumes large amounts of KV cache memory.
• Edge devices face strong power and connectivity constraints.

The ecosystem has already split as well. Alongside a small number of long-lived LLMs, Hugging Face hosts over 90,000 text-to-image models, with new variants continuing to increase.

Segments broadly divide into three:

• Real-time (under 100ms): voice assistants, real-time translation, autonomous driving
• Near real-time (100ms-2s): chatbots, code completion, retrieval-augmented generation
• Batch (seconds to hours): document processing, large-scale content generation

In multimodal, bottlenecks also differ. Chatbots need to maintain conversation state, so memory matters, while image and video generation require pure computation, needing as many as 50 sequential passes even for a single image. At the edge, a 3-billion-parameter model for Apple Intelligence runs on-device, Tesla operates a vision model using 72W on its FSD chip, and quantization, dedicated chips, and limited memory become new axes of optimization.

Ultimately, the inference market, growing to roughly $100 billion, is likely to produce multiple winners, much like the database market. Just as with Oracle, MongoDB, Databricks, and Snowflake, AI inference infrastructure could also fragment into specialized operators by use case.