Strategy for Owning the AI Intelligence Layer: A 4-Step Roadmap to Building In-House AI Based on Open-Weight Models

The core of AI application competition is shifting from UI or workflow to ownership of the Intelligence Layer. Industry leaders such as Satya Nadella and Alex Karp point out that relying on frontier labs for intelligence is akin to paying twice for proprietary knowledge and costs, recommending that enterprises build their own AI capabilities and own the weights.

In the past, training open models tended to fall into an infinite treadmill due to the rapid improvement of frontier models. However, open-weight frontier models like Kimi K3 and GLM 5.2 are evolving faster than expected, allowing companies to start from a baseline close to the frontier. As independent post-training stacks from Mercor, Fireworks, and LangChain mature, enterprises now have an environment where they can access frontier research lab tech stacks to train and infer their own models.

The Need for Intelligence Ownership

There are product slices where rental intelligence (using frontier APIs) becomes a constraint, and owning your own intelligence provides a strategic advantage for the following reasons:

• Cost: In a structure where inference costs increase COGS upon AI product success, owning the model is advantageous for protecting margins.
• Speed: In domains where low latency is critical, such as coding autocomplete or cybersecurity, small distilled custom models can outperform large general-purpose models.
• Proprietary Data: It is essential when you want core data for system improvement, such as feedback, evaluations, and customer interactions, to remain internal.
• Controlling your destiny: Product ownership means controlling the learning loop and the intelligence itself.

4-Step Roadmap to Building In-House AI

To build your own intelligence, you need an offensive, small, de novo team that performs evaluation construction and data formation, rather than simply plugging into the platform team. Harvey presented a case where a team of just 7 people conducted significant research.

1. Evals: Under the principle that "without good benchmarks, you cannot train a model," you must secure a task set (prompts, context, graders) that measures the system's ability to perform tasks. Harvey's Legal Agent Benchmark consists of 24 legal practice areas, over 1,200 agent tasks, and over 75,000 rubrics.
2. Harness and Context Engineering: Build a harness that manages the product logic around the model (routing, retrieval, tools, memory, etc.). A good harness delivers context at the right time so the model maintains performance on out-of-distribution tasks and makes agents inspectable.
3. Post-training: Select appropriate techniques based on improvement goals. Fact gaps are solved with RAG, output format errors with Supervised fine-tuning, product preference issues with Preference tuning, specialized tasks with RL (Reinforcement Learning), and speed/cost issues with Distillation.
4. Online learning: Capture execution trajectories in production environments to continuously improve the system. Failed tasks become new Evals, missing information moves to memory, and incorrect tool responses lead to harness modifications.

Conclusion

Owning your own stack may have a lower initial high floor but raises the potential ceiling. While large research labs build giant brains, the best product companies are cultivating fast, domain-obsessed little geniuses. The ecosystem of owning your own intelligence will evolve in a direction where individuality wins.