Netflix adopts DataJunction to unify metrics and streamline experimentation

Netflix implemented DataJunction to address the fragmentation and complexity of its metric definitions, particularly within its Experimentation Platform. Previously, onboarding a new metric required extensive business knowledge and complex engineering workflows, often taking weeks. This friction hindered data scientists and analytics engineers from quickly testing hypotheses.

Why DataJunction

The team evaluated several solutions, including internal frameworks and Minerva (from Airbnb), but chose DataJunction for its robust SQL parsing and generation engine. Unlike other options, DJ stores all definitions in a connected graph, similar to how relational databases handle metadata. This architecture allows for seamless SQL wrangling and rich metadata management, which was critical for Netflix's immediate use case.

Architecture and Implementation

Built initially as a side project due to resource constraints, the team prioritized an API-first approach, focusing on the core semantic layer rather than a flashy UI. Key architectural decisions included:

• Graph-First Design: Focusing on a solid semantic layer engine that parses SQL input and generates SQL output.
• Cubes as Consumption Protocol: Adopting analytical cubes to encapsulate metrics and dimensions, facilitating easy integration with tools like Druid and Superset.
• Feedback Loops: Linking dashboard metrics back to their definitions in DJ, allowing users to inspect and modify metrics directly from visualizations.

Impact on Experimentation

By centralizing metric definitions, DataJunction simplifies the creation and sharing of metrics across the organization. It provides flexible interfaces (UI, API, client libraries) that allow users to author and manage semantic models without deep tribal knowledge. This unification supports Netflix's broader goal of making analytics accessible across verticals and enabling internal LLMs to provide accountable explanations of business metrics.