Databricks Metric View Implementation Unifies Metric Definitions and Reduces Agent Costs by Approximately 50%

Resolving Metric Definition Inconsistencies and Performance Issues

The Data Intelligence team implemented Databricks Metric View to address issues of data inconsistency and performance degradation caused by differing metric definitions across dashboards, ad-hoc queries, and AI agents. In the previous environment, agents directly interpreted raw logs, applying different cleansing criteria each time, which frequently resulted in value discrepancies. Metric View is a Unity Catalog-based semantic layer designed to allow all consumers—including SQL, notebooks, BI tools, and agents—to share the same definitions by defining measures and dimensions in YAML.

Architecture Design and Caching Strategy

While maintaining the existing Bronze/Silver/Gold medallion architecture, Metric View was positioned as a Semantic layer using cleansed Silver data as its source. This prevented redundant costs for cleansing tasks such as expanding nested fields or removing duplicates. To improve query performance, the Materialization feature was utilized, with additive metrics storing pre-aggregated results and non-additive metrics storing individual rows to be calculated at query time. For calculations involving averages (avg) or percentages, sums and counts were defined separately or window measures were used to ensure cache accuracy.

Implementation Results and Operational Lessons

As a result of the implementation, the heaviest dashboards saw a 41% reduction in compute usage and a 6.7x reduction in scan volume. Specifically on the AI agent side, restructuring domain knowledge based on Metric View reduced token usage and costs by approximately 50% and decreased the number of turns by 30%. By having agents reference the same definitions as dashboards, numerical discrepancy issues were fundamentally resolved. Operationally, it was confirmed that careful design is required because dimension changes trigger recalculation for all periods, and cache hit rates can be improved by including filter conditions frequently used by consumers in the materialization.