Pinterest Launches Metrics Board to Automate Metric Creation and Enable Agent-Driven Analytics

Pinterest has deployed Metrics Board, an end-to-end platform designed to automate metric creation, enforce quality, and enable agent-driven analytics across its petabyte-scale data lake. The system addresses previous inefficiencies where bespoke pipelines led to inconsistent monitoring, redundancy, and metadata drift, by establishing a single source of truth for metric definitions.

Architecture and Workflow

Metrics Board operates as a thin semantic layer built on five pillars: inventory, compute, quality, publishing, and serving & discovery. Metric definitions are stored as YAML specs in a Git repository, treated with the same version control and code review processes as production code. Users define metrics via guided forms, and the system automatically generates the corresponding YAML and opens pull requests.

For computation, each metric definition triggers a dedicated Airflow DAG, isolating failures and allowing individual tuning of schedules and resources. The platform compiles SQL into SparkSQL or PrestoSQL and uses sqlglot to parse code and create dependency tasks. This automation reduces the time required to build and manage metrics from days or weeks to 1–2 hours.

Quality and Governance

Trust is enforced through a dual-engine validation system:

• Rule-based checks: Explicit expectations for day-over-day, week-over-week, and custom thresholds.
• ML-based anomaly detection: Models historical patterns to identify deviations that static rules miss.

Metrics carry a certification grade from Unknown to A+, while dashboards are classified into three tiers that dictate retention, query performance, and documentation requirements. This structure ensures that consumers can easily evaluate the reliability of the data they are using.

Agent Enablement and Impact

The platform exposes an MCP server, allowing LLM-based tools to discover metrics, read definitions, and reason about them. This integration supports Pinterest’s Analytics Agent, which answers natural-language questions by referencing certified metric definitions and golden queries.

Adoption metrics indicate significant success:

• Nearly 150 new metrics are created each month.
• Over 98% of metrics created for experimentation now go through Metrics Board.
• The platform saved approximately two engineering-months on a recent high-profile project, translating to thousands of hours saved monthly.