From Vendor to Vanguard: Lessons from Airbnb's Journey to Own Observability

Airbnb moved observability, previously entrusted to an external vendor, to an in-house platform built on Prometheus, taking direct ownership of the entire process of metric collection, storage, querying, and visualization. This transition was not a simple system swap but a project to reclaim control over data and user experience.

This migration was a large-scale undertaking that had to move 1,000 services, 300 million timeseries, 3,100 dashboards, and 300,000 alerts. At first, it seemed natural to migrate the most complex services first, but in practice this was a strategy that risked burdening the first customers with excessive uncertainty and training overhead.

Instead, the approach that succeeded was starting with simpler, more achievable goals. It proved effective to verify that the new storage engine could handle the load, build a translator to migrate dashboards and alerts, strengthen documentation and training, and onboard a small set of users to the new visualization tool first.

The key turning point lay in migrating the intent of queries. Rather than simply translating existing queries 1:1, for example when there was a calculation like p95, incorrect intermediate aggregations were ignored and the query was rewritten as a standardized histogram query. To do this, since the type couldn't be inferred from the metric name alone, a metadata engine was embedded within the translation layer to maintain metric type mapping based on the internal label otel_metric_type.

On the user experience side, they adopted PromQL, leveraging a query language and ecosystem that was more widely known than before. On top of this, an in-house skill for AI tools was attached to provide semantic information such as metric type and units, which allowed agents to generate more accurate PromQL, significantly reducing the time spent on incident debugging and dashboard authoring.

Finally, the existing alert framework was not left untouched either. A new authoring experience was introduced that treats alerts as a development workflow rather than a configuration file, providing autocomplete, builder-style assistance, backtesting, and diffing—a change that elevated the value of the migration beyond mere cost savings. In conclusion, this project was not about changing a data store, but about redesigning the ownership of observability and the developer experience itself.