Airbnb Scales Its Identity Graph into a Unified Knowledge Graph Infrastructure

Airbnb moved its identity graph, a core foundation for Trust and Safety, to an internally managed knowledge graph infrastructure. It aimed to resolve long-standing latency, operational instability, and control limitations that remained after passing through a relational DB, a KV store, and a third-party SaaS graph DB, by building an internal platform.

The new infrastructure combines JanusGraph, DynamoDB, and OpenSearch. On top of it, a management service providing per-tenant namespaces, schema/index management, and a Thrift API was added, forming a paved-path, multi-tenant graph platform. During migration, since both engines supported Gremlin, comparisons were made using shadow traffic, and the internal engine was put into production first before the vendor solution was decommissioned.

The graph scale is 7 billion nodes and 11 billion edges, with about 5 million new edges added per day. Since most queries require 4-8 hop traversals, P95/P99 long-tail latency at high-fanout nodes remained a key challenge.

Performance improvements were made on both the engine and client sides.

• Transaction optimization: Used DynamoDB's conditional write and transaction API instead of default locking.
• Parallel lookups: Parallelized getMultiSlices to reduce latency for high-fanout queries.
• Observability: Integrated Airbnb's distributed tracing into the internal fork.
• Query rewriting: Removed Path and SimplePath steps and minimized side-effect steps.

As a result, the new internal infrastructure was faster than the previous vendor across all Gremlin query patterns. P95/P99 for read queries dropped by 32-93%, and end-to-end read latency improved from P95 2.1s→1.0s and P99 5.0s→2.5s. Write P95 also improved from 353ms→156ms, and in load tests, write QPS was scaled up to 10x compared to before.

Operationally, the periodic instance reboots required by the vendor solution have been eliminated, and incident response and investigation have become faster and more transparent.