Pinterest Rebuilds Guardian Abuse Platform with StarRocks and FlinkSQL for Scalability and Compliance

·2026.11.17 01:01

Key point

The new modular architecture uses StarRocks for real-time analytics and FlinkSQL with Iceberg for action processing and replay, improving reliability and governance.

1 / 3

Details

  • Pinterest rebuilt its real-time abuse platform, Guardian, into a modular V2 architecture to address scalability, reliability, and compliance limits of the previous monolithic system.
  • Stream splitting and pruning in the data plane reduced the effective column scale of indexed datasets from O(20K) to O(K), lowering storage and processing inefficiencies.
  • Real-time analytics were rebuilt on StarRocks, utilizing Bloom filter indexes to achieve sub-second latency for high-selectivity feature discovery queries.
  • Asynchronous action processing and historical replay were unified under a FlinkSQL execution model, with Iceberg-backed datasets supporting up to 30 days of historical data for backfill and validation.
  • A thin translation layer maps the unified GSQL V2 interface to StarRocks SQL for analytics and FlinkSQL for continuous processing, preserving the fast investigation-to-action workflow for subject matter experts.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.