Pinterest engineers fuse Ray inference and metrics to cut model evaluation time by 32%

Details

Pinterest integrated a fused model evaluation pipeline using Ray and Apache Spark, reducing end-to-end wall-clock time by 25–32% compared to unfused sequential jobs.
The system treats metric decomposability as a first principle, defining a common contract for decomposable, holistic, and group-wise metrics to enable batch-level partial computation.
Fusion overlaps metrics calculation with GPU inference via the shared object store, removing the need to write predictions to S3 and re-read them, which eliminates cluster startup and data movement overheads.
Benchmarks on a Pin Ranking Model with up to 3M rows and 300 experiments show fused runtime grows linearly (~41s per experiment) while startup time remains constant, unlike unfused pipelines where the separate metric stage adds ~11s per experiment.
The approach supports specific recommender system metrics including MAE, ROC-AUC (via confusion matrix approximation), and Precision@K (via bucketed segments to ensure group integrity).

Key point