Trustpilot's Real-Time Data Enrichment Architecture Built with Gemma

Trustpilot faced the challenge of processing millions of user reviews in real time while overcoming strict latency and cost constraints. To address this, instead of relying on existing closed models, the company fine-tuned the open-weight model Gemma to build its core business pipeline.

The reasons for choosing an open-weight model are as follows:

• Model independence: The retraining lifecycle can be directly controlled without being tied to third-party vendor updates or API changes.
• Predictable economics: Shifting from a per-token pricing model to a fixed infrastructure cost model enables economical operation of large-scale predictions.
• Strengthened MLOps capabilities: Building its own models internalizes the core technology of review intelligence.
• Architectural continuity: Leveraging subsequent versions of the base model allows performance improvements with minimal engineering cost.

Trustpilot used the Gemini 2.0/2.5 Pro/Flash family as 'Teacher models' to generate high-quality training datasets. Through this, it built specialized models for specific tasks such as topic classification, NER (Named Entity Recognition), and sentiment analysis, based on google/gemma-2-9b.

The system architecture was designed based on Dataflow and the Gemini Enterprise Agent Platform. In particular, to separate business logic from LLM inference, two endpoints were built:

1. Classifier: Based on FastAPI, it handles pre/post-processing, prompt template management, and chaining.
2. LLM: A dedicated agent platform endpoint that serves the Gemma model via vLLM.