SilverTorch: Index as Model — A New Retrieval Paradigm for Recommendation Systems

The retrieval stage of traditional recommendation systems had a structure where multiple microservices were complexly intertwined. The user tower that generates user embeddings, the retrieval service that finds items, and the scoring service that assigns scores each operated independently. This approach created performance limitations by causing increased Latency due to data movement between services, inconsistency between model versions, and fragmentation of the development environment.

To solve this problem, SilverTorch presents a new paradigm called Index as Model. By redesigning the entire retrieval system as a single Neural Network, it integrates the existing microservices into modules within the model itself. Now the item index exists as a Tensor inside the model, and when a user opens the app, retrieval, filtering, reranking, and scoring are all completed with just a single model execution.

SilverTorch has demonstrated overwhelming performance improvements compared to the existing approach.

• 23.7x higher Throughput: The number of requests processed per second increased significantly compared to the existing multi-service approach based on the same architecture.
• 20.9x higher cost efficiency: Greatly improved total cost of ownership (TCO) efficiency compared to CPU-based solutions.
• Improved recommendation quality: Practically implemented Neural Reranking and multi-task scoring even within strict latency limits, increasing the accuracy of recommendations.