Optimizing a Conversational AI Storytelling Platform with CESC on Amazon ElastiCache for Valkey

Viewcomms introduced CESC in Tadak v2, which bundles world metadata and character state together with user input, and built a structure that reuses images from the same context using vector search in Amazon ElastiCache for Valkey. Even the same sentence returns a different image if the world and situation differ, while different expressions reuse the cache if the context is the same.

There were three main challenges.

• High-quality image generation took an average of 3–5 seconds, breaking the flow of conversation.
• Keyword matching alone struggled to sufficiently reflect the worldview and character settings.
• Generating a new image every time repeated a cost of about 50 KRW per image.

Amazon ElastiCache for Valkey has been supported since Valkey 7.2 and is compatible with Redis OSS. Serverless is 33% cheaper and node-based is 20% cheaper than Redis OSS, and Valkey 8.0, which recently began being supported, offers up to 230% throughput improvement and up to 70% latency improvement through its I/O Multi-thread architecture. Vector search and semantic caching are available in Valkey 8.2, handling billions of high-dimensional vectors with microsecond-level latency. It supports up to 99% Recall, both HNSW and FLAT, and combinations of vector, tag, and numeric filters.

Valkey GLIDE is an open-source client that provides multi-language support, enhanced availability, and observability, and handles vector index creation and management, vector storage, similarity search, and hybrid search. Tadak v2 was designed to structure events such as entering a world, encountering a boss, and acquiring an item, so that different users repeatedly create similar visual contexts, and performed hybrid search (pre-filtering + KNN) by connecting Aurora metadata and ElastiCache in the NodeJS backend.

Retrieved candidates are compared against the current context by an LLM verifier, which either selects the final one image or rejects them, and safetyFilterStatus is re-checked to prevent exposure to inappropriate content. When there is no cache hit, Text-First outputs the text first, and Async Generation generates the image to prepare for subsequent requests.

As a result, on a cache hit, response time dropped to under 100ms, and with the cache handling 35% of overall traffic, the generation API cost based on 1 million requests per month fell from 50,000,000 KRW to 32,500,000 KRW. Search cost is included in the fixed ElastiCache cost and is effectively negligible, with monthly savings at around 17,500,000 KRW. As worlds and stories accumulate, reusable context data also builds up together, securing a structure where response speed and consistency continue to improve.