The Display Server Reaches Its Limit, and Our Answer

The 11st Display Service Development Team reworked the architecture to handle growing and bursty traffic on the Display API Server (DPWAS), aiming to use fewer servers while maintaining stability.

The infrastructure team applied a Scale-up + Scale-in strategy, reducing the number of servers from 110 to 70 while upgrading from 8 cores to 16 cores. The development team focused on application-level optimization using a Tune Up + Fix in approach so the system could hold up without bottlenecks even with fewer servers.

The first issue to surface was a recurring CPU Spike every 2 minutes, even on servers with no traffic. Tracing the cause revealed that all 4 MongoDB instances were using the same maxConnectionIdleTime = 1 minute setting, and the MaintenanceTimer was recreating idle connections all at once, causing CPU usage to spike intensively.

This was resolved by staggering the timing of connection recreation.

• Set maxConnectionIdleTime differently for each MongoDB instance: 50 minutes, 55 minutes, 60 minutes, 65 minutes
• Kept minPoolSize = maxPoolSize to prevent connection shortages during burst traffic
• As a result, CPU Spikes on older equipment dropped from 25% to 9%, and on newer equipment stabilized from 8% to 2~3%

The second issue was delayed transactions occurring during burst traffic. Since the Top-Traffic Endpoint involves MongoDB queries, the team secured a sufficient connection pool size, and also increased the server-side DB's maxIncomingConnections from 5K to 10K to balance with the client side.

The team also observed cases where, alongside MongoDB SocketReadTimeout exceptions, some requests were apparently processed as 200 OK, prompting a review of how retry behavior actually affects response quality. For large-scale query sections, the request unit was also adjusted to lower command load.

The third pillar was server resource management. The team controlled the maximum local cache size, and improved cache key sorting and bulk data retrieval logic, reducing both memory and CPU usage together. They also confirmed that the display server uses ZGC, which lowers the risk of heap fragmentation, separating out this factor as a basis for judging memory stability.

Ultimately, the Display API Server achieved higher throughput with fewer servers. MAX TPS rose from 65.9K to 87.9K, and the operational structure was changed to secure both cost efficiency and stability simultaneously in a large-scale traffic environment.