MongoDB CQRS Performance Improvement: Solving an Unexpected Tomcat NDJSON Bottleneck

Naver Pay's settlement system was using a merchant-based sharded DB, but as data concentrated on top large merchants, query timeouts became frequent. To solve this, the CQRS pattern and MongoDB were introduced, and API query performance improved by more than 40x compared to the existing sharded DB.

However, Excel downloads did not get faster as expected. Neither MongoDB tuning nor switching Excel libraries had any effect, and the cause turned out not to be the DB or the library, but repeated blocking that occurred when Tomcat (Spring Web Embedded) streamed Flux responses as NDJSON.

Breaking down the problem area piece by piece yielded the following results:

• MongoDB query: Adjusting readConcern and applying projection improved query performance by about 10%, but had little impact on the overall Excel generation time.
• Excel library: POI SXSSF, EasyExcel, and FastExcel were compared, and FastExcel showed about a 2x improvement in generation performance. However, the overall bottleneck still remained.
• Network: Transferring the result file for 80,000 records finished in about 1 second, so the network was not the issue.

While testing the remaining area, Spring Web, a large gap was found between the processing speed of a regular JSON array response and an NDJSON response. Based on curl, JSON was at 37.7MB/s and NDJSON at 3.7MB/s. Structurally, this was because each line was serialized and then written & flushed, followed by another write & flush for the newline, causing blocking to repeat for every line.

The core issue was that in NDJSON responses, JsonEmitterSubscriber processes each item of the Flux individually, and Tomcat's ResponseBodyEmitter performs a write & flush every single time. In contrast, regular JSON combines everything into an array and performs write & flush only once, resulting in much less I/O blocking time. Following the same logic, in a Netty environment, speeds of about 19–20MB/s were achieved, roughly 7x faster than Tomcat, but changing the entire service was too costly in terms of scope of change and validation effort.

The solution was to change the streaming structure. Instead of returning Flux<T> as is, an extension function was created to write & flush directly to HttpServletResponse, applying buffering in units of 1000.

• Group items using buffer(1000)
• Write each item as one line of JSON
• Flush only once per batch
• Ensure completion with blockLast() at the end

With this approach, Excel download performance was finally improved. In the end, the bottleneck was not the DB, the library, or the network, but the way Reactive-style NDJSON was handled in a Servlet-based environment, confirming that performance issues can arise from an unexpected framework layer. It also left the lesson that you must measure section by section rather than looking at the whole, in order to find the real bottleneck.