Querydsl Bulk Data Processing Optimization: Applying Cursor Pagination and Batch Insert/Update

Existing Querydsl paging suffered from Count query bottlenecks and performance degradation due to offset-based queries. To address this, you can omit the Count query by using Slice, or reduce latency by executing Content and Count queries in parallel using coroutines. Test results showed that a task taking 1,500ms when executed sequentially decreased to 1,037ms when executed in parallel.

Read Performance: Cursor-Based Pagination

The offset method incurs cumulative full-scan costs as data grows, leading to linear performance degradation. In contrast, cursor-based pagination utilizes the PK index to maintain consistent performance regardless of data volume. With 1 million records, the offset method incurs 3.3x latency on the last page, whereas the cursor method remains constant at 1.0x.

Write Performance: Batch Insert and Update

JPA's IDENTITY strategy blocks JDBC Batch due to getGeneratedKeys() calls, resulting in individual INSERTs. To resolve this, you should use addBatch from Querydsl-SQL or apply the rewriteBatchedStatements=true option in MySQL. For 10,000 records, addBatch (approx. 0.26s) demonstrated a 96.5% performance improvement compared to saveAll (approx. 7.5s). For Updates, batch processing instead of individual queries via Dirty Checking can yield approximately 87.5~89.3% performance improvement.