Kurly Developer Overcomes Limitations of JPA saveAll: Improving 1:N Data Performance with JDBC Bulk Insert

Yu Hyeonghwan, a backend developer on Kurly's Partner Service Development Team, shared his experience improving BULK processing performance. A rapid surge in system resources during large-scale data processing is a major cause of failures, and to solve this, he focused on the Write area among Read, Process, and Write.

Limitations of JPA saveAll and the JDBC Alternative

Despite its name, JPA's saveAll does not generate BULK-style queries but instead issues INSERT queries one by one, causing performance degradation. This overhead from JPA's persistence context management is especially significant when loading externally received 1:N structured data into the DB. Accordingly, he switched to using JDBC to execute actual BULK INSERT queries.

Handling Foreign Keys via Auto Increment PK Inference

The biggest challenge with BULK INSERT is that the Primary Key (PK) of the inserted rows is unknown, making it impossible to set foreign keys for related tables. The author proposed a method where, for tables with Auto Increment applied, the first PK value returned via last_insert_id() is used as a basis to sequentially calculate the remaining PKs at the application level.

• How it works: After a BULK INSERT, the first returned PK (e.g., 57) is added to the index to predict the PK of each row (58, 59, ...).
• Application process: ① BULK INSERT into the Member table → ② Retrieve the first PK → ③ Map the foreign key to the Article table using the calculated PKs → ④ BULK INSERT into the Article table.

Performance Improvement Effects and Precautions

According to test results, loading 10,000 Member records and 30,000 Article records took 78 seconds when using JPA, but performance improved significantly when this JDBC approach was applied. However, care must be taken regarding persistence context mismatch issues when mixing JPA and JDBC, and since there is no single correct answer, choosing the right technology for the situation is important.