How Kurly's Inbound System Safely Syncs External Data

As Kurly shifted from a 1P-centered structure toward 3PL expansion, it needed to safely synchronize incoming inbound-schedule information from external partners. Unlike data that was previously generated only internally, the key challenge became receiving and processing different formats from various external channels.

The existing structure bundled saving received data and publishing Kafka messages into a single transaction. However, the following failure points remained.

• Cases where the message was published but the DB commit failed, breaking atomicity
• Cases where the consumer read the message but final creation failed because partner information or product information wasn't ready yet

To solve this problem, they introduced the Outbox pattern. Received data is first saved to an interface table, and messages to be published are not sent immediately but instead stored in the outbox_record table. Afterward, a separate Poller reads the outbox and publishes to Kafka, marking successful records' status to prevent duplicate publishing.

To reduce implementation burden, they used Namastack Outbox for Spring Boot 0.3.0. This library allowed them to delegate a significant portion of outbox table management, polling, retry, and status management, and what they had to prepare directly was roughly the following.

• Manually creating outbox-related tables
• Registering a Clock Bean
• Adding @EnableOutbox to the main application
• Adding @EnableScheduling and @EnableOutbox to the batch application that runs the Poller

The service logic also became simpler. Code that previously sent to Kafka immediately after saving was changed to serialize the message as JSON and save it to the OutboxRecordRepository. Actual publishing is handled by an OutboxRecordProcessor implementation, which reads the message according to eventType and sends it to Kafka.

As configuration examples, retry-processing options such as poll-interval, batch-size, max-retries, exponential backoff, and jittered policy could be finely tuned. The article explains based specifically on 0.3.0, and also notes that 0.4.x is only compatible with Spring Boot 4 or higher.