Building an Asynchronous Partnership Integration System with Kotlin Coroutines

After building the partnership data collection system, the integration area was still operating with Spring Batch-based synchronous processing. Since each processing target required sequentially calling 15 or more external APIs, batch server startup cost and long processing time remained bottlenecks.

Initially, the goal was a fully real-time processing structure based on Kafka event-driven + Kotlin Coroutine, but due to issues like duplicate data, speed control, and partner call limits, the approach shifted to a compromise of Batch + Kafka + Coroutine. The batch now only publishes messages for the processing targets to Kafka, while the core bottleneck—external API calls and DTO assembly—was made asynchronous with coroutines in the composite service.

The core comparison is as follows.

• Existing structure: Reader → Processor sequentially calls multiple APIs → Writer
• Improved structure: Event delivery via Kafka → composite service single endpoint call → concurrent internal API calls
• Effect: The batch's role is reduced, and the 15-API bottleneck is distributed via coroutines

As an example, a comparison was made between the Java Thread approach and the Kotlin Coroutine approach under conditions of 6 APIs, each with 1-second response time, and 3 fixed threads. With Java Thread, the 3 threads processed API-1~3 and then processed API-4~6 again, resulting in a total execution time of about 2 seconds, with resources wasted as threads were blocked during I/O.

In contrast, in the Kotlin Coroutine example, runBlocking, withContext(Dispatchers.IO.limitedParallelism(3)), coroutineScope, and async/await were used to process 6 coroutines with 3 threads. While maintaining the same number of threads, because coroutines are suspended during delay and then resumed, more efficient concurrency was achieved under the same resource conditions.

The core of coroutines lies in suspend functions and Continuation. A suspend function is transformed into a state machine at compile time, and the information at the point of suspension is stored in a Continuation object on the Heap. Afterward, the dispatcher calls it again to resume from the suspension point, so instead of holding and blocking a thread, only the execution state is retained while other work can be processed.

The coroutine components used in practice are also summarized below.

• suspend: Defines a function that can be suspended and resumed
• withContext: Switches the execution thread of a block
• Dispatchers.Default: For CPU-bound work
• Dispatchers.IO: For blocking I/O work
• withTimeout: Controls time limits
• coroutineScope: Groups and manages child coroutines

In summary, this transition was a choice that reflected realistic operational constraints rather than full real-time conversion. The batch is maintained but its role is reduced through event delivery, while the actual bottleneck—external API calls—is made asynchronous with Kotlin Coroutine, improving both performance and scalability at the same time.