Building a Budget-Friendly APM with AOP and MDC: From Interface History Management to Call Tracing

Interface history is the record of requests and responses that travel between systems, such as API, Message Queue, and Kafka. It helps across operations and development, enabling failure analysis, performance optimization, replay, recovery, and even usage pattern analysis.

The core tracking value was managed with MDC(Mapped Diagnostic Context). Since MDC is a structure that stores meta information per thread, it's good for maintaining identifiers like TRACE_ID, and when used together with Spring Cloud Sleuth, the tracking ID can be carried across multiple MS.

The system operates in three main stages.

• Determine the interface type and meta information, then store the trace ID and span ID in MDC.
• Before and after method calls, accumulate call stack and history information in a ConcurrentHashMap.
• When the request ends, determine whether to save in a finally block, then clean up the Map and MDC to prevent memory leaks.

The implementation combined Template Method Pattern, Decorator Pattern, and Chain of Responsibility Pattern. Providers classify traffic to determine the call type—API, Internal API, External API, Kafka Consumer, Kafka Producer—and decorators divide the work of measuring time, recording call stacks, and saving results.

There was a lot of trial and error since asynchronous processing and virtual threads also had to be considered. Initially, using plain ThreadLocal alone caused tracking to break the moment the thread changed, and InheritableThreadLocal, with both deep copy and shallow copy, caused problems like missing call stacks or memory leaks.

Afterward, context was propagated to asynchronous threads using BeanPostProcessor and TaskDecorator, but a problem remained where clearing a value in a child thread also affected the value in the parent thread. To solve this, frequently changing data was separated into ConcurrentHashMap, while unchanging tracking information was kept in MDC.

For HTTP requests, the trace ID could be maintained using Sleuth and MDC, but Kafka events were not automatically propagated. So a Kafka Interceptor was used, where the producer puts the TRACE_ID into the header and the consumer retrieves it from the header and puts it back into MDC, ensuring consistency.

Finally, using only the same TRACE_ID could cause a DB PK Constraint Error on retransmission within the same MS, so the span ID was included in the composite key to distinguish request segments. In the end, history storage, call tracing, multi-thread safety, and memory management were all combined into a structure close to a single budget-friendly APM.