A Retrospective on a Memory Issue Caused by Growing Micrometer Objects

While restructuring the maximum discount price logic with parallel processing based on JDK 21, Spring Boot 3.2.1, and CompletableFuture, an unexpected memory issue surfaced. After migrating the coupon API to an internal service, calls to look up product, member, and promotion information separately increased, and in the process, Micrometer-related objects kept accumulating.

The core symptom was that Meter$Id, Tag, and ImmutableTag remained in the Old generation even after GC and kept increasing. In /actuator/metrics/http.client.requests, the URI appeared normalized into the form {}, but in reality, due to the spring.cloud.loadbalancer.stats.micrometer.enabled: true setting, non-normalized URIs were being collected as tags at the LoadBalancer layer, continuously generating different Meters.

During root-cause verification, other tags such as status_code, host, and error were also checked, but availableTags remained the same across every request, so they were ruled out. Ultimately, the collection of LoadBalancer, Feign, and Resilience4j metrics was selectively disabled in application.yml, and where necessary, Micrometer-related auto-configuration was even excluded in @SpringBootApplication to block object creation.

Afterward, while tracking the memory growth phenomenon, the GC strategy was also adjusted. As Micrometer objects kept piling up, concurrent collection alone under ZGC struggled to handle the Old-generation growth and CPU load, so the team temporarily switched to G1GC. However, even when Mixed GC ran, the Old generation didn't shrink enough, and heap usage gradually kept rising.

When a forced Full GC was performed using jmap -histo:live, the number of objects in the Old generation dropped significantly, and jmap -histo also showed some object counts decreasing over time. Based on this, the team judged the phenomenon to be closer to insufficient collection intensity in G1GC rather than a leak, and ultimately reverted to the original ZGC.

Upon reverting, the settings -XX:+UseZGC, -XX:ZUncommitDelay=60, -XX:ConcGCThreads=4, and -Xms6g -Xmx10g were applied. In subsequent monitoring, Heap usage stayed stable and flat, leading to the conclusion that ZGC is more suitable in environments with high TPS and heavy object creation.

In summary, the following two points were key.

• Spring Boot 3.x + LoadBalancer Micrometer settings excessively increased Meter objects in PathVariable-based calls.
• G1GC could not collect objects of this pattern quickly enough, and ZGC was the more stable operational choice.