Why Doesn't a Node.js Container Shut Down Cleanly? (Graceful Shutdown Edition)

While operating a batch consumer, an issue surfaced where the job kept running even after a termination signal was sent, and it was eventually cut off by SIGKILL. This wasn't a simple signal-handling problem but a shutdown issue tangled up with the Linux PID 1 protection mechanism and the Node.js event loop together.

The first suspicion was the common explanation that "Node.js is PID 1, so it can't receive signals," but the actual core issue was broader than that. PID 1 can ignore signals when there's no handler, and more importantly, Node.js doesn't take charge of cleaning up child/grandchild processes or propagating signals. To solve this, dumb-init was placed as PID 1 inside the container, handling signal delivery and zombie process reaping.

• Used dumb-init as the entrypoint so termination signals are delivered all the way to the application
• Ran the app directly with something like node dist/main instead of npm start, reducing the chance of interfering with signal delivery
• Delegated the process management that PID 1 should handle to a dedicated init system

The next problem was more subtle. In onModuleDestroy, a 2-minute timeout was set using Promise.race, but even after the hook returned, the batch kept running, and the Pod didn't terminate until about 5 minutes later. The reason was that the losing batchPromise wasn't cancelled once the timeout expired, and internal async work like await sleep(10000) remained in the event loop, so Node.js judged that there was still work to do. In other words, the function finishing didn't mean the process was finished.

Using AbortController to interrupt the loop was also considered but not adopted. It would require inserting abort checks at every await point, and DB transactions or external libraries might not handle cancellation safely, which could actually increase the risk of data inconsistency. So the decision was made to let any batch that had already started run to completion, and to leave what happens after that to Kubernetes policy.

In the end, responsibility was split between the application and the infrastructure. On the application side, a shutdown hook was set up with app.enableShutdownHooks() and onModuleDestroy, waiting up to 120 seconds for the internal batch to complete. On the infrastructure side, terminationGracePeriodSeconds was set to 180 seconds, securing a longer grace period than the app's timeout.

The key wasn't just adding a shutdown hook, but understanding what remains in the event loop at the moment of shutdown. The Node.js event loop, PID 1, and Kubernetes termination policy all operate interlocked with each other, so fixing only one side can easily create a state where things seem "finished" but are actually still alive. In the end, what mattered wasn't over-engineering, but accurately seeing the nature of the problem and appropriately dividing the roles between the application and the infrastructure.