Kakao Releases Open-Source 'network-node-manager' to Resolve Kubernetes Network Issues

Kakao's Cloud Platform team has released network-node-manager, a tool that automatically manages network configurations for Kubernetes cluster nodes. Deployed as a DaemonSet, this tool runs with privileges in the Host Network Namespace, watches for changes to Service Objects, and dynamically updates iptables rules on nodes.

Key Issues Resolved

1. Connection Reset Issue During External Cluster Communication In Overlay Network environments, when Pods communicate with external servers, out-of-order response packets can be classified as INVALID due to a Linux conntrack bug. This prevents DNAT from being performed, causing the node to receive the packet and send a Connection Reset (RST), which ultimately leads to a "connection reset by peer" error inside the Pod. network-node-manager bypasses this issue by setting an iptables Filter Rule to DROP INVALID packets at the node. Since TCP retransmission mechanisms compensate for data loss, communication remains unaffected.

2. External-IP Access Issue in IPVS Mode When kube-proxy uses IPVS mode, issues arise with LoadBalancer Services configured with ExternalTrafficPolicy: Local. IPVS rules are set to drop external traffic on nodes where the Service Pod does not exist, preventing Client Pods on those nodes from accessing the External-IP. network-node-manager resolves this by adding a rule to DNAT packets sent by Client Pods to the External-IP into the Cluster-IP, allowing them to reach the Service Pod normally via IPVS Cluster-IP routing.

Deployment and Support Scope

This tool supports both iptables and IPVS kube-proxy modes and can handle IPv6 Services. During deployment, the cluster's Pod Network CIDR (IPv4/IPv6) and kube-proxy mode information must be specified via environment variables. Currently, it only watches Service Objects, with plans to add further network issue resolution features in future updates that are not dependent on CNI plugins.