[Case Study] Experiment with Internal Personal Development Environment Images Part 2: Challenges and Solutions in the Development Process

OpenStack sandbox images have the characteristic that the IP address changes with every boot, requiring automatic modification of IPs embedded in multiple locations such as kube-apiserver certificates, kubeconfig, and static pod manifests. To address this, we built an automation process using systemd oneshot as follows:

• Detect current IP and reissue apiserver certificates
• Replace IPs in static pod manifests and kubeconfig
• Replace kube-proxy ConfigMap and restart DaemonSet

The GitOps configuration evolved from starting with helm install to adopting Flux HelmRelease and then ArgoCD. Specifically, when using Flux, we utilized the configMapGenerator in kustomization to separate values files externally, facilitating debugging and validation via helm template.

For security, it is crucial not to leave credentials in plaintext within images and Git repositories. To prevent secrets from being exposed to all users who receive the image, we designed the system to fetch secrets at local boot time rather than build time, using Vault.