Migrating 220 Jenkins Jobs to Temporal Cuts Costs in Half and Ensures Stability

Migrated 220 jobs to Temporal to resolve recurring outages caused by Jenkins disk space shortages and plugin dependency issues
Reduced batch system operating costs by more than half and structurally prevented missing failure notifications and duplicate executions
Developers code batches in familiar languages and undergo PR reviews, with execution history retained in Aurora PostgreSQL for 30 days
Chose Temporal over Argo Workflows to reduce the learning burden of YAML DSL and directly reused existing batch images

Migration Process and Architecture

Standardized per-service workers in TypeScript, with the DevOps team providing a shared library to minimize developer workload
Included failure notifications, duplicate execution prevention, stall detection, and execution tracking as default features in the library to ensure consistent operations
Operated Jenkins and Temporal in parallel, gradually migrating service-by-service at the job level, with rollback capability configured via Argo CD

Karpenter prepares On-Demand nodes within an average of 30 seconds for batch executions, resolving resource shortage issues
Batch logs are sent to Datadog, enabling integrated log queries based on execution IDs
After migrating the remaining jobs, Jenkins EC2 instances will be decommissioned, and execution history will be migrated to a data analysis repository