GLM-5.3-Flash Builds Inference Infrastructure on Cluster of Over 100,000 Chinese Accelerators, Achieving ~3x Performance Boost

Self-Built Infrastructure Achievements of GLM-5.3-Flash

The Infra Agent based on GLM-5.3 built production inference services for GLM-5.3-Flash on a cluster of over 100,000 Chinese AI accelerators. By applying Intra-node tensor parallelism, ReplaySSM, W8A8 quantization, INT8/FP8/BF16 mixed-precision cache quantization, Layer Split, and the Encode-Prefill-Decode (EPD) distributed architecture in a limited memory environment, end-to-end serving performance was improved by approximately 3x. Hardware efficiency and cost per token reached levels comparable to mainstream NVIDIA GPUs, and the process from initial model adaptation to production readiness took less than two weeks.

Learning and Applying Optimization Techniques

The Infra Agent learned optimization techniques from existing kernels to extract 'Optimization Skeletons' and, based on these, re-evaluated tiling, memory access, and resource allocation strategies for new kernels. For example, in the KDA Decode kernel, it achieved a 1.71x speedup over v2 by eliminating redundant calculations and merging into a single thread block, partially sacrificing parallelism. These verified changes and application conditions were reflected back into the skeleton library, contributing to reduced effort in the next optimization cycle.

Division of Roles Between Humans and Agents

Engineers maintain responsibility for setting optimization goals and system boundaries, and reviewing critical changes related to numerical semantics, concurrency behavior, and production risks. The Infra Agent performs analysis, hypothesis generation, code changes, and experiments. While this structure does not reach full Recursive Self-Improvement (RSI), it demonstrates that system performance is continuously strengthened through an agent engineering loop where the model receives feedback and modifies its actions.