The Reality Behind Alibaba T-Head's AI Stack

An analysis of asllm, the inference engine for Alibaba T-Head's ZW810E PPU cluster, revealed that it uses the open-source project sglang (v0.5.9) almost as-is, while omitting the attribution required under the Apache 2.0 license.

According to the analysis, asllm is a private fork of sglang, with only a few files added for AES encryption applied to the Qwen3 MoE model for enterprise customers.

Additionally, a serious hang bug was discovered that causes the system to freeze under sustained load. This is caused by an O(N) heap walk problem arising from the MambaRadixCache.sanity_check() function running on every scheduler idle tick, which leads to performance degradation and freezing as cache depth increases.

The author stated that they reported the cause and a fix for the bug to Alibaba, but after two weeks passed without a patch, they resolved the issue by modifying the code directly in their own production environment.