AutoBe, Monthly Local LLM Backend Benchmark

AutoBe has released an official benchmark that monthly compares backend generation performance across GLM, Qwen, DeepSeek, and frontier models.

• Instead of the previous uncontrolled measurements, it applies controlled variables and a scoring rubric.
• The function calling harness has effectively narrowed the gap between frontier models and local models.
• gpt-5.4's DB/API design performance was similar to qwen3.5-35b-a3b, and claude-sonnet-4.6's logic performance was on par with qwen3.5-27b.
• A single run of a frontier model spans 200-300 million tokens, costing $1,000-$1,500 per model at GPT-5.5 pricing, so they will be excluded starting next month.
• The AutoBe SDK can drive an end-to-end AI frontend that is visually rough but fully functional, so frontend automation will be added to the benchmark within 2-3 months.
• Results that still require interpretation include gpt-5.4 scoring lower than mini, the slight difference between deepseek-v4-pro and Flash, and Qwen dense 27B outperforming the MoE.