Microsoft Releases AesCode: AI Model for Generating Editable HTML/CSS from Visual References

Microsoft has released AesCode, a new AI model designed to generate structured, editable HTML/CSS code for complex visual artifacts like slides, posters, and dashboards. The model addresses the limitation of standard code models, which lack visual feedback, by using image generators as aesthetic guides while maintaining text and logic accuracy through a prompt-based source of truth.

Core Methodology

AesCode employs a decoupled cross-modal rewards training pipeline that separates semantic requirements from visual cues:

• Design Graph Supervision: Uses a graph structure where nodes represent elements (text, charts, tables) and edges define relationships (alignment, order).
• Attribute-Specific Reward Routing: Combines six deterministic verifiers (execution, text accuracy, boundaries, data integrity, layout, whitespace) with a Visual Graph Rubric that evaluates visual attributes via yes/no questions bound to the design graph.
• Reward Normalization: Utilizes GDPO to normalize reward channels, preventing dense rule-based rewards from overwhelming sparse visual feedback.

Benchmark Performance

Evaluated on 300 infographic samples, AesCode demonstrates significant improvements over leading models, particularly in visual quality:

• AesCode-32B: Achieved an Overall score of 85.89 (Visual: 77.44, Rule: 94.33).
• AesCode-8B: Achieved an Overall score of 82.94 (Visual: 75.80, Rule: 90.07).
• Competitors: GPT-5.5 scored 81.28 and Claude Opus 4.8 scored 80.39 under similar reference conditions.

Key findings indicate that visual knowledge learned during training transfers to policy knowledge. When reference images are removed during inference, AesCode-8B’s visual score drops by only 1.0 point, whereas its backbone model (Qwen3-VL-8B) drops by 19.6 points, demonstrating effective internalization of design principles.

Availability and Requirements

The models are available on Hugging Face under the MIT License for code, with checkpoints following the Qwen3-VL backbone license:

• Models: microsoft/AesCode-8B and microsoft/AesCode-32B.
• Infrastructure: Requires Python 3.10, CUDA 12.6, and 8x NVIDIA B200 GPUs for training.
• Dependencies: Uses Playwright Chromium, PaddleOCR 2.10.0, and a patched version of verl for RL training.
• Data: The repository includes code but not the dataset; training used 3,000 SFT demonstrations and 7,408 RL prompts.