Design Creativity Bench Reveals LLM Diversity Collapse in UI Generation

LLM Diversity Collapse in Visual Output

A new analysis highlights a significant issue in current Large Language Models: diversity collapse. When prompted to generate visual assets like an SVG of a pelican riding a bicycle, frontier models consistently produce nearly identical, low-quality outputs. This contrasts sharply with the potential for diverse, high-quality visual interpretations, suggesting a systemic lack of creative variety in generative AI.

Design Creativity Bench Results

To quantify this problem, researchers introduced Design Creativity Bench, a benchmark evaluating AI models on 80 open-ended UI tasks across two distinct product domains. The evaluation measures three dimensions:

• Originality: Distinctness from other models' outputs.
• Creative Range: Adaptability of design decisions to different domains.
• Appropriateness: Satisfaction of functional requirements.

The results demonstrate a substantial gap between AI and human designers. The best-performing model, GPT-6 Astra, achieved a creativity score of 65.5, while the human reference score was 92.3. Other top models included GPT-6.1 Sol (63.7), Grok 4.7 (62.2), and Gemini 3.8 Flash (60.6).

Resources and Methodology

The full leaderboard, methodology, and reproducible evaluation framework are publicly available. The project aims to make design creativity a measurable dimension of frontier model capability, moving beyond single-output quality metrics. The dataset is hosted on Hugging Face, and the code is available on GitHub.