Running One Prompt Across 11 AI Models Yields Different Results

Through its partnership with OpenRouter, Netlify has broadened the range of models available via AI Gateway and significantly expanded the models supported by its coding agent, Agent Runners.

Key updates include:

• Expanded Model Diversity: Supports a variety of frontier models, including recently prominent open models such as Kimi K3, GLM 5.2, and DeepSeek V4.
• Model Evaluation via AXIS: Netlify evaluates models using its own open-source tool, AXIS. Performance is verified based on functional accuracy and cost efficiency, rather than just design quality, with a focus on capabilities such as database utilization.
• Performance and Cost Comparison by Model: Provides comparative test results where the same prompt is run across multiple models to build websites, helping users intuitively understand performance differences and credit consumption per model.

The tests validate model implementation capabilities and costs through three scenarios: building a coffee shop website, creating a To-do list app, and developing a recipe recommendation app. This helps developers select the most suitable model for their tasks.