ArtificialAnalysis's LLM Intelligence-Cost Graph Misleads with Log Scale and Data Center Pricing

Critics have pointed out that the LLM intelligence vs. cost graph released by ArtificialAnalysis contains visual distortions and incorrect pricing benchmarks, leading to misunderstandings. The graph displays the cheapest model for achieving each intelligence score, but by applying a logarithmic scale to the cost axis, it makes it difficult for viewers to perceive the massive price differences between low-cost and high-cost models.

Additionally, the slight price differences among low-cost models may appear exaggerated. A more significant issue is that the prices for open-source models are based on data center standards. This results in estimates much higher than the actual costs when users run them on local hardware, failing to reflect realistic usage environments.

Most users do not require frontier-level intelligence and can achieve satisfactory performance with Chinese open-source models alone. Therefore, the cost-efficiency assessment presented by the current graph lacks reliability as a basis for actual decision-making.