Adding Human Sense to AI Models

Intelligence operates Design Arena, a platform that evaluates the quality of AI-generated output based on real users' tastes. Co-founder Grace Li explained that she got the business idea from the problem that AI game engines could make functional games but not fun ones.

On Design Arena, users enter a prompt along with format and style, then compare multiple outputs in an "A vs B" format to rank them. A variety of visual content, including websites and images, can be evaluated, and users focus on getting the best result rather than which model created it.

For enterprise customers, the evaluation data accumulated in this process is the key value. AI companies can use Design Arena to gather large-scale human feedback on media generation models and incorporate real user preferences—hard to capture with automated benchmarks alone—into model improvement.

By leveraging user login data, the platform can also track taste shifts by continent and time zone. For example, web dashboards in Asia tend to favor more maximalist design styles, and this kind of data complements automated evaluation metrics, which can be gamed.

Intelligence raised a $7.9 million seed round led by Index Ventures, with participation from Conviction, A*, Valkyrie, and others. The company says Design Arena is used by 5.3 million people worldwide and currently records annual recurring revenue (ARR) of $60 million.

However, success in the human evaluation data market is not guaranteed. Yupp, which had more than 1.3 million users and counted several frontier models among its customers, raised $33 million but failed to build a sustainable business and shut down. In contrast, LM Arena, which evaluates text responses, raised a $150 million Series A just four months after launching its paid product.