Comparing Fable and Sol from a Taste Perspective (Both Are Bad)

Magic Hour CEO Runbo Li established an identical working environment to evaluate the 'taste' and creativity of Fable 5 and Sol 5.6. Both models were tasked with producing four 15-second videos (three ads and one mini-documentary) using the same creative process, skills, and instructions.

A local MagicHour MCP server was used for video and image generation, with a budget of approximately $8.7 (5,000 credits) per run across a total of eight runs. The models were also granted access to web search and video manipulation capabilities using ffmpeg.

The test involved creating ad videos for brands that had to be invented from scratch, such as a burger restaurant and men's sunscreen. The results were published on GitHub, and the author concluded that both models fell short of expectations ('bad').