A mysterious new image generation model is beating models from Midjourney, Black Forest Labs, and OpenAI on the crowdsourced Artificial Analysis benchmark.
The model, which goes by the name “red_panda,” is around 40 Elo points ahead of the next-best-ranking model, Black Forest Labs’ Flux1.1 Pro, on Artificial Analysis’ text-to-image leaderboard. Artificial Analysis uses Elo, a ranking system originally developed to calculate the relative skill level of chess players, to compare the performance of the various models it tests.
Similar to the community AI benchmark Chatbot Arena, Artificial Analysis ranks models through crowdsourcing. For image models, Artificial Analysis selects two models at random and feeds them a unique prompt. Then it presents the prompt and resulting images, and users choose which they think better reflects the prompt.
Granted, there’s some bias in this voting process. Artificial Analysis’ voters are AI enthusiasts, for the most part, and their choices might not reflect the preferences of the wider community of generative AI users.