AI Models Are Taking IQ Tests. Nobody Checked Their Cheat Sheets

Benchmark fans have a new event: an IQ contest for AI models. TrackingAI ranks contestants by the average of their last seven runs on Mensa Norway’s public test. Grok 4.6 High (Vision), Claude-5 Opus MAX, Bing Copilot and Mistral Large 3 appear near the top. It looks like a proper leaderboard, until you ask what the contestants were given.

Set aside whether an IQ test measures intelligence at all. Mensa Norway’s scale is expressed relative to people, with 100 as the population average. TrackingAI applies those scores to machines. More immediately, its methodology says Vision models receive puzzle images, while other models receive verbal descriptions. One contestant sees the puzzle; another is handed a description. Both get a place on the same chart.

One contestant sees the puzzle; another is handed a description. Both get a place on the same chart.

Then there is the question paper itself. Mensa Norway is a public online test. Its puzzles and answers could have appeared in a model’s training data. A model might reason through a problem, recognize it, or do some of both. The score alone cannot distinguish those possibilities, and public availability is not evidence that any particular model saw the questions during training.

TrackingAI also offers a separate Offline Test. The site’s authors say its questions have never appeared on the public internet or in AI training data; that is their claim, not an independently established fact. A published comparison lists Gemini 3.1 Pro at 145 on the public test and 121 on the Offline Test. That gap is worth noticing, but the tests are different. It does not prove that Gemini—or anyone else—copied an answer.

Watching models tackle puzzles is interesting enough without declaring an intelligence Olympics. Here, the entrants do not all see the same kind of exam paper, and we cannot establish which, if any, knew the public questions beforehand. If you’re going to copy, at least change a few things.