Humanity's Last Exam Scores: What the Leaderboards Show

Humanity's Last Exam is a 2,500-question academic benchmark for AI models, and as of 6 October 2026 its published top scores run from 54.8% to 61.4%, depending on which leaderboard you read. That gap is not a contradiction. The boards use different question sets, different judges and different tool settings, and the benchmark's own answer key has been challenged by outside experts. This guide sets out what the exam is, how its questions were chosen, what the three public boards report, how reliable the answers are, and why a high score does not translate into an IQ.

What is Humanity's Last Exam?

The Center for AI Safety (CAIS) and Scale AI assembled HLE, and the arXiv paper lists "Long Phan and 1154 other authors". The Nature version, "A benchmark of expert-level academic questions to assess AI capabilities", appeared online on 28 January 2026 (Nature 649, pages 1139 to 1146). The question set was finalised at 2,500 questions on 3 April 2025, after the first version of the paper had described a set of 3,000. Alexandr Wang of Scale AI is among the listed authors; you can see how we handle his own numbers on his IQ profile.

The questions come from nearly 1,000 subject experts, mostly professors, researchers and people with graduate degrees, affiliated with over 500 institutions in 50 countries. The latest paper version says about 14% of questions need both text and an image, and 24% are multiple choice; the rest are exact-match short answers. Contributors competed for a $500,000 prize pool: $5,000 for each of the top 50 questions and $500 for each of the next 500. The public examples show the flavour. One asks for a translation of a Palmyrene inscription from a transliteration; another asks how many paired tendons a hummingbird's tail sesamoid bone supports.

How were the questions chosen?

Difficulty was enforced by the models themselves. The paper's rule is that exact-match questions "must stump all models", while multiple-choice questions must stump all but one model, to allow for lucky guesses. The test models were GPT-4o, Gemini 1.5 Pro, Claude 3.5 Sonnet and o1, with two text-only models added for questions without images. Contributors logged over 70,000 attempts, and about 13,000 questions that beat the models went to human review.

Reviewers with graduate degrees scored each question in two rounds, which left 2,700 public questions. Community feedback and a search audit then trimmed that to 2,500. Scale's page describes the audit: a question counted as potentially searchable if a model with search tools answered it correctly but missed it without search, and each such question was checked by hand before removal.

The design has a side effect that Artificial Analysis states plainly. Because the questions were chosen against particular models, its methodology page discourages comparing those models with ones that were not used in curation, since the dataset is "potentially biased against the models used in the curation process". A benchmark that is hard by construction for last generation's models is not automatically hard for this generation's, and not automatically fair between them.

What are the highest Humanity's Last Exam scores?

We read three public boards on 6 October 2026. Their top entries, copied exactly, are:

• Scale AI leaderboard (page carries a note dated 17 September 2026): GPT 6 Astra 54.80 plus or minus 1.94, Fable 5.1 (xhigh) 46.50 plus or minus 2.00, gemini-3.1-pro-preview (thinking high) 46.44 plus or minus 1.96
• Artificial Analysis, 2,158 text-only questions: Claude Opus 5.5 (Max) 61.4%, Claude Fable 5.1 (Max) 59.1%, Claude Fable 5.1 (Xhigh) 58.7%
• HLE-Diamond release notes (22 September 2026), 1,000 questions, without tools: GPT-6 Astra 59.9%, Claude Opus 5.5 54.6%, GPT-6.1 Sol 53.2%, Claude Fable 5.1 50.7%
• HLE-Diamond with web and code tools: GPT-6 Astra 82.9%, Claude Opus 5.5 73.9%, Claude Fable 5.1 72.3%

Three things explain why the leaders differ. First, the question sets: the full set has 2,500 questions, Artificial Analysis drops the image questions to leave 2,158 "for comparability across models", and Diamond is a 1,000-question refinement. Second, tools: the Diamond page reports the same models without tools and with web search and code, and the with-tools column is 17 to 23 points higher for the three models listed. Third, the judge: CAIS and Scale grade with o3-mini, while Artificial Analysis says it uses an equality checker running on GPT-5.6 Luna (medium). A single sentence such as "the best model scores 55%" is only true for one named board on one date.

Scale ranks by statistical overlap rather than raw score. Its method counts how many models are significantly better than a given model, using 95% confidence intervals, so Fable 5.1 (xhigh), gemini-3.1-pro-preview, Gemini 3.8 Flash and gpt-5.4-pro-2026-03-05 all share rank 2 even though their scores run from 46.50 down to 44.32. The same logic applies when you read any two scores a point or two apart.

HLE-Diamond also splits its 1,000 questions into 500 reasoning and 500 knowledge items, and the models do much better on the first half. GPT-6 Astra scores 74.2% on the reasoning partition and 45.6% on the knowledge partition; Claude Opus 5.5 scores 62.4% and 46.8%. If the exam is read as a test of recalling obscure expert facts, the leading systems are still under 50%. If it is read as a test of working through hard problems, they are well above.

How fast have scores risen?

The first version of the paper, dated 24 January 2025, reported GPT-4o at 3.3%, Claude 3.5 Sonnet at 4.3%, Gemini 1.5 Pro at 5.0%, o1 at 9.1% and DeepSeek-R1 at 9.4% (on its text-only subset), with RMS calibration errors between 81.8% and 93.9%. On the finalised 2,500-question set the authors re-measured GPT-4o at 2.7% and o1 at 8.0%. They wrote that it was "plausible that models could exceed 50% accuracy on HLE by the end of 2025".

By the October 2026 readings, one model on Scale's board and at least five on Artificial Analysis's are above 50%. From 9.4% to 54.8% is about 45 points in roughly 20 months, or about 27 points a year, though that is our arithmetic across two different measurements and should be read as a rough pace, not a rate. Calibration has improved too: Scale shows a calibration error of 39 for GPT 6 Astra and 20 for Fable 5.1 (xhigh), against the 80s and 90s at launch. Calibration error measures the gap between how confident a model says it is and how often it is right.

How reliable is the answer key?

In July 2025 the research company FutureHouse published an audit of the text-only biology, health and chemistry questions (321 of them). Its expert evaluators examined 150 outputs from a literature-checking agent and concluded that 29 plus or minus 3.7% (95% confidence interval) of the text-only chemistry and biology questions had answers with directly conflicting evidence in peer-reviewed literature. FutureHouse put the cause down to the incentive structure used to build the exam, and released a cleaned subset.

The HLE paper reports its own audit. After removing or revising questions with imprecise formats, a second audit of a new sample of 200 questions gave "a final estimated expert disagreement rate of 15.4%" for the public set. A targeted review of a biology, chemistry and health subset gave approximately 18%. The authors say this is in line with other challenging expert-grade benchmarks. FutureHouse noted that the team's three-expert review found at least one reviewer disagreeing 25% of the time, closer to its own 29%.

Disagreement is not the same as error, and the two audits count different things on different samples, so they cannot be netted against each other. What they do show is that a few points at the top of the leaderboard may not be a clean measure of anything. The team responded with HLE-Rolling in October 2025 and HLE-Diamond in September 2026, which it describes as the product of a year-long process of cleaning and refinement. Scale's own page says Rolling was designed to give researchers a migration path once frontier models start to hit the noise ceiling on the original set. For the broader problem of why leaderboards disagree, see why AI benchmark scores disagree, and for a composite index that includes HLE, the Artificial Analysis Intelligence Index explained.

Is Humanity's Last Exam an IQ test?

No, and the reasons are structural. An IQ test samples reasoning on items that are new to the person, and its scores are anchored to a norm group of people, so a result can be placed on the IQ bell curve. HLE reports the percentage of questions answered correctly. It has no norm group of human test takers, and we found no human-expert baseline in the paper's text. A 55% on HLE is not a percentile among people, so there is nothing to convert; the same limit applies to any attempt to put an AI model on the IQ scale, which we covered in AI scoring on an IQ-style test is not an IQ and in our look at ARC-AGI-3.

The authors make a similar point themselves. Their site says that high accuracy would show expert-level performance on closed-ended, verifiable questions, but would not alone suggest autonomous research capabilities or "artificial general intelligence", and that HLE tests structured academic problems, not open-ended research. Whether AI models share a general factor at all is a separate research question, covered in is there a g factor for AI models. For the popular versions of the question, see our guides to ChatGPT's IQ score and whether AI is smarter than humans, and for the term itself, what AGI means.

What should a reader take from the leaderboards?

• Quote the board, the date and the question set with any HLE score; the full set, the text-only set and Diamond are different tests
• Treat gaps of a few points as noise: Scale groups models by confidence interval, and the answer key itself has an estimated 15% or more expert disagreement
• Separate no-tools from with-tools results; on Diamond the tools added 17 to 23 points for the three models listed
• Remember the selection effect: the questions were chosen because earlier models failed them
• Do not read a percent-correct score as an IQ; HLE has no human norm group

To see where a human score lands, take the IQ Metrics test and use the IQ percentile calculator to read it against everyone else. For the other AI benchmark explainers on this site, start with which AI model the index calls most intelligent and the ARC-AGI-3 score and its two harnesses.

Common questions