IQ Metrics
IIF Certified Assessment Start IQ Test
IIF Certified Assessment

Was the GPT-4 Bar Exam Score Really in the 90th Percentile?

OpenAI reported that GPT-4 scored 298 of 400 on a simulated bar exam, about the 90th percentile. A 2024 re-analysis put the same score near the 62nd percentile of first-time takers and the 45th among licensed lawyers. The score held; the comparison group changed.

Was the GPT-4 Bar Exam Score Really in the 90th Percentile?
Illustration generated for IQ Metrics. No photograph is used. IQ Metrics

What you need to know

  • The GPT-4 bar exam score of 298 out of 400 was reported by OpenAI in 2023 as about the 90th percentile. A 2024 re-analysis in Artificial Intelligence and Law puts it near the 62nd percentile of first-time test takers.
  • The 90th figure is close to what a February chart gives, where most test takers are repeat takers who failed in July. The same 298 is about the 68th percentile in a July chart and about the 45th among qualified attorneys.
  • On the essays the percentiles are lower still: about the 42nd among first-time takers and the 15th among attorneys. The re-analysis replicated the multiple-choice score (75.6% against 75.7%) but questions how the essays were graded.
  • A bar exam percentile is not an IQ. Converted with the normal curve, the four percentiles span about 98 to 119, a 21-point spread caused only by the choice of comparison group.

No: the GPT-4 bar exam score of 298 out of 400, which OpenAI reported in 2023 as the 90th percentile, is about the 62nd percentile of first-time test takers. A 2024 re-analysis by Martinez, in Artificial Intelligence and Law, found that the 90th figure came from a pool full of repeat takers. Against licensed lawyers, the same score is about the 45th percentile.

Below: what OpenAI reported, why one score can have four different percentiles, what the essay grading changes, and what a bar exam result can and cannot say about intelligence.

What did GPT-4 score on the bar exam?

OpenAI's GPT-4 Technical Report (March 2023) lists 298 out of 400 on the Uniform Bar Exam (UBE) and marks it as about the 90th percentile. It lists GPT-3.5 at 213 out of 400, about the 10th percentile. The report says GPT-4's score "falls in the top 10% of test takers". It also says the exam was run by its collaborators at Casetext and Stanford CodeX.

The UBE has a multiple-choice part, the Multistate Bar Examination (MBE), and two written parts, the Multistate Essay Examination (MEE) and the Multistate Performance Test (MPT). The written parts are scaled to match the MBE, and the two are added to give the total out of 400.

Why was the 90th percentile disputed?

Martinez found no documented method behind it. The OpenAI report explains how it worked out most of its percentiles, but not the UBE one. The sources it lists for "official score distributions" contain nothing about the UBE. The National Conference of Bar Examiners (NCBE), which writes the exam, does not publish UBE percentiles, so every chart online is an estimate.

The 90th figure is close to what a chart from the February 2019 Illinois bar exam gives: it puts a score of 300 at the 90th percentile. Martinez points out that most February test takers are repeat takers who failed in July, and repeat takers score much lower. A weaker pool makes any score look stronger.

What percentile is GPT-4's bar exam score against first-time takers and lawyers?

The GPT-4 bar exam percentile against first-time takers is about the 62nd overall, the 79th on the multiple-choice part and the 42nd on the essays. Table 1 of the paper gives three groups for GPT-4's 298:

  • July chart (includes repeat takers): 68th percentile overall, 86th on multiple choice, 48th on the essays
  • All first-time takers: 62nd overall, 79th on multiple choice, 42nd on the essays
  • Qualified attorneys (scores of 270 or more): 45th overall, 69th on multiple choice, 15th on the essays

The paper's abstract rounds the attorney figure to about the 48th percentile overall, but its own Table 1 prints the 45th for a score of 298. The 48th is the figure for a score of 300 in Table 4.

How were these worked out? Martinez started from the NCBE's published mean for first-time takers, 143.8 on the multiple-choice part. He assumed scores follow a normal curve and took the spread from public score data. So the first-time and attorney percentiles are estimates built on stated assumptions, not official figures.

Why does the comparison group move a percentile?

A percentile only says what share of a chosen group scored lower. Change the group and the same score lands somewhere else. Table 4 of the paper shows it for three scores:

  • A score of 300: 90th percentile in the February chart, 70th in the July chart, 64th among first-time takers, 48th among attorneys
  • A score of 280: 73rd, 48th, 41st and 15th
  • A score of 270, the most common UBE pass mark: 58th, 37th, 31st and 0th, because every attorney in that group scored at least 270

Table 2 of the paper turns any UBE score into a first-time percentile. These are its rows for scores near GPT-4's:

  • 330: 89th percentile among first-time takers
  • 310: 74th
  • 300: 64th
  • 290: 53rd
  • 280: 41st
  • 270: 31st

The same bar exam score was the 90th percentile against one group and the 45th against another.

Any percentile can be turned into an IQ-scale number with the normal curve, where the mean is 100 and the standard deviation (SD, the usual spread of scores) is 15. The 90th percentile is about 119, the 68th about 107, the 62nd about 105 and the 45th about 98. Our IQ percentile calculator does the same conversion.

That is our arithmetic, not a measurement. Law graduates are not the general population, so none of these numbers is GPT-4's IQ. It does show how far one choice of group can move a headline: about 21 points here.

Was the essay score reliable?

Martinez replicated the multiple-choice score but doubts the essay score. His replication averaged 75.6% correct (95% interval 74.7 to 76.4), against 75.7% in the original study, so there was no significant difference (p = .883). A different prompt changed it more: a minimal prompt gave 70.9% and a tailored one with worked examples gave 79.5%.

In scaled terms, the minimal prompt is a multiple-choice score of about 150, which is the 70th percentile in the July chart and the 64th among first-time takers. The tailored prompt is about 164, which is the 95th and the 87th. So the prompt moved the multiple-choice percentile by more than 20 points, as the choice of group did.

The essays are the weak spot. He lists three ways the grading differed from the official bar exam protocol: no formal rubric, graders without bar exam grading training, and no blind marking. He concludes that any essay score or percentile is not well justified. That matters because the essays carry the lowest percentiles in every group above.

Can AI pass the bar exam?

Can AI pass the bar exam? On the simulated exam, yes on the paper's numbers: 298 is 28 points above 270, the most common UBE pass mark. But the total includes essay marks the re-analysis questions.

Did AI pass the bar exam for real? No. A simulated exam is not a real sitting, and no jurisdiction graded GPT-4's answers.

The same 2023 paper that ran the ChatGPT bar exam trial, by Katz and colleagues, said ChatGPT would score below the 10th percentile while GPT-4 approached the 90th, as Martinez quotes it. In the re-analysis the jump from GPT-3.5 to GPT-4 is still large: from about the 2nd to about the 62nd percentile among first-time takers (Table 3). What changes is where GPT-4 lands, not whether it improved.

Does passing the bar exam show intelligence?

No single exam score measures intelligence in a machine. A bar exam tests legal knowledge and writing under time limits. As our guide to why AI benchmark scores disagree explains, the result depends on the prompt, the grader and the group. The same caution applies to the MMLU benchmark and to our overview of how smart AI is.

For people, the picture is different but related. Our guide to the average IQ of lawyers and our LSAT percentile guide show how the same percentile question arises for human test takers. For a wider comparison, see whether AI is smarter than humans.

Method, 14 November 2026: we read Table 1 and Appendix A of the OpenAI report, and the full text and four tables of Martinez (2024), published online on 30 March 2024 and in print in September 2025. We did not read the Katz et al. (2023) paper or the NCBE data behind the estimates, so those figures come from Martinez. An earlier working-paper version gave about the 63rd percentile; the journal version says about the 62nd, and we use that. The IQ-scale numbers are our arithmetic.
Your own number

Where would your own score land?

A percentile only means something against the group it came from. To see where your own reasoning score sits on the standard scale, take the IQ Metrics test.

Find your IQ score now! →
Secure & encryptedInstant results10–20 minutes

To compare your reasoning with other adults, take the IQ Metrics test.

Common questions

Did GPT-4 score in the 90th percentile on the bar exam?

OpenAI reported about the 90th percentile for a score of 298 out of 400. A 2024 re-analysis in Artificial Intelligence and Law put the same score near the 62nd percentile of first-time test takers, the 68th in a July chart and about the 45th among qualified attorneys (Martinez, Table 1).

What was GPT-4's bar exam score?

OpenAI's March 2023 report lists 298 out of 400 on a simulated Uniform Bar Exam, against 213 out of 400 for GPT-3.5. Martinez replicated the multiple-choice score (75.6% against 75.7%) but questions how the essays were graded.

Can AI pass the bar exam?

On the simulated exam, GPT-4's reported 298 is above 270, the most common UBE pass mark. But the total includes essay marks the re-analysis questions, and no real jurisdiction graded the answers, so it is not a licence to practise.

What percentile is GPT-4's bar exam score against first-time takers?

About the 62nd percentile overall, the 79th on multiple choice and the 42nd on the essays. These are estimates built on the NCBE's published first-time mean of 143.8 and a normal-curve assumption, not official figures.

Sources for this story

  1. OpenAI, GPT-4 Technical Report (arXiv 2303.08774, March 2023): Table 1 and Appendix A; read 14 November 2026 — OpenAI
  2. Martinez E. Re-evaluating GPT-4's bar exam performance. Artificial Intelligence and Law 33(3):581 (2025; online 30 March 2024), doi 10.1007/s10506-024-09396-9; full text read — Springer Nature
  3. Martinez (2024), Tables 1 to 4: percentile estimates for July test takers, first-time takers and qualified attorneys; read — Springer Nature
  4. IQ Metrics computation, 14 November 2026: the four percentiles converted to the IQ scale with the normal curve (mean 100, SD 15) — IQ Metrics

Corrections: spotted an error? Email corrections@iqmetrics.org and we will update this story and note the change here.

Share this story

Know someone who keeps seeing this number quoted without the scale it was measured on? Send it to them — it takes one tap.

Filed under#ai benchmarks#percentiles and norms#standardized exams#study quality

Read the research.
Then find your own number.

Our IIF-certified assessment reports your score with its scale, percentile and confidence range — and a breakdown of the cognitive domains behind it.

Start IQ Test →
Secure & encryptedInstant results10–20 minutes