What Makes an IQ Test Reliable? Standardisation, Reliability and Validity

A reputable IQ test has three things: standardisation (a large, recent norm sample and one way of giving the test), reliability (the same person scores about the same each time) and validity (the score measures reasoning and predicts what it should). The main clinical batteries report full-scale reliability of .95 to .98, which means a standard error of about 2 to 3 points: a reported 100 is best read as 95 to 105. A test that cannot show all three, or that prints a single number with no norm sample, is not an IQ test in the sense a psychologist uses. This guide explains each factor, shows the publishers’ own reliability figures and gives a checklist for judging any test.

• Standardisation: a norm sample of thousands, recent, matched to the population, with fixed instructions, timing and scoring.
• Reliability: consistency of scores. The full-scale reliability of the WAIS-IV is .97 to .98, of the WISC-V .96 and of the Stanford-Binet 5 .95 to .98, as reported by the publishers.
• Validity: the score measures what it claims and predicts outcomes: .81 with national exam results in one English study of more than 70,000 children (Deary and colleagues, 2007).
• Error: the standard error of measurement equals the standard deviation times the square root of one minus the reliability, so 15 times the square root of .03 is about 2.6 points at a reliability of .97.
• Limit: the figures describe well-made individual tests; they do not apply to an unnormed online quiz.

What three factors do reputable intelligence tests have?

Standardisation, reliability and validity. They answer different questions, and a test needs all three. A scale that always reads 5 kilograms heavy is reliable but not valid; a scale that reads right on average but jumps about from one weighing to the next is valid on average but not reliable. The table sets the three side by side.

Our guide to how IQ tests work covers the first factor in more detail, and do IQ tests really work covers whether the whole package holds up. For the wider debate see are IQ tests reliable measures of intelligence.

What is reliability in an IQ test?

Reliability is consistency. It is reported as a coefficient from 0 to 1, where 1 would mean a score with no random error at all, and it comes in four main forms.

Reliability falls as a score rests on fewer items. The Stanford-Binet 5 publisher reports .95 to .98 for the three IQs, .90 to .92 for the five factor indexes and .84 to .89 for the ten subtests, and the WPPSI-IV shows the same ordering (see below), which is why a single subtest or index difference should be read with more caution than the total. Short-term retest figures also overstate long-term stability. A meta-analysis of 205 longitudinal studies found a rank-order stability of about .76 for adults at age 20 over a five-year gap (Breit and colleagues, 2024). Our guide to how long an IQ test stays valid covers the timing.

How reliable are the main IQ tests?

The figures below are the publishers’ own, as described in our test guides, not independent studies. They show how close the best tests come to a score with almost no random error, and why even those tests report a margin of a few points.

Turning a reliability into error is simple arithmetic: the standard error of measurement equals the standard deviation times the square root of one minus the reliability. At .97 it is 15 times the square root of .03, about 2.6 points, so a 95% band is roughly 5 points either side. Our explainer on IQ score margin of error walks through it, and how to read an IQ test report shows where a real report prints the interval.

What is validity in an IQ test?

Validity asks whether the score measures what it claims. Three kinds matter. Content validity asks whether the items sample the abilities the test says it covers. Construct validity asks whether the score behaves like a measure of reasoning: the WJ V General Intellectual Ability, for example, correlates .66 to .85 with the general totals of other batteries such as the WAIS-IV and WISC-V in its technical abstract, which is strong agreement, not identity. Predictive validity asks whether the score forecasts outcomes. In more than 70,000 English children, intelligence at age 11 correlated 0.81 with a general achievement factor built from national exam results at 16 (Deary and colleagues, 2007), and a meta-analysis of longitudinal studies found .56 with education, .43 with occupation and .20 with income (Strenze, 2007).

Validity does not mean the score captures everything. It leaves out creativity, motivation and practical judgement, which is why the debate in the pros and cons of IQ testing and is intelligence limited to IQ continues alongside the technical evidence.

What is standardisation, and why do norms matter?

Standardisation has two halves. The first is the norm sample: the test is given to a large group chosen to match the population by age, sex, region, ethnicity and parents’ education, and each person’s raw score is turned into a position within their age band (the WPPSI-IV, for example, was normed on 1,700 children). The second is a fixed procedure: the same instructions, time limits, starting points and scoring rules for every examinee, so that a score from one room means the same as a score from another.

Norms have a shelf life. Raw performance rose for most of the twentieth century: a meta-analysis of 271 samples from 1909 to 2013 estimated gains of 0.28 IQ points a year, about 3 points a decade, and found that the gains have decreased in recent decades (Pietschnig and Voracek, 2015). A test normed twenty years ago would flatter today’s test taker by roughly 5 points as an illustration, which is why publishers re-norm and why the norming date matters; see why IQ norms expire.

What makes an IQ test less reliable?

• Too few items. Subtests and indexes are less reliable than the total, as the Stanford-Binet 5 figures show (.84 to .89 for subtests against .95 to .98 for the total).
• Extreme scores. Very few people in a norm sample score near the ends of the scale, so scores far from 100 carry more uncertainty (see the IQ ceiling effect).
• Non-standard conditions. Noise, interruptions, changed time limits or a poorly trained examiner add error, which is why individual tests are given by trained staff.
• Anxiety and fatigue. Test anxiety is related to lower performance at about r = -.20 (test anxiety and IQ scores).
• Practice. Retaking a cognitive test raises scores by about .26 standard deviations, roughly 4 points (Hausknecht and colleagues, 2007).
• Language and culture. Unfamiliar wording can lower a score without lowering ability (cultural bias in IQ tests).

Our guide to the factors that affect IQ test results ranks these by size.

How can you tell whether an IQ test is reliable?

• It names the test and edition and describes its norm sample, including size and norming year.
• It publishes reliability coefficients and a standard error of measurement, or a confidence interval around your score.
• It was given under standard conditions: a trained examiner for an individual test, or timed and proctored for a group test.
• It reports a score on a stated scale (mean 100 and standard deviation 15, 16 or 24) and a percentile.
• It does not promise to raise your score or to reveal a “true” IQ to the last point.
• For a school or clinical decision it comes from a psychologist, not a website (the official IQ test and professional IQ tests show what that looks like; our guide to good online IQ tests covers the rest).

Quick answers about IQ test reliability

• What makes an IQ test reliable? A recent norm sample, standard conditions, enough items and published reliability coefficients.
• What three factors do reputable intelligence tests have? Standardisation, reliability and validity.
• How reliable are IQ tests? Full-scale reliability of the main clinical tests is .95 to .98, a standard error of about 2 to 3 points.
• What is the difference between reliability and validity? Reliability is consistency; validity is whether the score measures what it claims.
• Are online IQ tests reliable? Not unless they publish a norm sample and reliability data.
• Can a reliable test still be wrong? Yes: a score has a margin of error, and anxiety, illness and practice move it by a few points.

Sources

• Deary, I.J., Strand, S., Smith, P. and Fernandes, C. (2007): Intelligence and educational achievement, Intelligence, 35(1), 13-21.
• Strenze, T. (2007): Intelligence and socioeconomic success: a meta-analytic review of longitudinal research, Intelligence, 35(5), 401-426.
• Breit, M., Scherrer, V., Tucker-Drob, E.M. and Preckel, F. (2024): The stability of cognitive abilities: a meta-analytic review of longitudinal studies, Psychological Bulletin, 150(4), 399-439.
• Pietschnig, J. and Voracek, M. (2015): One century of global IQ gains: a formal meta-analysis of the Flynn effect (1909-2013), Perspectives on Psychological Science, 10(3), 282-306.
• Hausknecht, J.P., Halpert, J.A., Di Paolo, N.T. and Moriarty Gerrard, M.O. (2007): Retesting in selection: a meta-analysis of coaching and practice effects for tests of cognitive ability, Journal of Applied Psychology, 92(2), 373-385.
• Reliability coefficients and standard errors are the publishers’ figures as summarised in our guides to an IQ of 135, the Stanford-Binet, the Woodcock-Johnson and the WPPSI-IV. The standard error formula is standard psychometrics; the worked figure is our arithmetic.