Concept · Intermediate
Accuracy, bias and testing
Face recognition accuracy is measured, not asserted: NIST's Face Recognition Technology Evaluation has tested hundreds of algorithms since 2017 and publishes false-match and false-non-match rates at fixed thresholds. The best algorithms are highly accurate on good photos; errors rise with image quality, age gaps and, for many algorithms, with demographic group. This entry explains the metrics, the published findings and why none of them transfer directly to a consumer face search engine.
Updated November 6, 2026 · 4 min read · reviewed November 6, 2026
Accuracy in face recognition has precise definitions and a public testing programme behind it. It also has a marketing vocabulary ('99% accurate') that is unrelated to the definitions. This entry sticks to the former.
What is being measured?
Two error types exist. A false match (false positive) is when two different people are scored as the same person; a false non-match is when two photos of one person are scored as different. Verification (1:1) tests report both at a chosen threshold. Identification (1:N) tests search a probe against a gallery and report the false negative identification rate (the right person is enrolled but not returned at rank one) and the false positive identification rate (a person not enrolled triggers a match). Every number depends on the threshold, the image quality and the gallery size; a figure quoted without those three is incomplete.
Who tests algorithms, and how?
The US National Institute of Standards and Technology (NIST) has run the Face Recognition Vendor Test since 2000 and as a continuous programme since 2017. In July 2023 it split the programme into the Face Recognition Technology Evaluation (FRTE, recognition) and the Face Analysis Technology Evaluation (FATE, attributes, quality, presentation attack detection). Developers submit algorithms; NIST runs them on sequestered datasets it holds (mugshots, visa photos, border crossing images, webcam images) and publishes results per algorithm. The 1:N identification report series (NIST IR 8271 in 2019, with rolling updates, the latest in March 2026) and the demographic effects report NIST IR 8280 (December 2019) are the most cited.
What did the tests find?
- Best algorithms, 2019 0.2 % missed at rank 1 About 0.2% of searches failed to return the correct identity at rank one
- Median algorithm, 2019 2 % missed at rank 1 Roughly an order of magnitude worse than the leaders
- Best algorithms, 2014 (FRVT 2014 for comparison) 4.1 % missed at rank 1 Reported in the same series as the pre-deep-learning baseline
Source: NIST IR 8271, Face Recognition Vendor Test Part 2: Identification (September 2019), and the 2014 baseline it cites; values rounded
The leap between 2014 and 2019 came from deep learning. The 2019 report noted that accuracy gains had been enormous, with the most accurate algorithms failing in about 0.2% of mugshot searches, while the spread between developers remained wide. Later updates through 2026 show continued improvement at the top and continued spread below it.
NIST IR 8280 (December 2019) examined demographic effects across 189 algorithms. Its central finding: for one-to-one verification, false positive rates were higher for West and East African and East Asian faces than for Eastern European faces in many algorithms, by factors between 10 and 100 depending on the algorithm; false positives were higher for women than men, and for the elderly and children. Algorithms developed in China showed smaller differentials on East Asian faces, pointing to training data as a cause. The report also found that one-to-many searches against a mugshot gallery produced higher false positive rates for Black women in several algorithms. The effects were algorithm-specific: the most accurate algorithms showed the smallest differentials.
| Factor | Direction | Evidence |
|---|---|---|
| Image quality (resolution, blur, lighting) | Largest | Every NIST report; the border and webcam datasets show far higher error than mugshots |
| Pose | Large | FRTE results on profile and off-angle sets |
| Age gap between photos | Moderate | NIST ageing studies; children change fastest |
| Demographic group | Small to large, by algorithm | NIST IR 8280 (2019); IR 8429 (2022) for 1:1 |
| Gallery size | Grows with N | False positive identification rate rises as the gallery grows |
| Threshold | Sets the trade-off | Operator choice; rarely published by consumer engines |
Why do these results not apply to a consumer face search?
- NIST tests algorithms, not products. A consumer engine's accuracy is the algorithm plus its index, its crawl quality and its thresholds, none of which NIST sees.
- Web photos are far worse than mugshots: compressed, filtered, posed, small. Error rates on NIST's hardest sets are the nearer comparison.
- The gallery is the open web, not a curated set of enrolled people. Most people in a query's results are not the query subject, so the false positive side dominates.
- No consumer engine has submitted to FRTE under its product name, as far as the published participant lists show.
How should a user read an engine's accuracy claim?
Ask three questions: at what threshold, on what images, against what gallery. If the claim does not say, it is not a measurement. FaceCheck.ID publishes a self-run comparison on 16 photo scenarios; PimEyes publishes no figure; most others publish none. A score shown next to a result is a similarity, not an accuracy; see embeddings and similarity.
Questions
Is facial recognition biased?
Many algorithms show higher false positive rates for some demographic groups, documented by NIST IR 8280 in 2019. The size of the effect varies by algorithm, and the most accurate algorithms show the least. Training data is the main cause identified.
What does 99% accurate mean?
Without a threshold, dataset and gallery size, nothing checkable. NIST reports error rates at stated operating points for that reason.
Which algorithm is best?
NIST publishes per-algorithm rankings that change with every update; the leaders in 2026 are a mix of commercial vendors. The ranking is by submitted algorithm, not by consumer product.
Has accuracy stopped improving?
No. Each FRTE update since 2019 has shown further gains at the top; the gap between leaders and the median remains wide.
Sources
- 1.NIST IR 8271, Face Recognition Vendor Test (FRVT) Part 2: Identification (Grother, Ngan, Hanaoka, September 2019) (checked September 11, 2019)
- 2.NIST IR 8280, Face Recognition Vendor Test (FRVT) Part 3: Demographic Effects (Grother, Ngan, Hanaoka, December 2019) (checked December 19, 2019)
- 3.NIST, FRTE 1:N Identification results page (report of 30 March 2026) (checked March 30, 2026)
- 4.NIST IR 8429, Face Recognition Vendor Test Part 8: Summarizing Demographic Differentials (2022) (checked July 20, 2022)
Entry reviewed November 6, 2026. Found an error? Tell us.
Related entries
- Concept Embeddings and similarity What a face embedding is, how networks place the same face close together, how cosine similarity becomes a score, and why a high score is not proof of identity.
- Organisation National Institute of Standards and Technology (NIST) The US standards agency whose FRTE and FATE programmes publish the only continuing independent tests of face recognition accuracy, including demographic effects.
- Term False positive False positive in face recognition defined: a reported match between different people, its relation to false negatives and thresholds, and how rates are measured.
- Term Doppelganger effect The doppelganger effect: why a face search returns look-alikes, how index size and photo quality make it worse, and how to verify a match with non-face evidence.
- Term One-to-many matching (1:N) One-to-many (1:N) face matching defined: a probe face against a gallery of many, how it differs from 1:1 verification, and why errors grow with gallery size.
Cite this entry
FaceSearch Wiki editors. "Accuracy, bias and testing". FaceSearch Wiki, updated November 6, 2026. https://facesearch.wiki/concepts/accuracy-bias-and-testing/