How Accurate Are AI Image Detectors? Why They Misjudge Photos, and How to Read an \"X% AI\" Score

Last reviewed October 19, 2026 · 8 min read · By the PEA team (Lumina Spark)

日本語版 (Japanese version)

Short answer: Less accurate than their headline numbers suggest. AI image detectors make many more errors on images from generators they weren't trained on and on compressed or resized images, which is what most images on social media are. They also flag real photos as AI. The US National Institute of Standards and Technology (NIST) concludes that none of the current techniques is a comprehensive solution on its own. Treat an "X% AI" score as one clue, and check it against the original file, a reverse image search, provenance data (C2PA) and where the image came from.

Four reasons detectors get it wrong

ReasonWhat happensSource
Generators the detector never sawA detector trained on one family of generators often misses images from newer ones, and tends to call them realOjha et al. (CVPR 2023), Corvi et al. (ICASSP 2023), NIST
Compression, resizing, social mediaRe-saving as JPEG or shrinking an image weakens the traces detectors rely on, and errors riseNIST, Corvi et al., TrueFake (2025)
Biased training dataDetectors can learn dataset quirks, such as JPEG settings and image size, instead of real signs of generationGrommelt et al. (2024)
AI images that look like real photosOn AI images that people also find hard to tell apart, almost all of the existing detectors tested judged them realYan et al. (ICLR 2025)

NIST's report on synthetic content (NIST AI 100-4, November 2024) says that when detectors face generators they weren't trained on, they tend to show substantial error rates, particularly after compression and resizing, and that detectors are often tied to, and may only perform well on, specific generators.

Lab accuracy vs. the real world

Detector marketing often quotes an accuracy figure measured on a fixed test set. On images that actually circulated online, performance drops sharply:

  • A NeurIPS 2025 benchmark (AIGIBench) found that detectors with high reported accuracy in controlled settings suffer significant performance drops on real-world data.
  • A benchmark built from deepfakes circulating in 2024 (Deepfake-Eval-2024) reported image models' AUC falling by 45% compared with earlier benchmarks.

So "99% accurate" doesn't mean 99% right on your image. Ask which generators and which kinds of processing the number was measured on. Japan's Ministry of Internal Affairs and Communications describes countermeasures against fakes as a cat-and-mouse game, and says that accurately identifying fake information is considered difficult even with detection technology.

Documented misjudgments

WhenWhat happenedSource
September 2023Testing one AI detector, Bellingcat found it labelled 6 of 20 real, award-winning photos as AI, and judged 7 of 10 compressed AI images realBellingcat
October 2023A free detector labelled a real photo posted by a government as AI-made; an expert found no sign of AI404 Media
June 2024Meta's automatic labels put "Made with AI" on real photos; Meta changed the label to "AI info"TechCrunch, Meta
May 2026X automatically labelled a real product photo of figures from Japanese maker Kaiyodo, shot by its staff photographer, as "AI-generated"ITmedia AI+ (Japanese)
September 2022An image shared as a drone photo of floods in Shizuoka, Japan, was made with an image generator, as its poster admitted (an AI image passing as real)ITmedia NEWS, Japan's Information and Communications White Paper

False positives can do real harm. NIST notes that judging human-made content as AI-generated can be extremely damaging.

How to read an "87% AI" score

An "87% AI" score doesn't mean there's an 87% chance the image is AI-made. What it means depends on how the detector was trained, where its threshold is set, and how many fakes are actually among the images being checked.

  • NIST notes that accuracy can be heavily skewed by class imbalance, and that a result is hard to interpret outside a particular context of use.
  • The paper introducing the DeepFake Detection Challenge dataset points out that when fakes are rare, the false positives of even an extremely accurate model will outnumber the true positives.

A worked example (illustrative numbers): suppose 1 in 100 images you check is AI-made, and a detector wrongly flags 5% of real photos. Out of 1,000 images, even if it catches all 10 AI images, it also flags about 50 of the 990 real ones. Of the roughly 60 images flagged "AI," only about 1 in 6 is actually AI-made.

Detection vs. provenance (C2PA) and watermarks (SynthID)

  • Provenance (C2PA / Content Credentials): a standard for attaching signed records of capture and editing to an image. The C2PA explainer says provenance complements detection, and that provenance information alone cannot tell you whether content is true, accurate or factual. Major social networks strip metadata on upload, so provenance is often gone once an image spreads.
  • Watermarks (SynthID): on October 7, 2026, Google announced it was expanding SynthID Detector to everyone. It can check images, video and audio made with the AI of Google and its partners. Images from other generators carry no SynthID watermark, so finding none doesn't mean an image is real. See What Is SynthID?

When provenance or a watermark is found, it's a strong clue. When nothing is found, that proves nothing either way. See How to Check If a Photo Is AI-Generated Before Using It as Evidence.

How to use a detector's result

  1. Don't decide on one number. A detector result is one clue. Don't accuse, discipline or reject someone on an "AI" score alone.
  2. Check the original file. Images saved from social media and screenshots have lost many traces to compression and resizing.
  3. Run a reverse image search. You may find the same image, or an old, unrelated photo.
  4. Look for provenance and camera data. C2PA records and shooting data, when present, are strong clues.
  5. Ask where it came from. Who took it, when and where often matters more than any detector score.
  6. Get expert help when it matters. For court, insurance or employment decisions, talk to a lawyer or a forensic examiner.

How PEA approaches this

Because of these limits, PEA doesn't give a single "AI %" score. It runs eight independent analyses, lists confirmed facts and concerns separately, and grades the image from 1 to 10. When there isn't enough to go on, such as a still from a video or a photo of a screen, it says "Not assessed" instead of guessing. PEA's results are one clue too, and it can miss things or be wrong. The first check is free and in English, with no account: try a free check.

Frequently asked questions

How accurate are AI image detectors?

It depends heavily on the image. NIST reports that detectors tend to show substantial error rates on images from generators they weren't trained on, particularly after compression and resizing, and benchmarks on real-world images show large drops compared with lab results. Ask what images any quoted accuracy figure was measured on.

Can an AI image detector flag a real photo as AI?

Yes. In Bellingcat's 2023 test, one detector labelled 6 of 20 real, award-winning photos as AI. Meta's automatic labels put "Made with AI" on real photos in 2024, and in 2026 X labelled a real product photo by Japanese maker Kaiyodo as AI-generated. NIST notes that such false positives can be extremely damaging.

What does "87% AI" mean on a detector?

Not that the image is 87% likely to be AI-made. The meaning depends on the detector's training and threshold and on how many fakes are among the images being checked. When fakes are rare, even a very accurate detector can produce more false alarms than real detections. Check the original file, run a reverse image search, and look for provenance data.

Which AI image detector is the most accurate?

There's no detector that is reliable on every image. Results depend on which generators made the image and how it was processed after, and NIST concludes that no single technique is a comprehensive solution. Prefer tools that show their reasons and say when they can't judge, and combine any result with other checks.

Does SynthID or C2PA tell me for sure whether an image is AI?

Only partly. If a SynthID watermark or C2PA record is found, it's a strong clue. But SynthID Detector only checks content made with the AI of Google and its partners, and C2PA records are often stripped by social networks. Finding nothing doesn't show an image is real.

Sources