Use AI content detection tools as a first screen, never as a verdict. Their scores are probabilities, and they miss AI text while flagging human writing, especially from non-native English writers. When a draft gets flagged, check sources, drafts, first-hand detail and edit history, then judge the page on quality, which is what Google ranks.
Key takeaways
- OpenAI pulled its own AI text classifier in July 2023 because it caught only 26% of AI text and mislabeled 9% of human text.
- A 2023 Stanford study found seven detectors flagged 61.22% of essays by non-native English writers as AI-generated.
- Google’s guidance targets low-quality and manipulative content, not the tool used to write it.
- A detector score should open a review, not close one. Evidence of human work beats a percentage.
Why detector scores need a second look
The most telling data point comes from a company that builds the models detectors try to catch. In January 2023, OpenAI released a classifier for spotting AI-written text. On its English challenge set, it correctly identified 26% of AI-written text and wrongly labeled human text as AI-written 9% of the time. On July 20, 2023, OpenAI added a note to the same page saying the classifier was no longer available “due to its low rate of accuracy.”

Think about what 9% means for a content team. If you publish 200 fully human posts a year and run each through a tool with that error rate, roughly 18 would come back flagged. Every one of those is a writer you might wrongly accuse, or a good post you might rewrite for no reason.
Independent research points the same way. A 2023 paper by Debora Weber-Wulff and colleagues, published in the International Journal for Educational Integrity, tested 12 public tools plus Turnitin and PlagiarismCheck. The authors concluded the tools are “neither accurate nor reliable”, lean toward calling text human, and get worse when the text is obfuscated or reworded.
Are AI detectors accurate enough to act on?
Short answer: accurate enough to point you somewhere, not accurate enough to decide anything alone. The honest version of “are AI detectors accurate” depends on three things the score never tells you.
Who wrote the text. In a 2023 Stanford study, Weixin Liang, James Zou and co-authors ran 91 TOEFL essays written by non-native English speakers through seven popular detectors. The detectors misclassified over half of them as AI-generated, with an average false positive rate of 61.22%. All seven detectors agreed that 18 essays (19.78%) were AI-written, and 89 of 91 (97.80%) were flagged by at least one. US eighth-grade essays, written by native speakers, were classified with near-perfect accuracy. The cause is simple: detectors reward varied, less predictable word choice, and writers with a smaller English vocabulary produce more predictable text.
How much of the text was mixed. Turnitin’s chief product officer, Annie Chechitelli, wrote in June 2023 that its document-level false positive rate is under 1% for documents with 20% or more AI writing, while its sentence-level false positive rate is around 4%. She added that 54% of wrongly flagged sentences sat right next to real AI writing. In practice, a human intro or a heavily edited paragraph near an AI-assisted section is where mistakes cluster.
Whether the text was edited. The Liang team found that asking ChatGPT to enrich the word choice of those same TOEFL essays dropped the false positive rate from 61.22% to 11.77%. The flip side is that light rewriting can push AI text past a detector. A score measures surface style, and style is easy to change in either direction.
What Google actually cares about
For SEO teams, the detector question often hides a ranking fear. Google answered it in its February 8, 2023 post, “Google Search’s guidance about AI-generated content”, under a heading that reads “Rewarding high-quality content, however it is produced.” The same post says that using automation, including AI, “to generate content with the primary purpose of manipulating ranking in search results” violates its spam policies.
So the risk is thin, mass-produced pages with no added value. A detector score is a weak proxy for that. A page with original data, a real author and a clear purpose is safe whether or not a tool calls it 80% AI. A page that rewrites competitors with no new information is weak even if a tool calls it 100% human. Our guide on editing AI drafts to meet E-E-A-T standards covers what that quality bar looks like on the page.
Platforms outside search are moving on this too. LinkedIn has moved to limit AI-generated posts in favor of authentic ones, which we covered in LinkedIn’s crackdown on AI content, and model makers are testing provenance signals such as Anthropic’s invisible watermarks in Claude text. Provenance from the source is a different and more promising approach than guessing from style after the fact.
How to use AI content detection tools in your editorial workflow
Treat AI content detection tools like a spell checker with a high error rate. Useful for triage, dangerous as a judge. Here is a workflow that keeps good content and good writers safe.

- Set the policy first. Decide what AI use you allow (outlines, research, first drafts, none) and write it into your contributor guidelines. A flag only matters against a rule. Our piece on AI content ethics for bloggers and SEOs has a starting point for disclosure rules.
- Run the scan. Use one detector consistently so your team learns its quirks. If you want a second opinion, use a second tool and note where they disagree. Disagreement is itself a signal that the score is noise.
- Read the flagged passages, not the headline number. Look at which sentences triggered the score. Generic intros, definitions and list summaries trip detectors in human writing all the time.
- Ask for evidence of process. Google Docs version history, notes, interview recordings, screenshots of tests, or a short call about the topic. A writer who did the work can show it in minutes.
- Judge the page on quality. Does it contain first-hand experience, sourced numbers and something a competitor page lacks? If yes, publish. If no, the problem is quality, whatever the score says.
- Log the decision. Record the score, the evidence and the outcome. Over a quarter, this shows you how often your detector was wrong on your own content.
Decision table: what to do with a detector flag
| Detector result | Evidence of human process | Content quality | Action |
|---|---|---|---|
| High AI score | Strong (drafts, notes, sources) | Original, specific | Publish. Treat as a false positive and log it. |
| High AI score | Strong | Generic in places | Publish after edits to the flagged sections for specificity, not for the score. |
| High AI score | None offered | Generic, no first-hand detail | Return to the writer with specific quality notes. Do not accuse on the score alone. |
| High AI score | None offered | Factual errors or invented sources | Reject. The fabricated facts are the reason, and they are provable. |
| Low AI score | Any | Generic, no added value | Revise or reject. A “human” score does not make thin content useful. |
| Low AI score | Any | Original, specific | Publish. |
| Tools disagree | Any | Any | Ignore the scores and go by evidence and quality. |
Copy-paste checklist for responding to a flag
- [ ] Which sentences were flagged, and are they intros, definitions or summaries?
- [ ] Is the writer a non-native English speaker or writing in a formal, template-like style?
- [ ] Does the draft include first-hand detail (tests, screenshots, client examples, quotes)?
- [ ] Does every statistic link to its original source, and does the source say what the draft claims?
- [ ] Can the writer share version history or research notes?
- [ ] Does the piece break your written AI policy, or only trigger a score?
- [ ] Would the page still be worth publishing if no detector existed?
- [ ] Is the decision and its reason logged?
Where AI detectors get it wrong, and what to check by hand
Most false flags fall into a few patterns. Knowing them saves you from rewriting good work.
Formal or technical writing. Legal summaries, API docs, schema guides and how-to steps use predictable phrasing by design. Low surprise in word choice reads as “AI” to a detector. Check instead whether the steps actually work.
Non-native English writers. The Liang study is the clearest warning here. If your guest contributors write in a second language, a raw detector score is close to useless for them. Check subject knowledge through a quick conversation or a follow-up question about their example.
Short text. Product descriptions, meta descriptions and FAQ answers give a detector very little to work with. Skip scanning them and review them for accuracy.
Mixed drafts. Turnitin’s own data shows wrong flags cluster at the seams between human and AI text. If a writer used AI for an outline and wrote the rest, expect noise around the transitions.
Edited or paraphrased AI text. This is the miss in the other direction. The Weber-Wulff study found obfuscation makes tools much worse at catching AI writing. A clean score is not proof of human authorship.
The manual checks that hold up better than any score are the ones in a standard AI content audit: verify every number at its source, look for claims only someone who did the work could make, check that examples are real and dated, and compare the page against what already ranks. Our evergreen AI-powered content audit checklist turns those into a repeatable review.
Which AI detector tools to keep in your stack
Pick tools for the job they actually do. A few listed on the Gazette:
- AI Natural Write is a rewriting tool for readability that also includes an AI text detector, so you can see a score and the flagged phrasing in one place.
- SynthID Detector reads an image’s Content Credentials and AI metadata to report whether a SynthID watermark is present. It is useful for checking images contributors submit, where provenance data is more reliable than style guessing.
- Humanize Ink rewrites stiff AI-assisted text into more natural language. Use tools like this to fix readability, and make sure the facts and first-hand detail come from a human. Rewording a thin draft to pass a detector leaves it just as thin.
For more options, browse the AI writing tools directory.
Frequently asked questions
Are AI detectors accurate?
Not reliably. OpenAI withdrew its own classifier in 2023 after it caught 26% of AI text and flagged 9% of human text, and a 2023 study by Weber-Wulff and colleagues called the 14 tools it tested “neither accurate nor reliable.” Use scores to decide what to review, not what to reject.
Does Google penalize content that an AI detector flags?
Google’s guidance says nothing about detector scores. Its February 2023 post rewards high-quality content however it is produced and treats AI content made mainly to manipulate rankings as spam. Quality and purpose matter, not the score.
Why did an AI detector flag my human-written article?
Detectors score how predictable the wording is. Formal, technical or template-style writing, short text and writing by non-native English speakers all read as predictable. A 2023 Stanford study found seven detectors flagged 61.22% of non-native English essays as AI-generated.
Should I reject guest posts that fail an AI detector?
Not on the score alone. Ask for version history or research notes, check every statistic against its source and look for first-hand detail. Reject for fabricated facts or thin content, which you can prove, rather than for a percentage you cannot.
Can editing AI text get it past a detector?
Often, yes. Research from 2023 shows rewording and obfuscation reduce detector accuracy sharply. That is why a low score is not proof of human authorship and why a quality review matters more than the scan.
Sources
- OpenAI, “New AI classifier for indicating AI-written text” (January 31, 2023; updated July 20, 2023), https://openai.com/index/new-ai-classifier-for-indicating-ai-written-text/
- Liang, Yuksekgonul, Mao, Wu and Zou, “GPT detectors are biased against non-native English writers” (2023, Patterns), https://arxiv.org/abs/2304.02819
- Weber-Wulff et al., “Testing of Detection Tools for AI-Generated Text” (2023, International Journal for Educational Integrity), https://arxiv.org/abs/2306.15666
- Turnitin, “Understanding the false positive rate for sentences of our AI writing detection capability” (June 14, 2023), https://www.turnitin.co.uk/blog/understanding-the-false-positive-rate-for-sentences-of-our-ai-writing-detection-capability
- Google Search Central, “Google Search’s guidance about AI-generated content” (February 8, 2023), https://developers.google.com/search/blog/2023/02/google-search-and-ai-content
