Why AI detection keeps breaking down on contact

The internet’s trust problem has a new shape. AI-generated text is turning up in job applications, product reviews, insurance claims, and the usual flood of spam. Images are in the mix too. That’s the backdrop for Pangram co-founder and CEO Max Spero’s appearance on TechCrunch’s Equity podcast, where he argued that “real or fake” is the wrong model for AI detection.

That’s the part worth paying attention to. Detection sounds binary. It isn’t. Once you get past toy examples, it turns into a messy classification problem with shifting thresholds, adversarial users, and a lot of false confidence.

The binary question breaks fast

A lot of people still talk about AI detection as if it were a yes-or-no button. Was this written by a model or by a person? That framing falls apart quickly.

Content rarely comes in clean categories. A job applicant might draft a resume with ChatGPT, then rewrite half of it. A customer review might be mostly human, with one paragraph generated to sound smoother. An insurance form might be filled out by a claimant who uses AI to summarize a messy incident report. So what are you detecting, exactly? Full generation? Assistance? Heavy editing? A model’s fingerprints on a human document?

That distinction matters because different buyers want different answers.

• A platform trying to stop spam wants high recall on obvious synthetic abuse.
• A hiring team may care about whether a candidate actually authored their cover letter.
• A compliance team may want evidence that a claim was machine-assisted, not proof beyond doubt.

Those are not the same problem. A detector that tries to answer all of them with one score is usually bluffing.

Detection is a moving target

AI text detection has a basic weakness: models change faster than detectors do. Every new generation gets better at mimicking human variation. It gets less robotic, less repetitive, less obviously AI-like. Then people add paraphrasers, humanizers, prompt tricks, translation hops, and manual edits. The signal gets weaker.

Under the hood, many detectors still lean on statistical patterns. They look for token likelihoods, repetition patterns, sentence uniformity, burstiness, or embedding-based classifiers trained on known outputs. Those methods can work in controlled settings. They also break when the input shifts.

That’s the problem. Detection systems are often good at detecting the distribution they were trained on. The internet doesn’t stay there for long.

That’s why vendors end up tuning for risk rather than certainty. A detector can be useful as a triage layer, a moderation signal, or a fraud score. It’s a poor basis for punitive decisions if you can’t explain the confidence, the error rate, and the failure modes.

Why “assisted” matters more than “generated”

Spero’s framing, at least as summarized by TechCrunch, points to the line between AI-assisted and AI-generated work. That’s where the operational mess lives.

Take hiring. If a candidate uses an LLM to clean up grammar, shorten a bullet list, or rephrase a summary, is that deception? Many teams would say no. If they paste a fully generated cover letter tailored to the role, some teams would say yes. If they use AI to write code, the answer may depend on the role, the policy, and whether they understand the code they submit.

The same ambiguity shows up in reviews and claims. A person can use AI to express a real experience more clearly. A spammer can generate thousands of fake reviews with almost no effort. One is assistance. The other is fabrication. Detection tools have a hard time separating those cleanly because both can produce fluent text.

That’s where policy matters more than model scores. Companies need to define acceptable assistance before they ask a detector to enforce some vague moral line. Otherwise they’re handing judgment to a classifier and hoping it understands context.

It won’t.

Accuracy is only part of the problem

Most conversations about AI detection fixate on accuracy numbers, but that’s only one piece of the system. Real deployments have to deal with latency, throughput, adversarial pressure, and explainability.

A detection model that takes 400 milliseconds per request might be fine for a moderation back office. It’s expensive for a high-volume submission pipeline. A detector that returns “87% likely AI” sounds useful until legal, HR, or trust and safety asks what that score means, how it was calibrated, and whether it drifts by language, domain, or writing style.

Then there’s fairness. Non-native English speakers, highly structured business writing, and formulaic domain text can look AI-like to naive detectors. That’s not a corner case. It’s a built-in risk when you train on a narrow idea of what human writing should look like.

A detector that flags too aggressively can turn into a discrimination machine with a dashboard.

Watermarking helps, but only in narrow cases

Some people want a clean fix through watermarking or provenance. That sounds neat until you look at how these systems behave.

Watermarks depend on model cooperation. If the generation model doesn’t embed one, or if the text is lightly edited, copied through another tool, translated, or compressed into another format, the signal can disappear. Provenance systems like signed metadata help more when content stays inside an ecosystem that preserves those signatures. Once content moves through screenshots, copy-paste, or platform rehosting, the chain gets fragile fast.

For images, provenance standards can do more than text-only heuristics because visual metadata and capture-chain information give you something concrete to inspect. Adoption is still uneven, though. The web is very good at stripping context.

So the best systems usually combine signals:

• content analysis,
• metadata and provenance,
• account behavior,
• submission patterns,
• and human review for edge cases.

That’s less elegant than a magic detector. It’s also closer to how fraud detection actually works.

What developers should take from this

If you’re building a product that touches user-generated text, don’t treat AI detection as a binary gate. Treat it like a risk signal with a narrow contract.

A few practical rules help:

1. Decide what you’re detecting. Full generation, heavy assistance, spam, or policy violation. Pick one.
2. Measure false positives by segment. Native and non-native writers, short and long text, different domains, different languages.
3. Don’t make the detector the judge. Use it to route content, not to auto-ban by default.
4. Keep a human appeal path. If the system can damage someone’s job application or claim, it needs review.
5. Instrument drift. Model updates, prompt changes, and new user behavior will shift the data.

That last point gets missed a lot. Detection quality decays quietly. Teams deploy a model, get a few weeks of clean-looking metrics, then the abuse patterns adapt and the system starts missing the interesting stuff while nailing harmless users.

That’s how these tools fail in production. Not with a dramatic collapse. Just a slow slide into noise.

Detection won’t solve the trust problem by itself

Synthetic content is now cheap enough to flood ordinary workflows, not just social feeds. That changes the economics of fraud and spam. It also changes the burden on platforms and enterprise systems that were never built to answer “was this human-written?” at scale.

Detection will stay useful, but only as part of a wider trust stack. Provenance where you can get it. Behavioral signals where you can’t. Clear policy everywhere. Human review when the stakes are real.

That’s a less flashy answer than “AI detector solves the problem.” It’s also the one that survives contact with the internet.