AI-text detectors have a bad reputation, mostly earned. Plenty of them are black boxes that flag a nervous student's essay as a robot and offer no way to check the claim. This page is the opposite: the corpus, the calibration, the operating point, and the parts it gets wrong. Every number below comes from a script you can run.
The detector reads prose and scores how much it looks like current-model AI writing, from a set of measurable features: the density of the words and stock phrases these models overuse, sentence-length uniformity, connective-opener overuse, chat-UI artifacts left in a paste, and a few dozen more. Those features feed a calibrated model that returns a probability. It is not a vibe and it is not a language model judging another language model; it is a scorer whose every input you can inspect.
It is measured against a labeled corpus of human-written and AI-written text, held separate from the data used to fit it. The human side deliberately includes the writing most likely to trip a detector: formal registers, non-native English, dry technical documentation. The point of stacking it that way is to measure the false-positive rate honestly instead of flattering it.
Measured on the curated corpus with the shipped calibration (reproducible, see below):
| Metric | Prose | What it means |
|---|---|---|
| AUROC | 0.984 | How well it ranks AI above human overall. 1.0 is perfect, 0.5 is a coin flip. |
| Caught at the 5% false-positive point | ~90% | Of the AI texts, this share is flagged while only 5% of human texts would be. |
| False positives on the human corpus | 0% | At the shipped operating threshold, on this curated human set. |
Source code has its own engine, measured the same way (AUROC 0.96 on a human-code set stacked with the samples most likely to false-positive). SlopScreen scores the prose of a PR or issue by default, because that is where the detector is strongest.
The detector is tuned to a 5% false-positive operating point: it would rather miss a bot than accuse a person. Below roughly twenty words, or on text it reads as non-English, it does not guess at all. It says it abstained and shows you why. A one-line "lgtm" gets no verdict, because no honest detector can give one.
False positives are real. Heavily formal writing, some non-native English, and boilerplate-heavy prose can score higher than the person who wrote them deserves. A human who writes cleanly in the register these models imitate can be flagged. Here is a concrete example I will not hide: text written densely about the vocabulary these models overuse trips the style signals, so some of my own writing about this detector scores as AI-ish. That is the failure mode, shown rather than papered over. It is exactly why every result in the extension is labeled a signal at a stated false-positive rate, never proof, and why the advice is always the same: judge the contribution, not the author. The detector points you at text worth a second look. You do the looking.
The scoring engine is the same one used here, called exactly as the extension calls it. The evaluation is one command against the labeled corpus:
python3 eval/run_eval.py
It prints the AUROC, the catch rate at the 5% false-positive point, and the human false-positive count, so the table above is not something you have to take on faith.