Published October 18, 2026 | Version v1

Free-Tier LLM Gateways as AI-Text Detectors: A Dated Benchmark of Model Accuracy, Quota Regimes, Availability Churn and Out-of-Sample Stability

Authors/Creators

  • 1. ESI Rabat

Description

General-purpose LLMs asked to output a structured AI-ness verdict can serve as text detectors at near-zero cost, and their availability is governed less by model quality than by the free-tier policies of the gateways that serve them. This report benchmarks 20+ models across four gateways (OpenRouter free pool, NVIDIA NIM free tier, Google AI Studio free tier, and a local multi-account router) on a 48-text in-sample corpus and a 48-text disjoint holdout drawn from HC3, all scored through the same strict-JSON prompt, a fixed decision band (AI >= 0.69, HUMAN <= 0.40) and a confidence gate (>= 0.60). Three findings stand out. First, model choice dominates tuning: identical prompts and parsers span 0.31 to 1.00 corpus accuracy. Second, each gateway enforces a materially different regime -- OpenRouter has two 429 classes that require opposite remedies, Google's per-model daily quota (nominally 20) delivers a volume that is per model and per project (5-46 requests in one window observed) and stacks linearly with extra project keys, and NVIDIA free serving windows open and close within hours -- so a free model's measured accuracy and its usability as a live lane are separate, per-gateway questions. Third, in-sample perfection does not transfer: a lane that erred zero times on the corpus its band was fitted on produced a false negative out of sample. Part II (window 2026-10-13 to 2026-10-17) adds four findings and one retraction. (1) Accuracy does not survive listing: the lane Part I ranked first was delisted by its gateway four days after its window closed (HTTP 410, permanent), and another advertised model answers 404 on every real call while still appearing in GET /v1/models -- so availability must be probed per model, per gateway, on the day of use, and a catalogue listing is not a health check. (2) Text length is a first-order variable: on 48 texts of >=3000 characters the shipped band made zero wrong calls, while on 24 texts under 300 characters the same rule made 6 wrong calls out of 24; the band refit for that regime erases them but cuts coverage from 92% to 29%. (3) Surface formatting was inflating the headline: removing scrape noise from the human side moved the two-model ensemble from 0.938 to 0.875 accuracy per corpus and produced its first AI-passes-as-human miss, while AUC barely moved. (4) Detectors can be made to show their work: asking for verbatim evidence spans in the same strict-JSON reply yields quotes on most answered texts (575 over 144), and their strengths occupy 0.30-0.95 rather than 0-1. The retraction: the two-model tie-break that Part I replayed at 0.938 decided 3 of 48 texts on the shipped primary and got all 3 wrong, so it now ships disabled -- on the evidence measured here, a single calibrated lane beats an unvalidated second opinion. The shipped configuration is reported with its operating point (one lane, band AI >= 0.80 / HUMAN <= 0.55, confidence floors 0.50; K=5 knee refit over train+val: 156 decided, 3 wrong, 0.981 accuracy per decided) and its transport ceiling. A practitioner's decision table and explicit limitations sections accompany the results.

Notes (English)

Preprint / technical report; not peer reviewed. Part I measurement window 2026-09-28 to 2026-10-13; Part II window 2026-10-13 to 2026-10-17. All availability, quota and latency claims are dated to their gateway and window and are expected to expire: two models recommended or analysed in Part I were unreachable (HTTP 410 and HTTP 404) within four days of the Part I window closing. The CSVs in this record are the tables the report argues from; raw third-party corpus text is not redistributed (see report §8). The same report is also served as a live asset by the benchmarking application at /research/report.html.

Files

model-ranking.csv

Files (925.6 kB)

Name Size
md5:2e103b16b372f27a1bc7fd7f92ff1ed8
7.3 kB Preview Download
md5:8cc154ee31a88a2eab4c4c184fa1bb87
367.5 kB Preview Download
md5:e2e82cafcf6935ab13caa7b9906eaefd
4.1 kB Preview Download
md5:de63fa36e27104ba536ac21e44ef3d22
53.6 kB Download
md5:ff7337f84aa920e2b908cdf9647720e7
471.2 kB Preview Download
md5:064b87bad30d42960735b2c235fc4fb5
21.9 kB Preview Download

Additional details

Related works

References
Preprint: 10.48550/arXiv.2301.07597 (DOI)
Preprint: 10.48550/arXiv.2309.02731 (DOI)