The Claude fake detector is now an open-source library

In April we published the numbers on fake Claude resellers: 183 channels across 8 providers that answered as Kiro, Codeium or some other model wearing a Claude name tag. The probes that caught them lived inside our sync pipeline. They now live in a package anyone can install.

ai-model-verifier is on npm and GitHub under AGPL-3.0. It takes a base URL, an API key and a model name and returns one of three verdicts, genuine, suspicious or unverified, together with the rule that fired and the raw probe responses next to it.

Why a library

The same checks now run in three places: in your browser on the model tester, on our servers for the public rankings, and inside the sync pipeline that decides which upstream lanes UnoRouter exposes at all. Three copies of a pattern list drift apart. One package does not.

It is isomorphic on purpose. Web-standard APIs only, no node: imports, zero runtime dependencies. The HTTP call is injected, so a browser can route around CORS and a server can route through an SSRF-safe fetch. Providers and detectors are separate entry points, so checking Anthropic never bundles the Gemini config.

What is new since April

The four behaviour probes from the first post are still the core: the kitten story, the haiku, the identity question and the model-name question, now for Anthropic, OpenAI and Gemini shaped endpoints. Around them sit checks that do not depend on asking the model who it is.

Thinking signature. Claude attaches a server-generated signature to every thinking block. A relay serving another model cannot produce one. Identity probes can be coached with a system prompt; a signature cannot.
Token accounting. A relay can forward genuine Claude and still inflate input_tokens on every call. Two prompts that differ by a fixed run of text must differ in billed tokens by a known band, count_tokens must agree with what was billed, and output must stay under max_tokens. Opus 4.7 and 4.8 use a different tokenizer, so the band doubles as a tier check.
Tier mismatch and substitution. Opus billed, Sonnet served slips past behaviour probes, because Sonnet writes a perfectly good kitten story. The model-name probe catches a reply that names the cheaper tier, and the envelope check catches an endpoint that echoes a different model id than the one requested.
Response mixing and language leaks. Every prompt carries a nonce. A reply that echoes someone else's nonce means the proxy is mixing responses between users. CJK characters in a reply to an English prompt point to a substituted Chinese model.
Envelope metadata. A chatcmpl- id inside an Anthropic-shaped reply, or OpenAI usage keys in a Messages response, means a translation layer sits between you and the model. Reported as observation, not verdict: our own gateway is exactly such a layer.
Throughput. Bigger models are slower per token, so a lane billed as Opus that runs at Sonnet speed is worth a look. Sampled, never thresholded: the useful comparison is across lanes serving the same model, and only the caller can assemble that.

What it does not prove

A valid signature proves that a genuine Anthropic path answered. It does not prove which tier: Sonnet returns a perfectly good signature when sold as Opus, so a pass there still needs the tier checks. Replaying the thinking block so Anthropic re-validates the signature sounds like the definitive test. We built it, and it does not work through a relay. Two live upstreams returned 200 for a genuine block and for the same block with 308 bytes of random base64 in place of the signature. A relay re-issues the turn to its own backend, so nothing ever validates it. The option is off by default and documented as such.

The signature and token checks are ported from veridrop, also AGPL, whose tolerances were calibrated against the official API. We kept the loose-looking bounds because real Anthropic responses sat near them.

Use it

The model tester runs the full set in your browser, signature and token checks included, and nothing leaves your machine except the calls to your provider. The rankings page publishes results people chose to share. For your own pipeline:

typescript

import { runVerification } from "ai-model-verifier";

const result = await runVerification({
  provider: "anthropic",
  baseUrl: "https://YOUR-PROVIDER",
  apiKey: process.env.YOUR_KEY,
  model: "claude-opus-4-8",
  mode: "server",
  checkSignature: true,
  checkTokenTruth: true,
});

console.log(result.verdict, result.reasons);
// "suspicious" [ "tier-mismatch: requested claude-opus-4-8, served sonnet" ]

Opt in to what you need. checkSignature and checkTokenTruth each cost extra requests, so they stay off unless asked for. Envelope metadata and throughput are read from responses the run already made and cost nothing.

Test a provider, read the April post for the numbers that started this, or open an issue on GitHub if a real endpoint gets a wrong verdict.