What counts as drift

Evaluation runs against a synthetic Go payments API whose spec matches it exactly in the clean state. Drift is then injected, so ground truth is known by construction.

The 12 drifts

ID	Severity	What changed
D01	high	Response field renamed in code, spec unchanged
D02	medium	Handler returns 200, spec documents 201
D03	high	Endpoint in code, absent from spec
D04	high	Spec documents an endpoint that is no longer routed
D05	critical	Money field changed from decimal string to `float64`
D06	high	Code requires a field the spec marks optional
D07	high	Query param read as `per_page`, documented as `perPage`
D08	critical	Endpoint now requires auth, documented as public
D09	low	Pagination default changed, docs stale
D10	critical	Webhook signature header renamed
D11	medium	Validation loosened below the documented constraint
D12	medium	New 429 response undocumented

Authentication, checked at the guard

D08 is caught by reading the handler: an endpoint the spec calls public whose code reads a credential. That direction is unambiguous, but it only answers half the question. Authentication is usually applied by middleware instead of inside the handler, so a handler that reads no header tells you nothing about whether the endpoint is protected. Where a language’s extractor records route middleware, both directions are checked against the guard instead:

Kind	Severity	What it means
`auth_guard_missing`	critical	The spec declares a security scheme for the operation, and the route carries no middleware that reads any credential the spec names. Documented as protected, registered open
`auth_guard_undocumented`	high	The route is guarded, and the spec documents the operation as needing no authentication. An integrator following the spec is answered 401

The pattern that motivates the first one is ordering. router.use(mw) guards only the routes registered after it, so a route sitting above that line is open however protected the rest of the file looks:

router.post("/hooks/incoming", handleHook);   // open
router.use(authenticate);
router.get("/logs", listLogs);                // guarded

Both rules are reported per file, and only for files that guard something. A project that applies authentication once at the application level registers no route middleware at all, so calling every one of its routes unguarded would fill a page with false alarms. What makes something a guard is what it reads, not what it is called. The spec’s components.securitySchemes names the credential, and any middleware whose source reads that header counts as an authentication guard, whatever its name.

The three worst ones

D05 Money precision

"1500.50" becomes 1500.50. String parsers break, float parsers lose precision.

D08 Silent auth

A documented-public endpoint starts returning 401.

D10 Webhook signature

Header renamed. Every delivery fails verification on the receiving end.

All three are silent. Code compiles, tests pass, endpoint returns 200. The damage lands in the partner’s system.

The 4 decoys

A local rename, an added comment, an extracted helper, a reordered field. These are code changes that leave the contract alone, so a finding on any of them is a false positive. Without decoys, an auditor that reports everything scores perfectly.

Targets

Committed before the first run, so they cannot be adjusted afterwards.

Recall ≥ 0.80

Precision ≥ 0.85

3/3 critical

4/4 decoys clean

Primary metric is F1: recall alone rewards noise, precision alone rewards silence.

Getting started

How it works

Reference

The 12 drifts

Authentication, checked at the guard

The three worst ones

D05 Money precision

D08 Silent auth

D10 Webhook signature

The 4 decoys

Targets

Recall ≥ 0.80

Precision ≥ 0.85

3/3 critical

4/4 decoys clean

​The 12 drifts

​Authentication, checked at the guard

​The three worst ones

D05 Money precision

D08 Silent auth

D10 Webhook signature

​The 4 decoys

​Targets

Recall ≥ 0.80

Precision ≥ 0.85

3/3 critical

4/4 decoys clean

The 12 drifts

Authentication, checked at the guard

The three worst ones

The 4 decoys

Targets