Quickstart

1. Generate the workflow

Requires: Docker, and nothing else. Run this inside the repository you want audited:

docker run --rm --pull=always -v "$PWD:/github/workspace" -w /github/workspace \
  ghcr.io/samso9th/contract-auditor:v1 init

--pull=always matters: v1 is a moving tag, and docker run reuses whatever it already has on disk rather than checking whether the tag now points somewhere else. Without it you keep running the version you first pulled.

It reads the repository and writes .github/workflows/contract-audit.yml. Commit that file and you are done. Every value in that file was worked out by reading your repository, and each one has a comment next to it saying where it came from, so you can check the working:

Input	Derived from
`language`	how strongly each language’s markers are present, and where they tie, which one’s routes account for the spec
`spec`	the candidate document describing the most operations, JSON or YAML
`source-dir`	of the directories the repository declares a unit - a conventional source root, or anything with its own dependency manifest - the one implementing most of the spec with the fewest routes it never mentions
`strip-prefix`	the path component of the spec’s own `servers[].url`, or its `basePath` on Swagger 2.0
`contract-middleware`	the spec names the credential integrators were promised, in `securitySchemes` or `securityDefinitions`. Of the guards in use, the one protecting the endpoints that spec documents - preferring one whose source reads the credential, since a framework often does that reading instead
trigger branches	`git ls-remote`, falling back to the local refs

The middleware row is the one that saves you reading an unfamiliar codebase. Most projects register two APIs in the same place: the one outside developers hold a key for, and the one a dashboard talks to with a session token. Their paths look alike, so what separates them is which guard they sit behind.

init refuses to overwrite a workflow that already exists. Pass --force to replace it, --stdout to print it instead, or --json to see what was detected without writing anything.

Without Docker

Coming soon. Anyone with the codebase checked out already has one of these installed, so it will be the shortest way in:

Runtime	Planned
Node	`npx contract-auditor init`
Python	`pipx run contract-auditor init`
PHP	`composer exec contract-auditor init`
Go	`go run github.com/samso9th/contract-auditor@latest init`
Rust	`cargo run contract-auditor init`

Until then the image above needs only Docker.

2. What runs on GitHub

Once the workflow is committed, every pull request into one of your shared branches is audited, and so is the merged result on those branches. One pull request can add a route while another edits the spec, so each one is audited against its own merge preview, not against whatever they both eventually land on. Each run leaves:

A comment on the pull request, edited in place on each run so you get one comment and not a thread. It gives counts by severity and by what disagrees. It does not list the endpoint paths: a route that is registered but undocumented is usually undocumented on purpose, and listing them would publish an inventory to everyone who can see the pull request.
A fix brief, uploaded as an artifact and linked from that comment. It has every finding, the evidence, and the test that proved it, written so you can paste it straight into Cursor, Codex or Claude Code.

Every input the action takes is already in the generated file, pointing at secrets that may not exist yet. Each is skipped silently when its secret is unset, so the workflow runs exactly as generated. Adding a secret under Settings → Secrets and variables → Actions turns a feature on without editing any YAML:

Secret	Turns on
`OPENROUTER_API_KEY`	the judgment pass, for drift that needs reading comprehension rather than lookup
`CONTRACT_AUDIT_SLACK_WEBHOOK_URL`	Slack notifications, counts only
`CONTRACT_AUDIT_TELEGRAM_BOT_TOKEN`, `..._CHAT_ID`	Telegram notifications
`CONTRACT_AUDIT_WEBHOOK_URL`, `..._SECRET`	the full JSON report POSTed to a sink of your own
`CONTRACT_AUDIT_MEMORY_*`	self-improvement, so later runs know which complaints have been refuted

If that file ever needs editing, it should only be to delete lines you do not want. Where each credential comes from, and setting it → Full input reference and troubleshooting →

3. Running it locally

Only needed to work on the auditor itself, or to audit a repository without putting anything in CI.

Tool	Version	Needed for
Go	1.22+	Building the fixture, running verification tests
Python	3.9+, stdlib only	Everything else
OpenRouter key	n/a	Model-backed steps only

No pip install, no Docker, no database.

git clone https://github.com/Samso9th/contract-auditor
cd contract-auditor

Audit a repository directly

python3 auditor/init.py --repo /path/to/api --json   # what it detects
python3 auditor/run.py \
  --repo /path/to/api \
  --spec /path/to/openapi.json \
  --out reports/runs/my-api

A real repository has no known list of correct answers to score against, so what you get is a ranked report.

Reproduce the evaluation

Build the 16 evaluation cases

make cases

12 injected drifts and 4 decoys. Each is compiled before it counts as a case.

Test the deterministic tools

make test-tools

No API key, no cost.

Confirm the scorer is sound

make check

Expect precision 1.0, recall 1.0, decoys 4/4. Anything else means the scorer is broken, not the agent.

Score the no-model layer

make deterministic

How much of the problem needs no AI at all.

Add a key and run the full auditor

echo 'OPENROUTER_API_KEY=sk-or-...' >> .env
make agent && make score

Any OpenAI-compatible endpoint works. Set OPENROUTER_BASE_URL to redirect.

Reset with make clean. Reset with make clean.

Getting started

How it works

Reference

1. Generate the workflow

Without Docker

2. What runs on GitHub

3. Running it locally

Audit a repository directly

Reproduce the evaluation

​1. Generate the workflow

​Without Docker

​2. What runs on GitHub

​3. Running it locally

​Audit a repository directly

​Reproduce the evaluation

1. Generate the workflow

Without Docker

2. What runs on GitHub

3. Running it locally

Audit a repository directly

Reproduce the evaluation