> ## Documentation Index > Fetch the complete documentation index at: https://contract-auditor.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # GitHub Actions > Run the auditor on every pull request. The minimum setup needs no API key. The auditor ships as a GitHub Action backed by a container on GHCR. It audits the repository on each pull request, writes findings onto the diff as inline annotations, and optionally pushes verified drift to Slack or Telegram. You do not have to write any of this by hand. `init` reads the repository and writes the whole file, including the branches to audit and every optional input already wired to a secret: see the [Quickstart](/quickstart). This page is the reference for what it wrote, and for changing it afterwards. ## Minimum setup No API key, no secret. The deterministic layer alone scores F1 0.889 against the project's own evaluation, so this is worth running before you decide anything about model spend. ```yaml .github/workflows/contract-audit.yml theme={null} name: Contract audit on: [pull_request] permissions: contents: read security-events: write # required for the SARIF upload jobs: audit: runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 - id: audit uses: samso9th/contract-auditor@v1 continue-on-error: true # report first, block later with: spec: openapi.json source-dir: internal strip-prefix: /api/v1 fail-on: none - uses: github/codeql-action/upload-sarif@v4 if: always() with: sarif_file: ${{ steps.audit.outputs.sarif }} ``` Findings now appear as annotations on the exact lines of the pull request diff. A Markdown summary is written to the run's job summary automatically, so there is something readable even before the SARIF upload is configured. ## Getting the three paths right Most first runs fail on one of these. | Input | What it means | Finding yours | | -------------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | | `spec` | The OpenAPI document your integrators build against | Commonly `openapi.json`, `docs/openapi.json`, `api/openapi.yaml`, or wherever your generator writes it | | `source-dir` | Where route registrations and handlers live | Per language, below | | `strip-prefix` | The prefix present in code but absent from spec paths | If code registers `/api/v1/payouts` and the spec documents `/payouts`, this is `/api/v1` | The last two are what differ between projects: | Your project | `source-dir` | `strip-prefix` | Detected by | | --------------------------------- | ---------------------------------------------------------------------------------- | -------------- | ----------------------------------------------------------- | | **Go** (net/http, gin, chi, echo) | `internal`, `cmd`, or the module root | `/v1` | `go.mod`, or any `.go` file | | **TypeScript** (Express) | `src` | `/api/v1` | `package.json`, or any `.ts` file | | **Python** (FastAPI, Flask) | the package your app lives in, commonly `app` | `/api/v1` | `requirements.txt`, `pyproject.toml`, `setup.py`, `Pipfile` | | **PHP** (Laravel) | the project root, so both `routes/api.php` and `app/Http/Controllers` are readable | `/api` | `artisan`, `composer.json`, `routes/api.php` | `language` is detected from the markers in the last column and only needs setting in a polyglot repository, where the first marker found wins and may not be the one you meant. If a first run reports that **every** route is missing from the spec, `strip-prefix` is wrong. Your spec's `servers[].url` usually names it. ## Telling the contract apart from the dashboard Most codebases register two APIs in one place: the one integrators hold an API key for, and the one a dashboard or admin console talks to with a session token. Only the first was ever promised to anyone. Their paths do not separate them, which is why a list of path globs goes stale the week after it is written. What separates them is which guard they sit behind. ```yaml theme={null} - uses: samso9th/contract-auditor@v1 with: spec: docs/openapi.json source-dir: src strip-prefix: /api/v1 contract-middleware: authenticate ``` Only routes registered with that middleware are audited, on both sides. An operation that the spec documents and a non-contract guard protects will not then read as missing from the code, and a route added next week lands on the right side without anyone editing a list. Both registration styles are read: ```ts theme={null} router.use(authenticate); // guards the whole router router.get("/orders", listOrders); // in the contract router.get("/stats", adminAuthenticate, getStats); // out of it ``` Route middleware is extracted for every supported language. Where a project supplies none, the run fails and says so instead of quietly excluding every route. Use `exclude-paths` there instead. ### What the guard tells you beyond the filter Restricting the audit to one guard also answers a question nobody asks in code review: **what can this credential actually reach?** The findings that come back are exactly the endpoints behind the guard that reads the credential your spec promises integrators. An endpoint you believed was dashboard-only appearing in that list is a finding about your architecture, not your documentation. It happens for an ordinary reason. A guard written to accept an API key **or** a session token protects every route behind it with both, so a key issued to move money also reaches account management. Nothing contradicts it, because nothing wrote it down. Two rules report the clear-cut cases directly, wherever route middleware is recorded: | Kind | Severity | Fires when | | ------------------------- | ------------ | -------------------------------------------------------------------------------------------------------------------- | | `auth_guard_missing` | **critical** | the spec declares security for the operation and the route carries no guard that reads any credential the spec names | | `auth_guard_undocumented` | high | the route is guarded and the spec documents the operation as public, so an integrator following it is answered 401 | Both are reported per file and only for files that guard something, so a project applying authentication once at application level gets silence rather than a page of false alarms. `router.use(mw)` guards only the routes registered **after** it. A route above that line is open however protected the rest of the file looks, and that is the shape `auth_guard_missing` exists to catch. ## Routes that are not part of the contract Most codebases register endpoints no integrator was ever promised: a dashboard's own session routes, internal health checks, an admin surface. Reporting them as drift is not wrong, just irrelevant, and a report that is mostly irrelevant stops being read. ```yaml theme={null} - uses: samso9th/contract-auditor@v1 with: spec: docs/openapi.json source-dir: src strip-prefix: /api/v1 exclude-paths: | /auth/* /internal/* ``` Patterns are matched against the path **as your spec writes it**, so after `strip-prefix` has been removed, and `*` crosses slashes: `/auth/*` covers `/auth/me/password`. A trailing `/*` covers the collection itself as well, so `/auth/*` also excludes `/auth` while leaving `/authorize` alone. Commas work as well as newlines, for a short list on one line. An excluded path leaves the audit in **both** directions. It counts as neither missing from the spec nor missing from the code, because the spec is not wrong to stay quiet about a route you have declared internal. Excluding every endpoint fails the run rather than reporting a clean audit. The usual way to do that by accident is writing the prefix back in: `/api/v1/*` matches nothing, because the prefix is already gone by the time patterns are applied. ## Adding the judgment pass The deterministic rules settle everything mechanical. Three kinds of drift need reading comprehension instead: a handler that requires a field the spec calls optional, a default that quietly changed, and a validation bound loosened below what is documented. ```yaml theme={null} - id: audit uses: samso9th/contract-auditor@v1 with: spec: docs/openapi.json source-dir: src language: typescript strip-prefix: /api/v1 api-key: ${{ secrets.OPENROUTER_API_KEY }} workers: "16" fail-on: critical ``` Add `OPENROUTER_API_KEY` under **Settings → Secrets and variables → Actions**. Any OpenAI-compatible endpoint works; set `base-url` to point elsewhere. [API keys and secrets](/secrets) covers where each credential comes from, and why a pull request from a fork never receives one. Every claim, from the rules and the model alike, is executed against your handler before it reaches the report. In the project's own evaluation the model's raw precision was **0.23**; the gate refuted every false claim and the report came out at precision **1.0**. What you end up reading has been checked against your own code. See [the verification gate](/verification-gate). Cost across the full 16-case evaluation was **\$0.044 for 159 model calls**. One repository audit is a fraction of that. ## Handing findings to a coding agent Every run produces a fix brief: one document covering every finding, what was observed, and the test that proved it. There are three ways to use it, and none of them need storage set up. The files are uploaded as a GitHub Actions artifact, which needs no bucket, no account and no credentials. ```yaml theme={null} - uses: actions/upload-artifact@v7 if: always() with: name: contract-audit-brief path: ${{ steps.audit.outputs.brief-dir }} ``` The pull request comment contains the whole brief in a collapsible block. Expand it, copy it, paste it into Cursor, Codex or Claude Code. Nothing to download. Best when you want a fix started immediately and do not care about running the tests yourself. `_brief.md` from the run's Artifacts section. The same document as a file, for attaching to a ticket or keeping alongside a branch. `_brief.zip` contains the brief and a `tests/` directory holding every generated test as a runnable file. This is the most useful option. You do not have to tell the agent what "fixed" means, because each test already asserts what the specification promises and currently fails. When it passes, that finding is resolved. The brief opens by telling the agent the one judgement the tool deliberately leaves alone: whether the code drifted or the document went stale. Getting that backwards turns a documentation edit into a breaking change for everyone already integrated, so the brief asks the agent to decide explicitly and say why. ## Inputs | Input | Default | Notes | | --------------------- | -------------------- | --------------------------------------------------------------------------------------------------- | | `spec` | n/a | **Required.** Path to your OpenAPI document. | | `source-dir` | `.` | Directory containing the API source. | | `language` | `auto` | `go`, `typescript`, or detect. | | `strip-prefix` | n/a | Prefix to strip so routes match spec paths. | | `contract-middleware` | n/a | Only audit routes guarded by these middleware, one name per line. Also enables the two guard rules. | | `exclude-paths` | n/a | Globs to leave out of the audit, one per line. Matched after `strip-prefix` is removed. | | `api-key` | n/a | Omit for deterministic-only: no cost, no secret. | | `base-url` | OpenRouter | Any OpenAI-compatible endpoint. | | `model` | `z-ai/glm-5.3-flash` | Any model id your endpoint serves. | | `reasoning` | n/a | `off`, `low`, `medium`, `high`. Slower and dearer; omit for the model's own default. | | `workers` | `8` | Concurrent model calls. Raise to shorten wall clock. | | `fail-on` | `high` | `critical`, `high`, `medium`, `low`, `none`. | | `memory-url` | n/a | Turns on self-improvement. Storage you own; nothing is kept in your repo. | | `memory-key-id` | n/a | Access key id, or Cloudinary API key. Pass a secret. | | `memory-secret` | n/a | Secret key, or Cloudinary API secret. | | `memory-token` | n/a | Bearer token for an HTTP store, or an IPFS pinning JWT. | | `memory-region` | n/a | Region for an S3-compatible store. | | `webhook-url` | n/a | Full JSON report POSTed to a data sink. | | `webhook-secret` | n/a | Sent as `X-Auditor-Token`. | | `slack-webhook-url` | n/a | Formatted Slack message, verified findings only. | | `telegram-bot-token` | n/a | With `telegram-chat-id`. | | `notify-min-severity` | `high` | Lowest severity worth interrupting a human for. | **Outputs:** `findings`, `critical`, `high`, `sarif`, `summary`, `brief`, `brief-zip`, `brief-dir`. ## Turning on self-improvement The auditor can learn from its own mistakes across runs, but only if you tell it where to keep the record. There is no default location and no shared store. Leave `memory-url` out and it keeps no memory at all. ```yaml theme={null} - uses: samso9th/contract-auditor@v1 with: spec: openapi.json api-key: ${{ secrets.OPENROUTER_API_KEY }} memory-url: s3://my-bucket/contract-auditor memory-key-id: ${{ secrets.MEMORY_KEY_ID }} memory-secret: ${{ secrets.MEMORY_SECRET }} memory-region: eu-west-1 ``` Any S3-compatible bucket works. For Cloudflare R2, MinIO, Spaces or Backblaze, add the endpoint to the URL: ```yaml theme={null} memory-url: s3://my-bucket/contract-auditor?endpoint=https://.r2.cloudflarestorage.com ``` Other backends, same shape: | Store | `memory-url` | Credentials | | ----------------------------------------- | -------------------------------- | -------------------------------- | | Any HTTPS endpoint that takes GET and PUT | `https://host/ledger.jsonl` | `memory-token` | | Cloudinary raw storage | `cloudinary://cloud-name/ledger` | `memory-key-id`, `memory-secret` | | IPFS via a Pinata-compatible pinner | `ipfs://ledger` | `memory-token` | Two things change when you set this. Each run reads the ledger before auditing and appends to it afterwards, so the store has to be reachable from the runner. And the verification gate starts running against your repository, because a claim with no verdict teaches nothing: a temporary test is written next to your code, executed, and deleted. A claim its test disproves is dropped from the report. One whose test cannot be built is kept and marked, never silently lost. Memory only ever adjusts priors. It changes which past mistakes the model is shown and how findings are ranked, and it can never suppress a finding: if the generated test still fails, the finding ships. About one endpoint in twenty is audited with memory disabled on purpose, so a wrong assumption keeps getting tested instead of settling in permanently. Nothing is written into your checkout, and no ledger ships inside the action's image: the build deletes any that reaches it and fails if one remains. ## Alerting a human Two different jobs, deliberately separate. Formatted, verified-only, and **silent on a clean run**. ```yaml theme={null} with: slack-webhook-url: ${{ secrets.SLACK_WEBHOOK_URL }} notify-min-severity: critical ``` See [notifications](/notifications) for the formatting and filtering rules. Raw JSON, every finding, for Postgres or a warehouse. ```yaml theme={null} with: webhook-url: ${{ secrets.AUDITOR_WEBHOOK_URL }} webhook-secret: ${{ secrets.AUDITOR_WEBHOOK_SECRET }} ``` `file` and `line` are structured fields, so a sink never parses prose to place a finding. ```json theme={null} { "tool": "contract-auditor", "repository": "owner/repo", "sha": "abc123", "run_url": "https://github.com/owner/repo/actions/runs/123", "summary": { "total": 3, "by_severity": { "critical": 1, "high": 2 } }, "findings": [ { "path": "/payouts", "method": "post", "kind": "response_type_mismatch", "detail": "amount", "severity": "critical", "file": "handlers/types.go", "line": 27, "evidence": "PayoutResponse.Amount is float64 (JSON number); spec declares string", "verdict": "confirmed" } ] } ``` ```yaml theme={null} - uses: marocchino/sticky-pull-request-comment@v2 if: github.event_name == 'pull_request' with: path: ${{ steps.audit.outputs.summary }} ``` Needs `pull-requests: write` in the job's `permissions`. ## Rolling it out to a team `fail-on: none` with `continue-on-error: true`. Nothing blocks. Add the SARIF upload, so they appear where people already look. Fix what is real. Record deliberate differences in `auditor/memory/allowlist.json`. Each entry needs a reason and a date, so the allowlist does not become a place where findings go to be forgotten. `fail-on: critical`, then `high` once the signal is trusted. Do not block merges on day one. That is the quickest way to get the tool removed in week two. ## Scheduled audits Pull request runs catch drift as it is introduced. A weekly run catches the drift that was already there before you adopted the tool, and drift in the spec itself. ```yaml theme={null} on: schedule: - cron: "0 9 * * 1" # Mondays, 09:00 UTC pull_request: ``` ## Troubleshooting `strip-prefix` is wrong. If code registers `/api/v1/payouts` and the spec documents `/payouts`, set `strip-prefix: /api/v1`. Dashboard session routes, health checks and admin surfaces are registered in code and deliberately absent from an integrator-facing spec, so every audit reports them. Leave them out with `exclude-paths`, one glob per line, matched against the path as the spec writes it. The package is private, or was never linked to its repository. Check with `gh api /user/packages/container/contract-auditor --jq '.visibility'`. Publishing from a public repository normally yields a public package automatically; if it reports `private`, set Package settings → visibility → Public. Settings → Actions → General → Workflow permissions → **Read and write**. The workflow requesting `packages: write` is not enough on its own; the repository default caps what the token can be granted. The gate could not compile or run its generated test, usually a Go toolchain mismatch: the image pins Go 1.24. These findings are kept and flagged, never silently dropped, because a broken toolchain must never look like a clean bill of health. Deterministic findings are unaffected; they execute nothing. The model returned something unparseable for those endpoints even after a retry, so the judgment pass did not read them. That gets reported rather than hidden, because an endpoint nobody read is not the same as an endpoint with no drift. Route extraction works for TypeScript; the verification gate does not yet. Unverified claims are not let through, so they are reported as unsupported.