Breaking Changes in CI

Inside one repository the compiler already stops a change that breaks a call site. The API's other callers — a mobile app on last month's release, a partner's integration, another team's service — have no compiler to stop it. courier-studio diff is that check: it sets the definition in your branch against the one at a git ref, says what each change does to the callers, counts the recorded calls each changed endpoint has, and fails on a change that breaks them.

npx @noverax/courier-studio diff --base origin/main

What breaks a caller

Each endpoint is matched by its name in the client — orders.get — so a route that moved is a route changed, not one endpoint removed and another added.

A type that changed to another altogether — a string that became a number — breaks callers on both sides.

The base to compare with

--base <ref> reads the definition at a git ref — origin/main, a tag, a commit; origin/main by default. The ref is checked out in a worktree of its own inside .courier, which keeps itself out of git, so its courier.config.ts loads with the project's installed packages; the worktree is removed when the command ends. In CI, fetch the base branch first: actions/checkout with fetch-depth: 0.

--base-openapi <path> compares with OpenAPI documents written before instead — a folder of <source>.openapi.json, as courier-studio openapi --out <folder> writes them, or one file — for a base kept as documents, or one from another repository.

The recorded calls each change touches

With recorded tests in tests/recorded — or the recordings --recordings <path> names — each changed endpoint says how many recorded calls it has, and the summary how many of them the new definition no longer fits. A breaking change on an endpoint your app calls a thousand times a day reads differently from one on an endpoint nobody calls.

Agent access

The report ends with what the change does to the tools AI agents may call, by the project's agents policy at the base and in the branch: each tool added, each kept from agents, each offered under another name, access or confirmation — a new endpoint offer: "all" hands to agents, a deny that now keeps a group out, a write that now needs a person:

Agent access: 1 tool added, 1 tool kept from agents: agents may call 4 tools
  + shop.orders.refund  orders_refund  destructive, a person confirms  marked .agent()
  - billing.charge  billing_charge  write

In Markdown it is a table a reviewer reads beside the definition's changes, and a pull request that changes only what agents may call gets its comment too. A config whose policy the MCP server would refuse is named where it fails, before the change or after it. It reports, and fails nothing: --fail-on counts the definition's changes alone. A base given as OpenAPI documents records no policy, so the report then says how many tools agents may call in the branch, and nothing about the base.

In a pull request

on: pull_request

permissions:
  contents: read
  pull-requests: write

jobs:
  api-changes:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - run: npm ci
      - name: Report what the API change breaks
        run: npx @noverax/courier-studio diff --base origin/${{ github.base_ref }} --format markdown --out "$GITHUB_STEP_SUMMARY" --comment
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

--comment posts the report in the pull request's conversation, where its reviewers read it, as one comment: the next run edits that comment rather than adding another, and leaves it alone when the report is the same. A pull request that leaves the definition and what agents may call alone gets no comment, and one whose breaking change was taken back sees its comment say so. Each project in a repository keeps a comment of its own, named by its folder.

Posting takes a token that may write to pull requests — permissions: pull-requests: write — which a pull request from a fork does not get. On GitHub Enterprise Server the job names the server's API, and the report goes there. Posting is a report, not the check: when the report cannot be posted, the job's log says why, and the command passes or fails as it would without --comment.