# Built-in Middleware

Reference for retry, auth, logging, rate limiting, caching and header-variable middleware shipped with Courier.

Courier ships six production-ready middleware functions. Import them from `@noverax/courier/middleware`:

```ts
import {
  retryMiddleware,
  authMiddleware,
  logMiddleware,
  rateLimitMiddleware,
  cacheMiddleware,
  headerVariablesMiddleware,
} from "@noverax/courier/middleware"
```

---

## retryMiddleware

Automatic retry with exponential backoff. By default it retries server errors (status >= 500) and requests that got no answer at all — a network failure or a timeout. A 4xx is never retried by default: the same request would be refused again.

```ts title="signature"
declare function retryMiddleware(options?: RetryOptions): Middleware

interface RetryOptions {
  /** How many times a failed attempt is made again (default: 3) */
  limit?: number
  /** The wait before the first retry, in ms (default: 300) */
  delay?: number
  /** How the wait grows between retries (default: "exponential") */
  backoff?: "fixed" | "exponential"
  /** Which answers deserve another attempt: statuses, or a test (default: 0 and >= 500) */
  retryOn?: readonly number[] | ((response: MiddlewareResponse) => boolean)
  /** Retry endpoints that write as well as those that read (default: only a write with an idempotency key) */
  mutations?: boolean
}
```

With `backoff: "exponential"` the wait doubles from `delay` — with the default 300 ms, the waits are 300 ms, 600 ms, 1200 ms; with `"fixed"` every wait is `delay`. An endpoint's `.retry()` and a `courier()` base's `retry` take the same `RetryOptions`.

```ts title="lib/sources.ts"
const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    retryMiddleware({ limit: 3, delay: 300 }),
  ],
  groups: { /* ... */ },
})
```

### Custom retry condition

Give `retryOn` the statuses to retry, 0 standing for a request that got no answer, or a test of the answer:

```ts title="lib/sources.ts"
retryMiddleware({ limit: 2, delay: 500, retryOn: [0, 429, 502, 503, 504] })

retryMiddleware({
  retryOn: (response) => response.status === 429 && response.headers.has("retry-after"),
})
```

> **What the condition sees**
>
> A non-2xx answer rejects the call with an `HTTPError`; `retryOn` is shown the answer it carries — its status, body and headers. A request that got no answer is shown as status 0, so a condition that should also retry network failures has to accept it. Errors that are not HTTP failures — a response that fails its schema, a middleware that throws — are never retried. After the last attempt, its error is re-thrown.

### Writes are sent once — unless they carry a key

An endpoint that writes — a `post`, `put`, `patch` or `del` not declared a read with `.reads()`, or a `.via()` endpoint, which counts as a write until `.reads()` says otherwise — is not retried by default. A write whose answer was lost may already have been carried out, and sending it again can charge a card twice.

An endpoint that declares an `idempotencyKey` is retried: each call sends one key in that header, the same on every attempt, and an upstream that deduplicates by it applies the write once however many times it arrives.

```ts title="api/orders.ts"
place: endpoint
  .post("orders", { idempotencyKey: "header:Idempotency-Key" })
  .retry({ limit: 3 }) // retried: the key makes a repeated write safe
```

`mutations: true` retries every write, for an upstream that recognises a repeated one some other way; `mutations: false` retries none, keys included:

```ts title="lib/sources.ts"
retryMiddleware({ limit: 3, mutations: true })
```

A `retry` policy on a `courier()` base or an endpoint's `.retry()` follows the same rule and takes the same `mutations` flag. Middleware reads a call's key as `ctx.idempotencyKey`, and Studio lists it with the call, and where it stands among the calls sent with the same key.

### Cancellations

A cancellation is never retried, whatever `retryOn` says: not a call whose signal the caller aborted, whatever reason it gave, and not a read that a newer call to its endpoint cancelled through `cancelPrevious` (status 499). A caller that aborts while the middleware waits between attempts is answered at once with its signal's reason.

---

## authMiddleware

JWT/OAuth token injection with automatic refresh on 401.

```ts title="signature"
declare function authMiddleware(options: {
  /** Function to get the current auth token */
  getToken: () => string | Promise<string>
  /** Optional function to refresh the token on 401 */
  refreshToken?: () => string | Promise<string>
  /** Header name (default: "Authorization") */
  header?: string
  /** Auth scheme prefix (default: "Bearer") */
  scheme?: string
}): Middleware
```

### Basic usage

```ts title="lib/sources.ts"
const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    authMiddleware({
      getToken: () => localStorage.getItem("access_token")!,
    }),
  ],
  groups: { /* ... */ },
})
```

### With token refresh

When `refreshToken` is provided and the upstream answers 401, the middleware calls `refreshToken()`, updates the header, and sends the request again once. If the new token is refused too, the call rejects with that second 401. Requests refused while a refresh is running wait for it instead of starting their own, so `refreshToken()` runs once for all of them:

```ts title="lib/sources.ts"
authMiddleware({
  getToken: () => tokenStore.accessToken,
  refreshToken: async () => {
    const { accessToken } = await refreshAccessToken(tokenStore.refreshToken)
    tokenStore.accessToken = accessToken
    return accessToken
  },
})
```

### API key authentication

Set `scheme` to an empty string and use a custom header for non-Bearer auth:

```ts title="lib/sources.ts"
authMiddleware({
  getToken: () => process.env.API_KEY!,
  header: "X-API-Key",
  scheme: "",
})
```

---

## logMiddleware

Request/response logging with timing information.

```ts title="signature"
declare function logMiddleware(options?: {
  /** Receives each line, and the error of a failed request (default: Courier's logger) */
  logger?: (message: string, error?: unknown) => void
}): Middleware
```

Logs each request as `source:group:method → status (duration)`. On errors, logs `source:group:method → ERROR (duration)` and passes the error along. By default the lines go to Courier's logger: a success at `info` on stdout, a failure at `warn` on stderr with the error's stack.

```ts title="lib/sources.ts"
const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    logMiddleware(),
  ],
  groups: { /* ... */ },
})

// Output:
// api:users:getAll → 200 (42ms)
// api:users:create → 201 (89ms)
// api:auth:login → ERROR (1203ms)
```

### Custom logger

Send logs to your observability platform:

```ts title="lib/sources.ts"
import * as Sentry from "@sentry/node"

logMiddleware({
  logger: (message) => {
    Sentry.addBreadcrumb({ category: "api", message })
  },
})
```

---

## rateLimitMiddleware

Client-side rate limiting. Delays requests that exceed the configured limit within a time window.

```ts title="signature"
declare function rateLimitMiddleware(options: {
  /** Maximum number of requests in the window */
  maxRequests: number
  /** Time window in milliseconds */
  window: number
}): Middleware
```

```ts title="lib/sources.ts"
const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    rateLimitMiddleware({ maxRequests: 10, window: 1000 }),
  ],
  groups: { /* ... */ },
})
```

> **Queuing, not rejecting**
>
> Unlike server-side rate limiters that return 429 errors, this middleware **delays** requests until the window allows them. Your code does not need to handle rate limit errors -- the middleware transparently queues requests.

The limit holds however many calls arrive at once: at most `maxRequests` requests go out in any `window` milliseconds, and calls beyond it wait their turn in the order they came. The window is measured from when each request actually went, so a busy event loop that runs the limiter's timers late delays the queue rather than letting several requests out together. A caller that aborts its signal leaves the queue at once and rejects with the signal's reason; its turn passes to the call queued behind it.

---

## cacheMiddleware

In-memory response caching with configurable TTL.

```ts title="signature"
declare function cacheMiddleware(options: {
  /** How responses are partitioned between callers: "public", "session", or a function naming the caller */
  scope: "public" | "session" | ((ctx: MiddlewareContext) => string)
  /** Time-to-live in milliseconds (default: 60000) */
  ttl?: number
  /** Entries retained before the oldest are dropped (default: 1000) */
  maxEntries?: number
  /** Custom storage; a Map satisfies it (default: in-memory, per middleware instance) */
  storage?: CacheStore
}): Middleware
```

Caches successful responses (2xx status) of endpoints that read, keyed by the scope's partition, `source:group:method` and the variables. Non-2xx responses are never cached, nor the answers to a request the session makes for itself, such as the [double-submit token](/docs/server-side/session) a write asks for: it is made for the cookie its answer sets.

```ts title="lib/sources.ts"
const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    cacheMiddleware({ scope: "public", ttl: 30_000 }), // 30 seconds
  ],
  groups: { /* ... */ },
})
```

Every caller receives a copy of its own, made with `structuredClone`, so editing a result never changes what the next caller gets. An answer holding a value that cannot be copied, such as a function, is not cached; class instances come back as plain objects.

Calls that miss together — a burst on an entry that is empty or has just expired — reach the API once: the first fetches, and the others wait for its answer and take a copy, as a hit would. One that stops waiting leaves at once while the others go on. When the first one's answer is not kept — an error, a `Set-Cookie` — each of the others makes its own call.

A call that carries the user's [session](/docs/server-side/session) is never answered from a `"public"` cache, nor stored in it: its answer belongs to that user, and the endpoint is named once in the log. Give such an API's per-user reads a cache of their own with `scope: "session"` — one partition per set of credentials the call sends, its `Cookie` and its `Authorization`, named by their SHA-256 digest so the credentials never sit in a store's keys:

```ts
cacheMiddleware({ scope: "session", ttl: 10_000 })
```

In a browser or a React Native app, the platform sends the session's cookies itself, out of a script's reach. There, every call of an API with a `session` counts as carrying it, so a `"public"` cache keeps none of its answers, and `scope: "session"` partitions by what a script can read — a bearer token, the double-submit token an API renews with the session — and starts a new partition with every write made with the session, as it starts and once it is answered. A user who signs out and another who signs in on the same device, in the same tab or another, never share a cached answer, nor see one read while the sign-in was under way.

### Custom storage

Provide your own `Map` instance to control the cache externally (e.g., for manual invalidation):

```ts title="lib/sources.ts"
const apiCache = new Map()

const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    cacheMiddleware({ scope: "public", ttl: 60_000, storage: apiCache }),
  ],
  groups: { /* ... */ },
})

// Invalidate the entire cache
function invalidateCache() {
  apiCache.clear()
}
```

> **Variables affect the cache key**
>
> Two requests to the same endpoint with different variables produce different cache keys. `users:getById` with `{ id: 1 }` and `{ id: 2 }` are cached separately.

---

## headerVariablesMiddleware

Sends some of a call's variables as request headers instead of in its path, query or body: an API's header parameters, such as a digest or a version the caller picks per call. Generated clients use it for the header parameters an OpenAPI document declares.

```ts title="signature"
declare function headerVariablesMiddleware(names: readonly string[]): Middleware
```

```ts title="lib/sources.ts"
import { createCaller } from "@noverax/courier"
import { courier, defineApi } from "@noverax/courier/define"
import { headerVariablesMiddleware } from "@noverax/courier/middleware"

const { endpoint } = courier()

const api = defineApi({
  baseURL: "https://api.example.com",
  endpoints: {
    find: endpoint
      .get("things")
      .variables<{ q: string; "X-Digest"?: string }>()
      .use(headerVariablesMiddleware(["X-Digest"])),
  },
})

// GET /things?q=red, with the header X-Digest: sha-256=abc
await createCaller(api).find({ q: "red", "X-Digest": "sha-256=abc" })
```

The variables are validated whole, then the named ones are taken out and set as headers under their own names; one left unset sends no header. A list is written comma-separated and an object as its keys and values in turn, as OpenAPI's `simple` style writes a header. For a call through the proxy it runs in your application's server, so the values travel with the variables and the browser sets no such header itself.

---

## Composing Built-in Middleware

Order matters. Place middleware in the array from outermost to innermost:

```ts title="lib/sources.ts"
import { defineApi } from "@noverax/courier/define"
import {
  authMiddleware,
  cacheMiddleware,
  logMiddleware,
  rateLimitMiddleware,
  retryMiddleware,
} from "@noverax/courier/middleware"

const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    logMiddleware(),                                    // 1. Log everything
    cacheMiddleware({ scope: "public", ttl: 30_000 }),  // 2. Return cached if available
    rateLimitMiddleware({ maxRequests: 10, window: 1000 }), // 3. Throttle outgoing
    retryMiddleware({ limit: 3, delay: 300 }),          // 4. Retry on failure
    authMiddleware({ getToken: () => getToken() }),     // 5. Attach token last
  ],
  groups: { /* ... */ },
})
```

## Next Steps

- [Custom Middleware](/docs/middleware/custom) -- write your own middleware
- [Middleware Overview](/docs/middleware/overview) -- the onion model and core types