Built-in Middleware

Courier ships six production-ready middleware functions. Import them from @noverax/courier/middleware:

import {
  retryMiddleware,
  authMiddleware,
  logMiddleware,
  rateLimitMiddleware,
  cacheMiddleware,
  headerVariablesMiddleware,
} from "@noverax/courier/middleware"

retryMiddleware

Automatic retry with exponential backoff. By default it retries server errors (status >= 500) and requests that got no answer at all — a network failure or a timeout. A 4xx is never retried by default: the same request would be refused again.

declare function retryMiddleware(options?: RetryOptions): Middleware

interface RetryOptions {
  /** How many times a failed attempt is made again (default: 3) */
  limit?: number
  /** The wait before the first retry, in ms (default: 300) */
  delay?: number
  /** How the wait grows between retries (default: "exponential") */
  backoff?: "fixed" | "exponential"
  /** Which answers deserve another attempt: statuses, or a test (default: 0 and >= 500) */
  retryOn?: readonly number[] | ((response: MiddlewareResponse) => boolean)
  /** Retry endpoints that write as well as those that read (default: only a write with an idempotency key) */
  mutations?: boolean
}

With backoff: "exponential" the wait doubles from delay — with the default 300 ms, the waits are 300 ms, 600 ms, 1200 ms; with "fixed" every wait is delay. An endpoint's .retry() and a courier() base's retry take the same RetryOptions.

const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    retryMiddleware({ limit: 3, delay: 300 }),
  ],
  groups: { /* ... */ },
})

Custom retry condition

Give retryOn the statuses to retry, 0 standing for a request that got no answer, or a test of the answer:

retryMiddleware({ limit: 2, delay: 500, retryOn: [0, 429, 502, 503, 504] })

retryMiddleware({
  retryOn: (response) => response.status === 429 && response.headers.has("retry-after"),
})

Writes are sent once — unless they carry a key

An endpoint that writes — a post, put, patch or del not declared a read with .reads(), or a .via() endpoint, which counts as a write until .reads() says otherwise — is not retried by default. A write whose answer was lost may already have been carried out, and sending it again can charge a card twice.

An endpoint that declares an idempotencyKey is retried: each call sends one key in that header, the same on every attempt, and an upstream that deduplicates by it applies the write once however many times it arrives.

place: endpoint
  .post("orders", { idempotencyKey: "header:Idempotency-Key" })
  .retry({ limit: 3 }) // retried: the key makes a repeated write safe

mutations: true retries every write, for an upstream that recognises a repeated one some other way; mutations: false retries none, keys included:

retryMiddleware({ limit: 3, mutations: true })

A retry policy on a courier() base or an endpoint's .retry() follows the same rule and takes the same mutations flag. Middleware reads a call's key as ctx.idempotencyKey, and Studio lists it with the call, and where it stands among the calls sent with the same key.

Cancellations

A cancellation is never retried, whatever retryOn says: not a call whose signal the caller aborted, whatever reason it gave, and not a request that deduplicate replaced with a newer identical one (status 499). A caller that aborts while the middleware waits between attempts is answered at once with its signal's reason.

authMiddleware

JWT/OAuth token injection with automatic refresh on 401.

declare function authMiddleware(options: {
  /** Function to get the current auth token */
  getToken: () => string | Promise<string>
  /** Optional function to refresh the token on 401 */
  refreshToken?: () => string | Promise<string>
  /** Header name (default: "Authorization") */
  header?: string
  /** Auth scheme prefix (default: "Bearer") */
  scheme?: string
}): Middleware

Basic usage

const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    authMiddleware({
      getToken: () => localStorage.getItem("access_token")!,
    }),
  ],
  groups: { /* ... */ },
})

With token refresh

When refreshToken is provided and the upstream answers 401, the middleware calls refreshToken(), updates the header, and sends the request again once. If the new token is refused too, the call rejects with that second 401. Requests refused while a refresh is running wait for it instead of starting their own, so refreshToken() runs once for all of them:

authMiddleware({
  getToken: () => tokenStore.accessToken,
  refreshToken: async () => {
    const { accessToken } = await refreshAccessToken(tokenStore.refreshToken)
    tokenStore.accessToken = accessToken
    return accessToken
  },
})

API key authentication

Set scheme to an empty string and use a custom header for non-Bearer auth:

authMiddleware({
  getToken: () => process.env.API_KEY!,
  header: "X-API-Key",
  scheme: "",
})

logMiddleware

Request/response logging with timing information.

declare function logMiddleware(options?: {
  /** Receives each line, and the error of a failed request (default: Courier's logger) */
  logger?: (message: string, error?: unknown) => void
}): Middleware

Logs each request as source:group:method → status (duration). On errors, logs source:group:method → ERROR (duration) and passes the error along. By default the lines go to Courier's logger: a success at info on stdout, a failure at warn on stderr with the error's stack.

const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    logMiddleware(),
  ],
  groups: { /* ... */ },
})

// Output:
// api:users:getAll → 200 (42ms)
// api:users:create → 201 (89ms)
// api:auth:login → ERROR (1203ms)

Custom logger

Send logs to your observability platform:

import * as Sentry from "@sentry/node"

logMiddleware({
  logger: (message) => {
    Sentry.addBreadcrumb({ category: "api", message })
  },
})

rateLimitMiddleware

Client-side rate limiting. Delays requests that exceed the configured limit within a time window.

declare function rateLimitMiddleware(options: {
  /** Maximum number of requests in the window */
  maxRequests: number
  /** Time window in milliseconds */
  window: number
}): Middleware

const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    rateLimitMiddleware({ maxRequests: 10, window: 1000 }),
  ],
  groups: { /* ... */ },
})

The limit holds however many calls arrive at once: at most maxRequests requests go out in any window milliseconds, and calls beyond it wait their turn in the order they came. The window is measured from when each request actually went, so a busy event loop that runs the limiter's timers late delays the queue rather than letting several requests out together. A caller that aborts its signal leaves the queue at once and rejects with the signal's reason; its turn passes to the call queued behind it.

cacheMiddleware

In-memory response caching with configurable TTL.

declare function cacheMiddleware(options: {
  /** How responses are partitioned between callers: "public", "session", or a function naming the caller */
  scope: "public" | "session" | ((ctx: MiddlewareContext) => string)
  /** Time-to-live in milliseconds (default: 60000) */
  ttl?: number
  /** Entries retained before the oldest are dropped (default: 1000) */
  maxEntries?: number
  /** Custom storage; a Map satisfies it (default: in-memory, per middleware instance) */
  storage?: CacheStore
}): Middleware

Caches successful responses (2xx status) of endpoints that read, keyed by the scope's partition, source:group:method and the variables. Non-2xx responses are never cached, nor the answers to a request the session makes for itself, such as the double-submit token a write asks for: it is made for the cookie its answer sets.

const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    cacheMiddleware({ scope: "public", ttl: 30_000 }), // 30 seconds
  ],
  groups: { /* ... */ },
})

Every caller receives a copy of its own, made with structuredClone, so editing a result never changes what the next caller gets. An answer holding a value that cannot be copied, such as a function, is not cached; class instances come back as plain objects.

A call that carries the user's session is never answered from a "public" cache, nor stored in it: its answer belongs to that user, and the endpoint is named once in the log. Give such an API's per-user reads a cache of their own with scope: "session" — one partition per set of credentials the call sends, its Cookie and its Authorization, named by their SHA-256 digest so the credentials never sit in a store's keys:

cacheMiddleware({ scope: "session", ttl: 10_000 })

In a browser or a React Native app, the platform sends the session's cookies itself, out of a script's reach. There, every call of an API with a session counts as carrying it, so a "public" cache keeps none of its answers, and scope: "session" partitions by what a script can read — a bearer token, the double-submit token an API renews with the session — and starts a new partition with every write made with the session, as it starts and once it is answered. A user who signs out and another who signs in on the same device, in the same tab or another, never share a cached answer, nor see one read while the sign-in was under way.

Custom storage

Provide your own Map instance to control the cache externally (e.g., for manual invalidation):

const apiCache = new Map()

const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    cacheMiddleware({ scope: "public", ttl: 60_000, storage: apiCache }),
  ],
  groups: { /* ... */ },
})

// Invalidate the entire cache
function invalidateCache() {
  apiCache.clear()
}

headerVariablesMiddleware

Sends some of a call's variables as request headers instead of in its path, query or body: an API's header parameters, such as a digest or a version the caller picks per call. Generated clients use it for the header parameters an OpenAPI document declares.

declare function headerVariablesMiddleware(names: readonly string[]): Middleware

import { createCaller } from "@noverax/courier"
import { courier, defineApi } from "@noverax/courier/define"
import { headerVariablesMiddleware } from "@noverax/courier/middleware"

const { endpoint } = courier()

const api = defineApi({
  baseURL: "https://api.example.com",
  endpoints: {
    find: endpoint
      .get("things")
      .variables<{ q: string; "X-Digest"?: string }>()
      .use(headerVariablesMiddleware(["X-Digest"])),
  },
})

// GET /things?q=red, with the header X-Digest: sha-256=abc
await createCaller(api).find({ q: "red", "X-Digest": "sha-256=abc" })

The variables are validated whole, then the named ones are taken out and set as headers under their own names; one left unset sends no header. A list is written comma-separated and an object as its keys and values in turn, as OpenAPI's simple style writes a header. For a call through the proxy it runs in your application's server, so the values travel with the variables and the browser sets no such header itself.

Composing Built-in Middleware

Order matters. Place middleware in the array from outermost to innermost:

import { defineApi } from "@noverax/courier/define"
import {
  authMiddleware,
  cacheMiddleware,
  logMiddleware,
  rateLimitMiddleware,
  retryMiddleware,
} from "@noverax/courier/middleware"

const api = defineApi({
  baseURL: "https://api.example.com",
  middleware: [
    logMiddleware(),                                    // 1. Log everything
    cacheMiddleware({ scope: "public", ttl: 30_000 }),  // 2. Return cached if available
    rateLimitMiddleware({ maxRequests: 10, window: 1000 }), // 3. Throttle outgoing
    retryMiddleware({ limit: 3, delay: 300 }),          // 4. Retry on failure
    authMiddleware({ getToken: () => getToken() }),     // 5. Attach token last
  ],
  groups: { /* ... */ },
})

Next Steps

• Custom Middleware -- write your own middleware
• Middleware Overview -- the onion model and core types