Skip to content

AdvancedPerformance at Scale

Performance at Scale

How Courier, Studio, its mocks and the VS Code extension behave on a large company's project — five APIs with a thousand endpoints, the hundreds of files that call them — every figure measured every night and held to a budget, and how to run the measurements on your own machine.

7 min readboth modes

The project they are measured on

Every night Courier writes a project the size of a large company's and measures everything it runs on it. The project is the same every time, generated from a fixed seed:

PartWhat it holds
APIs5 sources, 1000 endpoints: 970 HTTP endpoints in 169 groups, a group inside every fourth one, 25 streams and 5 channels
Schemaseach group's records, input and patch in zod, with nested objects, arrays, unions and a discriminated union
Code1420 files: 600 server modules calling the APIs directly and 100 pages calling them through the app's proxy — 3300 call sites — and 39 internal libraries that import each other
Couriercourier.config.ts, the app's server with createCourierServer, two boundaries, and courier.mocks.ts with a mock for each API, 200 records a collection

The figures below were measured on an Apple M2 Max laptop with Node 24. The nightly job runs on a CI machine, and its budgets leave room for one about three times slower: they catch a change that makes anything markedly slower or heavier, not a machine that is slower.

The figures

Types and the client

MeasuredFigure
tsc --noEmit over the whole project2.4 s, 476 MB, 135,000 types, no errors
Defining the 1000 endpoints and creating the callers132 ms, 16 MB of heap
A call through the client, beyond fetch and JSON.parse themselves24 µs at the median, 72 µs at p99
Importing the project's OpenAPI document — 970 operations, 941 KB — in Studioread in 42 ms, 341 files of code written in 186 ms; tsc over them 2.5 s, no errors

Large answers

An endpoint answering a list read through its zod schema, with Studio attached and without. Studio is sent at most 16 KB of any body — the first items of a list, a part of each large field — whatever its size; it asks the app for the whole one when you open it.

AnswerWithout StudioWith Studio attachedSent to Studio
1,000 records, 0.3 MB2.6 ms4.6 ms18 KB
10,000 records, 3.5 MB16 ms28 ms18 KB
50,000 records, 17.6 MB93 ms154 ms18 KB

Studio and its mocks

MeasuredFigure
Studio answering, from its start1.6 s on a project it has never loaded, 0.9 s after
The five mocks listening, from Studio's start2.2 s
Memory while they start1.4 GB, for about ten seconds
Memory at rest — Studio and five mock processes640 MB: Studio about 180 MB, each mock 110–140 MB
Processor at rest0.5 % of one core
The mocks under 32 concurrent connections8,000 requests a second, 3.3 ms at the median, 16 ms at p99, none failed

Each mock runs in a process of its own, so one that crashes or hangs leaves the others serving, and the mocks share the machine's cores under load. Each process holds Node itself and everything courier.mocks.ts loads — every API's definitions, the seeded records — which is what most of its memory is.

The app's server with Studio attached

The app's own server — Courier's proxy in front of the five APIs, with its middleware and response schemas — called by its pages, 32 calls at a time:

Calls a secondMedianp99
Straight to the backend, for comparison8,0003.2 ms12.8 ms
Through the app's server5,7005.1 ms12.3 ms
Through the app's server, attached to Studio3,8008.1 ms13.8 ms

A server attached to Studio reports some seventeen events for each call it serves — the call's start and end, each middleware, each validation, the fetch, the answer's shape — and sends them in batches: every event of one turn of its event loop leaves together, in one message, once the turn is over. Only development servers attach; in production nothing is reported.

The editor and VS Code

What an editor asks Studio, with 2000 requests in its log:

AskedAnswered
Every endpoint's figures, which the extension asks for every 5 s4 ms
The latest requests1 ms
The audit's findings at the calls2 ms; 163 ms when the requests changed since
The calls in one fileunder 1 ms
Where each endpoint is declared3 ms; 166 ms the first time

Courier for VS Code over the same project, inside VS Code:

MeasuredFigure
Hints over the first file opened, the extension's first questions to Studio included288 ms
Hints over each next file75 ms at the median
Going from a call to the endpoint's declaration6 ms
The extension host's heap, every extension in it61 MB

Studio's pages

Studio's pages in Chromium, with an attached app's traffic streaming in — about 900 calls a second, some 24 MB of events a second reaching the page:

MeasuredFigure
The page loaded215 ms
The Timeline showing the 2000 requests Studio kept0.6 s
The Explorer's graph of 1000 endpoints215 ms
Tasks holding the page's main thread over 50 ms, Timeline or Explorer open, while traffic streams innone
The page's JavaScript heap140 MB
What the page is sent of the five mocks as it opens: their routes, scenarios and each collection's count0.9 MB
The Store tab of a mock, its 3.4 MB of records asked for as the Mocks section opened70 ms

Running them yourself

The measurements are a script of the repository. It writes the project to the system's temp folder, starts Studio, its mocks, the app's server, Chromium and VS Code, measures them, and stops each one. It prints a line a figure with its budget, and exits with 1 when one is over:

bash
npm run test:scale

A larger company, some of the steps alone, or the report as JSON:

bash
node scripts/scale/run.mjs --endpoints 3000 --only tsc,runtime,studio,proxy --json scale-report.json

The steps are tsc, runtime, codegen, payloads, studio, and inside it proxy, editor, extension, mocks and ui. Studio listens on port 7401, its mocks from 7410, the app on 7400; --ports moves all of them.

On a project of your own of this size

  • Offer agents what they need. The project marks every endpoint for agents, which makes 970 tools and 2 MB of their descriptions — far past the 40 where some clients start leaving tools out. The MCP server lists them in 30 ms; what to offer is the project's agents policy.
  • Studio keeps the latest 2000 requests and sends a page that opens as many of them as fit 16 MB. Clear the log from the Timeline to start afresh.
  • Each mock is a process of its own. One you do not need running still takes its memory; stop it from the Mocks page.
  • A mock's records reach the page that shows them. A page is told each collection's count as it opens; the Mocks section asks Studio for the records of the mock it shows, from the moment its link is pointed at, and keeps them current from then on. A page that never opens it is never sent them.
  • Transforms are kept in node_modules/.cache/courier. Studio, an attached app, the MCP server and courier-server reuse each module's transform until its bytes change, so only the first start pays for them; COURIER_TRANSFORM_CACHE=0 turns that off.