The number everyone looked at was the load average. On one of our Macs, a 12-core machine with about eight agent sessions open, it was sometimes past 30. Load 30 on 12 cores reads like two and a half times more work than the machine can do, and for days that is exactly how we treated it.
The first fix watched the wrong number
On 29 October a second Mac joined our CI runner pool, and we wrote down rule 21: your own machine runs the tests for what you changed, and CI runs everything else. The next day an audit of our agent sessions went looking for why work felt slow. It found that 87 of 185 load readings in session transcripts were over 24, twice the core count. The worst was 410, on 29 October. One session had watched its gates time out and capped itself at three sessions running only delta tests.
So we built heavy. It is a small semaphore. Wrap a gate, a build or a browser suite in it and it takes one of two slots shared by every session on the machine, then waits until the 1-minute load average drops below the number of cores. If it has waited 15 minutes in total it gives up with exit code 75, and the work goes to CI instead.
The audit had raised the doubt itself. Its list of things it was unsure about ended with “memory pressure as a cause of load”. The load gate went in first anyway.
That was a reasonable first move and a wrong one. Load average is the number every guide tells you to read, and ours was not subtle: 87 of 185 readings over twice the core count, one at 410. Sessions were timing out. A guard that stops new heavy work from piling onto a struggling machine is not harmful, and it is still in place. The mistake was stopping at the first number that looked guilty, for days before anyone looked at idle CPU next to swap.
What the machine was actually doing
On 3 November at 09:15 we measured it properly, with eight sessions open.
The cores were not the limit. Memory was.
A Mac that fills its memory starts compressing pages, then writes them to disk, then reads them back when a process needs them again. Each read is a thread waiting for the disk. Load average counts work that is queued, not cores that are busy, and on a machine paging all day a lot of work is queued behind the disk. More cores would not have helped. A second machine that only added cores would have fixed the wrong thing.
The next question was who was holding all that memory.
What was holding the memory
Resident memory by owner, 3 November 2026 at 09:15, with eight agent sessions open.
Source: Investigation notes, 3 November 2026
None of it was CI. The CI runners on that Mac were idle at the time. The surprise was the 210 dev server processes, local Cloudflare Workers runtimes and their bundler, holding 2.3 GB between them. Sixty of the runtime processes were more than six days old, left running by two worktrees of one app that nobody was using any more.
That is a very agent-shaped leak. A person who starts a dev server tends to notice the terminal still open a week later. An agent session can start one in a worktree, finish its work and end, and the next session has no reason to go looking for it.
The fixes
heavy now watches memory as well. After the load check it waits while the kernel reports memory pressure at warning or worse, and by default it only runs at normal. Same 15-minute budget, same exit code 75, same answer: if the machine has no room, CI does the work.
The clean-up is a sweep of the stale dev servers and fewer browsers: one real Chrome instead of three kinds of browser, the desktop app’s built-in browser for agents, and Playwright kept headless. We have not run the sweep yet. Those processes belong to other sessions, and our rule is that one session does not kill another’s work without checking it is finished. It is a task with an owner.
The bigger fix is more room. We have a Windows PC, and we have set it up to be a Linux CI worker:
- WSL2 with Ubuntu 24.04 rather than native Windows runners, because our workflows are bash and Linux-first.
- Two GitHub runners under their own label, so a repo only sends jobs there once it opts in, and one command sends it back.
- The PC’s own use always wins. The Linux VM gets at most half the RAM and 60% of the cores, runs at below-normal Windows priority, and caps the runners at 7 GB inside it. Pause and resume buttons sit on the desktop, and GitHub re-queues any job whose runner disappears.
- No reach into Windows: no C: drive, no starting Windows programs, a non-admin user, and no deploy credentials. Deploys stay in CI on main.
It can take web gates, Playwright suites, Worker builds, lint and typechecks. Nothing that needs Xcode goes near it. As of today the setup scripts are written and syntax-checked, but they have not run on the PC itself. The first run needs a person at the keyboard, and we expect to fix a step or two.
Rule 21 ties it together. Local runs the delta and CI runs everything else. No job names a machine, only a label. Before a long local build, check the pool, and if your own machine’s runners are busy, push instead of competing with them. Once the PC has proved itself on one repo, the plan is to take the overloaded Mac down to a single runner.
Update, 7 November 2026
Neither half is finished. The Windows worker is still waiting for its first run on the PC, and the sweep of stale dev servers on that Mac is still an open task. The memory check in heavy has been merged since 3 November.
