This is the third piece of today’s fresh telemetry pass across Leaf, following our 40-billion-token rework audit and what 1,199 randomized planning rooms revealed about model routing.

Over the last ten days, I published two major changes to Leaf while the paint was barely dry:

  1. On November 4, in Pruning Agent Prompts, we cut 65% of the words in prompts/ (taking _base.md from 2,861 words down to 55) and deleted 900 lines of prose-pinning regex tests. My runtime evidence at the time was a 29-hour sample of 10 planning rooms and 9 build runs on Calamus. I called out three risks: BrandCast (my main digital signage monorepo) hadn’t run a batch yet, task shapes had shifted, and Tech Lead input tokens on 1-node runs had gone the wrong way (+29%). I promised to re-measure past 50 Development runs.
  2. On November 6, in Teaching My Dark Factory to Dream, we deleted 6,067 lines of noisy LLM “friction counting” in favor of eight deterministic SQL flags, a daily Dream consolidation pass, and a wiki health tracker that immediately tattled on us: 73% (108 of 148) of factory’s own auto-generated wiki pages had never been read once.

Eight days and more than a thousand runs later (4,044 total completed plan_runs in factory.db), the full-fleet numbers are in. Almost every bet paid off, and the one metric that looked broken taught me a great lesson about statistical traps in agent harnesses.

The nuance: how a 78% token spike turned out to be a win

When I re-ran that exact 1-node Tech Lead query tonight across 8 full days before the November 4 prompt cut (October 27 to November 4) versus 8 full days after (November 4 to November 12), my heart sank for about thirty seconds.

Across 122 post-cut 1-node Development runs (compared to 38 before), Tech Lead turns didn’t stabilize; they climbed 41% (14.0 to 19.8), raw input tokens more than doubled (500K to 1,032K), and Tech Lead IET jumped 78% (100.5K to 178.8K). Meanwhile, same-model Sonnet engineer wall time per pod on Development runs had also climbed (285.5s to 618.6s).

If I had stopped querying at the pod level, I would have rolled back the Tech Lead prompt tonight. Instead, I zoomed out one level to check whole plans and planning rooms across all 13 repositories:

Receipts scorecard from Oct 27 to Nov 12, 2026: Top table compares 8 days before vs. 8 days after the 65% prompt cut showing 655 planning rooms dropping 40% in IET and 28% in wall time, Development plans rising 35.5% (121 to 164) at 7.4% lower build IET per plan, nodes per plan collapsing from 4.05 to 1.54, and 1-node Dev diffs growing 2.86x larger (135 to 387 lines, 3.8 to 8.0 CI jobs). Bottom cards show Dream telemetry auto-fixing 107 of 118 systemic bugs (90.7%) with a 93.1% yield on retried pods, and wiki unread pages dropping from 73% to 34% on factory (81.4% read on brandcast).

Look at the whole board together:

  • Planning passed the BrandCast stress test (649 rooms before vs. 655 rooms after, including 258 BrandCast rooms): Average planning room wall time dropped 28% (471.8s down to 338.4s), raw input tokens per room dropped 47% (5.06M down to 2.69M), and planning IET per room fell 40% (882.2K down to 526.4K).
  • Feature throughput jumped +35.5% at lower token cost per feature: Completed human Development plans climbed from 121 (15.1/day) to 164 (20.5/day), while average build IET per shipped Development plan fell 7.4% (2.89M down to 2.68M) on top of the 40% planning savings.

How can per-pod engineer time and 1-node Tech Lead tokens go up while per-feature build cost goes down and throughput rises 35.5%?

Because we walked straight into Simpson’s paradox, the classic contingency-table trap Edward H. Simpson formalized in 1951 where subgroup trends reverse once you account for a shifting confounding variable. Yes. I had to look this one up. Hat tip to the new Gemini Carbon model for the great catch here.

Look at the fourth and fifth rows of the table: pruning 89% of the Architect prompt collapsed the average Development plan from 4.05 nodes down to 1.54 nodes.

Under our old 7,700-word prompt binder, Architects routinely chopped a coherent feature into four coupled micro-nodes just to look thorough. Back then, the only work that stayed a “1-node Development plan” (n = 38) was a tiny 134.9-line tweak in a small repo running 3.79 CI jobs (15.8% send-backs). After the cut, Architects stopped over-splitting work and kept whole features inside a single coherent node (n = 122), pushing the average “1-node run” to 386.5 changed lines (2.86x larger), 7.95 CI jobs (2.1x more, including BrandCast’s full 9-job suite), and 27.9% Tech Lead send-backs.

Before the cut, a 1-node plan was a typo fix.
After the cut, a 1-node plan was the whole feature.

Comparing “1-node runs” across that boundary was comparing bicycles to box trucks because both have two axles. And consolidating 4.05 nodes into 1.54 came with one genuine engineering trade-off worth naming honestly: average wall-clock time per Development plan rose from 18.99 minutes to 26.86 minutes (+41%). One engineer writing 387 lines serially takes longer on the wall clock than four parallel pods writing 100 lines each, even though it burns fewer total tokens and eliminates cross-node merge ceremony. With 60 pod slots in the closet, trading eight extra minutes of background wall time for 35.5% more shipped features per week is a deal I’ll take every day. I will say that there is a trade-off because it feels slower when I’m actively testing something. That’s new muscle memory I will be working on.

Dream telemetry at 1,806 flagged rooms (90.7% self-healed)

Six days after turning on the Analyst and Dreamer telemetry loops, the backlog of unanalyzed outlier rooms sits at zero.

Across 1,806 flagged rooms (932 read end-to-end by single-room Analysts and 874 turns-only outliers sampled out via our 1-in-5 rule) and 28 daily Dream consolidation passes:

  • 118 distinct systemic bugs were isolated, and 107 (90.7%) are already fixed and closed done. Across all 13 onboarded repositories tonight (158 all-time closed trajectory issues), only 1 trajectory bug remains open.
  • Two SQL flags turned out to be pure gold. Rooms where a pod had to be started more than once (retried) linked to a concrete root-cause bug 93.1% of the time (417 of 448), and rooms where a pod exited without a completion report (unfinished) yielded a bug 78.2% of the time (104 of 133). Even dead time (10-minute silent gaps), which sat at 0 of 19 in last week’s initial backfill, began earning its keep at 17.9% (10 of 56) once real suite checkout hangs appeared.
  • Timestamp recurrence tracking caught 18 regressions (15.3%). Whenever an Analyst or Dreamer links a new room to an already closed issue, Leaf checks whether the room closed after the fix landed and flips the cause family to Recurred after fix. That check caught two subtle platform bugs that refused to die on the first patch: factory#3102 (where a race killed a plan’s first Tech Lead pod seconds after boot) and factory#3279 (where CI suite pods intermittently stalled 2 to 5 minutes in host checkout across brandcast and calamus).

Without recurrence timestamps, both of those bugs would still be sitting in the “closed and celebrated” column while quietly taxing rooms every night.

How we got agents to actually read the wiki

The most embarrassing number in last Friday’s post was wiki readership: 73% (108 of 148) of factory’s auto-generated wiki pages had zero reads, even though Technical Writer and Lead Writer pods were burning millions of tokens keeping them tidy.

Our new rule from the prompt prune forbade adding a lecture to architect.md saying “Please remember to read the wiki!” If smart models ignore a document, the document isn’t useful enough yet.

Instead, as part of commit c2a39960, we audited the pages and made Factory’s deterministic code stamp a live <!-- factory:measured --> block onto the bottom of every repository area wiki page on startup, on wiki.write, and whenever scorecard.ts scores a merged node in that directory:

<!-- factory:measured -->
## Measured cost

| area | class | effort | level | n | minutes | tokens | sendbacks | before_edit |
| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
| src | 2 | medium | area | 255 | 0.5 | 66000 | 0.07 | 2 |
| src | 4 | high | area | 97 | 3.2 | 292000 | 0.24 | 4 |

Scoped checks: `npm test` median 104s (n=412)
<!-- /factory:measured -->

Every time a writer agent edits an area page, Factory strips any old block and writes fresh medians from SQLite (orientation tool calls before first edit across 10,430 unpacked trajectories, class/effort costs, and scoped test durations).

Six days later, look at /api/wiki on our two largest repositories:

  • BrandCast (118 pages): Unread pages dropped from 44% down to 18.6% (96 of 118 pages read, 81.4%), with 225 explicit wiki citations inside planning messages (233 writer jobs and 33 lead passes costing 158.9M cached input tokens and 561K output tokens).
  • Factory (150 pages): Unread pages dropped by more than half, from 73.0% down to 34.0% (51 unread), with 123 explicit planning citations.

We didn’t have to beg the PM or Architect to read the wiki. Once the wiki carried live operational facts that agents couldn’t get by grepping TypeScript source files, reading it became the fastest path to getting their plan approved.

So what?

Checking your own receipts a week after shipping harness changes is where the real engineering happens:

  • Watch out for Simpson’s paradox when plan shapes shift. Per-pod and “1-node run” averages lie when an upstream prompt change consolidates 4-node graphs into 1.5-node graphs. Always pair pod-level metrics with cost and throughput per shipped issue.
  • Track whether closed bugs stay dead. An autonomous self-healing loop that files and merges fixes looks magical right up until 15.3% of those causes quietly recur a few days later. Compare room close timestamps against issue close timestamps automatically.
  • Make documentation useful by injecting live telemetry, not prompt nagging. If planning agents skip your repo docs, don’t scold them in AGENTS.md. Stamp measured directory costs, send-back rates, and test runtimes directly into the pages so reading the docs gives the agent an immediate edge.