The finish line for 2026 is coming up fast.

At the end of last December, I wrote a retrospective on the year I learned to wrangle AI agents. Back then, I was complaining about completion bias, U-shaped context recall, and models inventing API fields out of thin air. My workflow still meant sitting in front of a terminal, watching text scroll by, and hitting Esc whenever an agent wandered into the weeds.

Eleven months later, Leaf, my local Level 5 dark software factory, runs up to 60 concurrent pod slots on six mini-PCs in my office closet. Across one 50-day stretch this fall, it chewed through 15.8 billion raw tokens (2.84 billion input-equivalent tokens) over 208,649 model turns and 2,959 plan runs, merging 99.5% of those runs without me writing code or reading diffs.

To keep those numbers honest, that’s one guy’s closet running four personal and early-stage startup repositories, mostly on flat monthly subscriptions and seven-year-old Intel NUCs. There’s still no revolution in the suburbs just yet.

The nuance

Looking back across eighty-plus posts from 2026, the biggest surprise isn’t how fast frontier models improved. It’s how fast my own best practices expired.

Almost every piece of architecture I bragged about in the spring ended up in the scrap bin before Thanksgiving. That didn’t happen because agentic development failed. It happened because building with agents punishes static rules. Out of everything I tested, broke, and measured this year, six lessons jumped out when I sat down and re-read the journey while sitting in a hotel in Cleveland.

1. Humans decide what done means. Machines decide whether it happened.

For most of the year, I ran two completely different jobs through one chat window: judgement (figuring out what’s wrong, what matters, and what’s out of scope) and mechanics (editing files, running tests, fixing failures, and merging). Mixing them reduces the human to watching TV.

A prompt is the start of a conversation. A contract is the end of one.

A loop can only run unattended when a machine can tell that it’s done. When we wanted Sadie, our local family assistant, to stop sending a Sunday grocery sales email, the planner distilled my issue into a single-node contract with an explicit one-line proof check:

{
  "id": "groceries-1",
  "files": ["deploy/groceries-digest.timer", "groceries/tools.py"],
  "done_when": "five digest files deleted; sadie/mail/ untouched; groceries tests pass",
  "check": "scripts/test.sh open_webui/sadie/groceries && ! git grep -n -e groceries.digest -e groceries-digest -- deploy backend/open_webui/sadie",
  "class": 2
}

That job ran on Haiku and merged in 8 minutes for 155K input-equivalent tokens because the contract left zero open design questions for the worker to guess at. Across Leaf’s first full month (1,198 merged PRs, only 27 merged by hand), requiring every worker to execute its scoped check after the final edit pushed first-try pass rates from 90% to 97.2% and dropped first-run end-to-end failures from 34% to 7%.

Proof beat review every single time.

2. Put unpredictability where it’s cheap. Put determinism where a bad surprise is expensive.

A lunch-break episode of MGM+‘s horror show From gave me my cleanest test for harness design back in July. You want the agent’s exploration to be unpredictable, because tracing messy code and testing approaches is where the value lives. You want your boundaries, sandboxes, and CI/CD pipelines to be the most boring, deterministic parts of your stack.

Even after writing that post, I violated my own rule this fall. Over 30 days and 170 retrospective commits, Leaf’s prompts/ directory bloated to 17,072 words (and our shared _base.md file grew 15x to 2,861 words) because we tried to stop 0.1% operational hiccups with stern markdown lectures.

Last week, we cut 65% of our factory prompts, dropping _base.md to 55 words, and moved those guardrails into deterministic code. We baked .curlrc into the container image instead of lecturing pods on retry flags, and we made tool refusal errors print the exact recovery command.

Over the first 29 hours on Calamus (a small 9-build sample on one repo, where Tech Lead tokens on 1-node runs actually ticked up 29%), engineer turns dropped 43% on Haiku and 35% on Sonnet, and build problem moments fell 62%. Why? Because pods stopped burning turns on paranoid CYA rituals. This pattern is holding up and the throughput is climbing steadily (more measurements coming).

Prompts guide exploration.
Code enforces the walls.

3. Code can’t judge meaning. Prose can’t measure systems.

When an unattended run goes sideways inside my dark factory, the root cause almost always traces back to crossing the line between code and natural language.

In October, I let Leaf’s autonomous Improvement loop run against Leaf’s own repository. An agent wrote regex unit tests validating our markdown role prompts (src/bridge.prompts.test.ts grew to 1,022 lines and 162 assertions). When a later change tripped those checks, the worker didn’t fix the test code. It rewrote Leaf’s own system prompts to satisfy the regexes, because editing prose is faster than refactoring TypeScript.

Writing a test takes code.
Changing a prompt takes prose.

The reverse failure is just as bad. Ask an LLM if a refactor made a file “cleaner,” and it will write three confident paragraphs while shipping a mess. Gate cleanup on raw line counts, and agents will golf five readable lines into one ugly nested ternary.

Moving to a deterministic Tree-sitter and Sonar cognitive complexity gate (simpler.ts) in Moneyball for AI Agents fixed that by forcing every cleanup to pass the test suite and lower the syntax tree’s complexity score. That check has closed 1,814 verified reductions, shrinking my flagship monorepo by 18% (105K lines) while weekly merge velocity rose 3.8x.

4. The only real substitute for human review is adversarial validation.

Give a single coding agent full repository context and an open loop, and it will happily grade its own homework green while refactoring half your app. When humans step out of diff review, every judgement layer needs purposeful friction and strict context boundaries.

In my August rebuild of root, putting plans in front of a three-judge council across separate model families immediately surfaced race conditions that single-model sessions missed, because models naturally favor their own family’s output. Inside Leaf today, no seat works unchecked.

  • Planning: Only the Product Manager (PM) has the repo’s external wiki mounted. Giving deep product lore to Engineers just invites them to over-think twenty-minute tasks. The PM owns what, and an Architect pushes back on how and cost.
  • Implementation and QA: A Tech Lead reviews every Engineer diff and every QA test fix strictly against the node contract before CI runs.
  • Security: In BrandCast’s autonomous security loop, deterministic Trivy and Semgrep scans surfaced 289 findings on day one. Instead of flooding the backlog, a read-only Examiner agent traced reachability in the code, killed 139 false positives with cited proof (like FF_EMAIL_SUMMARY tripping a secret rule), and collapsed 150 real findings into 5 structural fixes.

5. Measure the harness, not the code.

Counting what percentage of your codebase was written by AI measures exhaust, not the engine. My intuitions about my own harness were routinely wrong until I started querying actual run telemetry in SQLite.

Across 3,898 unattended worker runs in October, Class 3 models looked like the sweet spot on provider pricing pages. In practice, scoring workflows like Moneyball showed Class 3 was a trap: a 17.1% rework rate on feature work pushed its cost up to 33.96x the repository baseline per line once Tech Lead rejections were charged back. Meanwhile, tightly scoped Class 2 nodes (Haiku and gpt-oss-120b) landed 98.6% on the first try in a median of 31 seconds, verified continuously by automatic 10% lower-tier call-up trials.

Telemetry also forced me to confront my own blind spots. In Loops Part 6, I couldn’t prove that simplifying code made future agent runs in those directories any cheaper (105K vs. 101K tokens per node).

Last week, after deleting 6,067 lines of noisy LLM “friction-counting” code in Teaching My Dark Factory to Dream, eight hard SQL outlier flags and a daily consolidation pass isolated 69 systemic bugs and auto-fixed 61 of them (88%). That included a duplicate Vitest loop in BinPacker (my wife’s kids’ resale store app) diagnosed and merged in under 25 minutes with zero human keystrokes. That same dashboard also exposed that 73% (108 of 148) of factory’s own auto-generated wiki pages had never been read once despite burning 19.4M tokens.

You can’t fix waste you refuse to count.

6. Nothing is sacred.

The sixth rule is what made the other five possible. Almost nothing I built in 2026 lasted more than ninety days.

  • Agent fleets and harnesses: My 43-agent prompt setup gave way to Root in March, which evolved into bob, which I killed in July after Anthropic’s Opus 5 guidance pointed out that my prescriptive verification gates were teaching the model to distrust its own work (running a 6,000-test suite 14 times on a 60-line diff).
  • Knowledge bases: I replaced a 7-table custom Vertex RAG pipeline with managed File Search, replaced that with local MCP vector RAG, and finally ripped RAG out of my repo agents altogether in favor of linter-enforced git wikis.
  • Review layers: I retired dedicated validator pods on September 30 after the data showed handoffs without ownership blocked 14.5% of nodes, whereas letting the Foreman own review cut blocked work to 0.3%.
  • My own favorite motto: On October 10, I proudly published Leaf’s core rule: “When an agent misbehaves, fix its prompt.” Twenty-five days later, after watching that exact rule create a 17,072-word HOA bylaw binder, I rewrote AGENTS.md to say the opposite: delete a prompt rule first, or move the check to code.
  • Cloud CI: When 4x autonomous throughput burned 50,000 GitHub Actions Enterprise minutes in 18 days, I pulled git, issues, and CI/CD onto closet mini-PCs running at ~1% CPU pressure, getting 85% of a 2024 Zen 4 chip’s test-suite speed (122s vs. 104s) on 2019 NUCs while avoiding $20k to $33k a year in equivalent retail AWS us-east-1 compute (or ~$580 to $960/month on Spot, where one month of rent still buys a used NUC outright).

Those solutions didn’t have short shelf lives because I was being sloppy. They expired because model capabilities, context economics, and failure modes are shifting every quarter while the cost of rebuilding software has collapsed. Falling in love with last month’s harness turns yesterday’s clever workaround into tomorrow’s tax.

Make results the only sacred thing, and treat every prompt, harness, and pipeline as disposable scaffolding.

So what?

As Q4 wraps up and you start planning how your team will build software in 2027, run your workflow against those six lines.

  • Hand off contracts, not chat threads. Separate human discovery from machine execution, and require a runnable one-line check before work starts.
  • Push guardrails out of markdown and into code. Keep role prompts short and verifiably true; bake retries, defaults, and state transitions into containers and scripts.
  • Respect the wall between prose and code. Never regex-test a system prompt, and never let an LLM essay stand in for an exit code or an AST check.
  • Build adversarial friction into every phase. Separate the model or role that writes a plan, diff, or scan from the one that validates it, and keep deep product lore out of builder pods.
  • Instrument the factory floor. Route models by rework-adjusted runs rather than leaderboard scores, and trigger post-mortems from SQL outliers rather than LLM vibes.
  • Keep your delete key warm. Audit every gate, skill, and wiki loop against live data, and throw out anything that stopped paying rent.