Writing specs for agents

I counted last week, and I wrote more words of spec than words of code by a factor of about six. I’m not sure how I feel about that, but it’s where the job has gone, so I’ve been paying more attention to what a good one looks like.

When I started writing specs for agents last year, I wrote them like small requirements docs, the kind of thing I’d have handed a contractor: background, what we want, some notes on approach, acceptance criteria at the bottom. They worked, mostly. A year and a lot of agent sessions later, my specs are shorter, and the sections are in a different order, and the background section is nearly gone.

What a requirements doc assumed

A requirements doc for people was written for a reader who already knew a lot. They’d been in the meetings, they knew which customer was angry, they knew that the billing code is the scary part and the admin pages aren’t. The doc only had to say the new thing. Everything else came from the hallway, or from asking.

An agent has none of the hallway and very little of the asking. It does have unlimited patience for reading code, so it doesn’t need a background section explaining how the system works. It’ll go and look, and its picture of the code will be more accurate than my summary would be. What it can’t get by reading is everything that was decided somewhere other than the code: which of two reasonable approaches we picked and why, what we’re deliberately not doing, and which parts I’m nervous about.

So that’s what the spec is now. Here’s a real one from last week, lightly trimmed:

# Move invoice PDF rendering off the request path

Goal: generating an invoice PDF no longer blocks the web request.

Decided: a background job renders the PDF and stores it; the page
polls. NOT streaming, NOT a separate service. We talked about both.

Out of scope: the PDF layout itself, the email that attaches it,
the admin "regenerate" button (separate ticket).

Done means:
- existing invoice specs pass, plus new ones for the job
- `bin/check` is clean
- a PDF for the seed account with 400 line items renders in the job

Stop and ask if: you need to change the invoices table, or touch
anything under app/billing/ other than the controller.

The “Decided” line is the one that matters most, and I learned that the hard way. Without it, a capable agent will often pick a different reasonable design from the one we agreed on, and do it well, and then I’m reviewing a good implementation of the wrong thing. I wrote about this in the Kiro post as a problem with generated specs, the one decision buried somewhere in the middle. It’s just as much a problem with hand-written ones if the decision isn’t stated as a decision.

“Out of scope” does a lot of work for three short lines. Agents are helpful in a way that’s mostly good and occasionally expensive, and if they see the email template while they’re in there, they’ll tidy it up.

“Done means” has to be things the agent can check itself. “The page feels faster” isn’t. A command that passes is, and so is a specific case that renders. If I can’t write that section, I don’t understand the task yet, which is useful to find out before the agent starts and not after.

“Stop and ask” is how I put the autonomy table into a single task. It names the parts where I’d rather lose twenty minutes waiting for the agent to ask me than find a migration in the diff.

The step before the spec

For anything bigger than this one, I don’t start with the spec at all. I start by asking the agent to read the relevant code and come back with proposals, numbered, each with a sentence of reasoning, and to change nothing yet. A typical list has eight or ten items. Some are obviously right, one or two are things I hadn’t thought of, and a couple are plausible and wrong, usually because of something that was decided outside the code.

Then I pick by number. “Do 1, 2, 4 and 7. Not 3, we tried that in 2023 and the queue backs up. 5 later, separate PR.” That reply, with the reasons, is most of the spec. The agent writes it up in the format above, I edit it, and only then does anything change.

I started doing this on my personal projects and it’s spread to about half the team, which I didn’t push. It works because reviewing a list of ten one-line proposals takes me three minutes, and reviewing a diff that made all ten decisions for me takes thirty, and by then the wrong ones are tangled up with the right ones.

What I’ve stopped writing

Implementation notes, mostly. I used to write “use the existing job framework” or “follow the pattern in the exports module”. The agent finds those by itself now, nearly every time, and when it doesn’t, it’s a cheaper fix in review than a paragraph I write on every spec just in case.

Long acceptance criteria in prose, too. If I find myself writing a paragraph about what correct behavior looks like, I write a test for it instead, or I put the case in “Done means” and let the agent write the test.

And the background section, which I wrote for every requirements doc for fifteen years. It was always more for me than for the reader. Writing it was how I found out whether I understood the problem, and I still do that, but in a scratch file now, and the agent never sees it.