Design docs people actually read

A couple of years ago I went through every design doc in our wiki from the previous twelve months and counted the comments. The median was two. One of the two was usually “looks good”, and the other was usually the author replying to it. A few docs had a real discussion in them, and when I looked closer, nearly all of those were docs where the author had gone and asked specific people to read specific sections.

That told me most of what I needed to know. The docs weren’t bad, mostly. They were long, they arrived after the decision had more or less been made, and nobody had a reason to read them by any particular time. This is what we do now. It’s been about eighteen months, and I’m writing it down because three different people have asked me for “the template” this year, and the template is the less important half.

The front page

Every doc starts with a section that fits on one screen. It says what the problem is in two or three sentences, what we’re proposing, what we’re explicitly not doing, what decision we need, from whom, and by when. That last part is the one that changed the most. “Approve the move to a queue per tenant, from the platform lead and me, by Thursday” is a request somebody can act on. “Feedback welcome” isn’t, and for years it was how nearly every one of our docs ended.

The front page is also the only part a lot of people will read, and that’s fine. Someone in support or sales who wants to know whether a change will affect customers should get that from the first screen and then go back to their day.

The rest

After the front page, the template has five headings: context, the proposal, alternatives we considered, how we’ll roll it out and back out, and what it costs. We used to have fourteen. People filled most of them with “N/A”, and a template that’s mostly N/A teaches people to skim the template.

Alternatives has one rule, which is that “do nothing” has to be one of them, written up as honestly as the proposal. About one doc in five, writing that section is how the author finds out that doing nothing is fine for another year.

The cost section is new this year. We had a weekend in January where a reasonable-looking change to a batch job ran up a five-figure AWS bill, and the thing that stayed with me afterwards was that two people reviewed the change and nobody looked at what it would cost. So now every doc says roughly what the design will cost to run per month, at today’s volume and at ten times today’s volume, even if the answer is “about the same as now”. Most of the time it’s a sentence. Twice it’s changed the design.

There’s a soft length limit of six pages, not counting appendices. When a doc goes past it, it’s almost always two decisions wearing one title, and splitting it in two makes both easier to review.

The review

The doc goes out at least two working days before the review. Everyone listed as a reviewer is expected to leave at least one comment, even if it’s “read it, no concerns”, so the author knows who actually read it. Comments go in the doc, not in chat, because a reply in chat disappears and the next person to read the doc never sees it.

The review meeting is thirty minutes, and the first ten are silent reading. Amazon is famous for doing this with its six-page memos, and I resisted it for a long time because it felt like a waste of a meeting. It isn’t. Even with two days’ notice, a third of the room hasn’t read it properly, and ten minutes of everyone reading the same thing at the same time, with comments going in as they go, gets the room to the same place faster than anything else I’ve tried. The author doesn’t present. If the doc needs a presentation to make sense, the doc needs fixing.

The other twenty minutes are only for the comments that disagree with each other or with the proposal. We don’t talk through the parts everyone agrees with. And the meeting ends with one of three things written at the top of the front page, with a date: decided, and what; not decided, and what we need to find out first; or dropped, and why. That line is the part I’d keep if I had to throw away everything else here. A year from now, someone is going to find this doc while trying to understand why the system works the way it does, and the first thing they’ll need is whether any of it actually happened. I wrote last year that the why is the part that’s gone in two years, and a design doc that ends with a decision line is the cheapest way I know of keeping it.

When to write one

Not for everything. We tried “every project gets a doc” and got docs written after the code to tick a box. The rule now is a doc for anything that’s more than a couple of weeks of one person’s time, anything that changes a data model or a public API, and anything that would be expensive to undo. The last one does most of the work. Plenty of big projects are easy to reverse, and some one-line changes aren’t.

Being remote since 2020 made all of this matter more. When we were in an office, a lot of design happened at a whiteboard, and the doc, if there was one, was a record of the whiteboard. Now the doc is where it happens, and if it’s unreadable, it doesn’t happen anywhere. Most weeks I’d guess I spend more time reading other people’s docs than writing anything of my own, and I’ve stopped thinking of that as time away from the real work.

The median comment count on last quarter’s docs was twelve, for what it’s worth. I counted again in July.