I'd rather read the prompt

Last month I read a design doc that was four pages long and contained, as far as I could tell, one decision. It had an overview, goals and non-goals, a section on alternatives with three options and a balanced paragraph about each, risks, mitigations, and a conclusion that restated the overview. All of it was grammatical, and almost none of it was written by the engineer whose name was on it. I knew because I asked, and to his credit he told me straight away. He’d given a model a few lines describing what he wanted to do and asked it to write the doc.

I asked him to send me the few lines. They were better than the doc. They had the decision, the one constraint that made it hard, and a question he wasn’t sure about, which the four pages had smoothed into a sentence under “Risks” that didn’t ask anything.

So Clayton Ramsey’s I’d rather read the prompt, from the start of the month, landed with me. He grades student assignments, and he’s describing answers that are obviously generated: long, confident, padded, and empty. His argument is that a model has no thoughts of its own about your problem, so whatever was worth reading was in the prompt, and the output is the prompt plus filler. “If it’s not worth writing, it’s not worth reading.” His closing section is headed “Just show me the prompt”, and that’s more or less what I ended up saying to my engineer.

I don’t fully agree with him, for what it’s worth. He’s against using these tools for writing at all, and I use them every day on writing, mostly to tell me what’s unclear in something I’ve already written, and occasionally to turn an angry email into a polite one. I’d also say a model is quite good at the parts of a doc that really are boilerplate, like a rollout checklist, where nobody wanted your original thoughts anyway. Where I agree with him completely is the part that matters, which is that the reason you write a design doc is to find out what you think. The docs people actually read are short, they lead with the decision, and they’re written by someone who had to work out what the decision was by writing it down. Generating the doc skips the part that was the point, and leaves the reader with more to read and less to think about.

It’s worse for the reader than it was for the writer, too. The writer saved an hour. Six people on the review each spent fifteen minutes reading four pages to find one decision, and some of them probably didn’t find it.

So we’ve added two lines to the design doc template. “Write this yourself. If a model helped, say how.” And for status updates, which had started to get the same treatment, I’ve asked people to send bullet points. I’d rather read three honest lines with a typo in them than a paragraph of something that sounds like a press release about our sprint.