Do things that don't scale, an engineering reading

Paul Graham’s latest essay, Do Things that Don’t Scale, is aimed at founders. Startups don’t take off by themselves, so in the beginning founders have to do things by hand that look ridiculous at scale. The Airbnb guys went door to door in New York photographing apartments. The Collison brothers would say “right then, give me your laptop” and set people up with Stripe on the spot. Wufoo sent handwritten thank-you notes.

I think the same essay with the nouns swapped is some of the best engineering advice I’ve read this year. A lot of the engineering mistakes I’ve made, and watched other teams make, came from automating something before we understood it.

The urge to automate is a good instinct aimed at the wrong moment. The first time you do a task you don’t actually know what it involves, which steps matter, which are habit, which ones fail and how. My rule of thumb now is to do it by hand three times and write down each step as I go. The first time you learn what the task is, the second time you see what changes and what doesn’t, and by the third time your notes basically are the spec for the automation, and they’re better than anything you’d have designed up front.

Our first dozen production deploys were someone following a text file of commands. That felt embarrassing at the time. But when we finally wrote the deploy script we knew exactly where things went wrong: the migration that had to run before the restart, the cache that needed warming, the one box with a slightly different config that nobody remembered until it broke. The script handled all of it from day one because we’d already tripped over every one of them by hand.

Same thing with customer data imports. We spent months planning a self-serve import tool. Then one engineer just did the first twenty imports by hand, from whatever spreadsheets people sent us, and it turned out there were four kinds of mess that covered almost every case. The tool we eventually built handled those four and nothing else, and it was maybe a tenth the size of the original design.

The section of Graham’s essay engineers most need to hear is the one about delighting your early users, the Wufoo notes and so on. When engineers answer support email themselves, even occasionally, two things happen. Bugs that would have sat in a queue for weeks get fixed that afternoon, because the person who can fix them just felt the user’s pain. And they start making better calls without being told, because now they know who they’re building for. It doesn’t scale and you can’t do it forever, but in a small company I can’t think of a better use of an engineer’s afternoon.

None of this is an excuse to never automate. If you’re on the tenth deploy-by-text-file and still finding new surprises, the problem isn’t the lack of a script, it’s whatever you’re deploying. But the failure I see much more often is the opposite. A team builds a beautiful general-purpose platform for a workflow nobody has actually done yet, and every abstraction in it is a guess.

Graham ends on the idea that the unscalable things you do early become part of the company’s DNA, so if you work hard to delight a handful of users now, you’ll keep doing it when there are thousands of them. He also has a bit in there about how the big launch almost never works, which I agree with.