Book review: Kill It with Fire

We have one service left on .NET Framework. It’s a Windows service, written in the company’s second year, that builds the invoice exports some of our customers load into their accounting systems: about forty different file layouts, a scheduler, and roughly 9,000 lines of C# that nobody on the team today wrote. It works. It has worked for years. It’s also the last Windows server we run in production, and in August we decided to replace it.

About a week into that, a friend sent me Kill It with Fire, by Marianne Bellotti (No Starch Press, March 2021), with a note that said “before you do anything you regret”. I’m five years late to it. Bellotti spent years at the US Digital Service working on some of the oldest systems the federal government runs, and the book is what she learned about modernizing them without wrecking anything. It’s about 250 pages and there’s very little filler.

Her starting point is that the word “legacy” mostly gets applied to systems that succeeded. Nobody needs to modernize a system that failed; it got turned off. The old thing is still running because people depend on it, and everything awkward about it is the accumulated record of what they depend on. That reframing alone was worth the price for me, because “the old export service” had become a bit of a joke on our team, and it’s actually one of the few pieces of our system that has never had an incident I can remember.

The first chapter, “Time Is a Flat Circle”, argues that a lot of what looks like progress in technology is the industry swinging between the same few designs, centralized and distributed, thin clients and thick ones, and that knowing where you are in the swing helps you tell a real improvement from fashion. I mostly agree. I’ve lived through enough of those swings, from the Windows client-server apps of the early 2000s onward, to recognize it, though I think she pushes it a little further than it goes.

The part I underlined most is her case against the full rewrite whose goal is feature parity. Parity sounds like the safe, modest goal, and it’s the opposite. To match the old system you have to know everything it does, which you don’t, because a lot of what it does was never written down and some of it was never intended. And a parity project delivers nothing to anyone until it’s finished, so the people funding it lose patience long before the end. She’d much rather you find a narrower goal that someone actually cares about, deliver it, and keep the momentum. Her chapter on momentum is mostly about people and morale, not code, and that’s true of most of the book. The technology chapters are good. The ones about getting an organization to keep going are better.

There’s also a chapter called “Design as Destiny”, about how the structure of the organization ends up in the structure of the system, Conway’s law with a lot of examples. I didn’t learn much new there, but it’s a clearer statement than most.

Reading it with agents in the room

This is where my project and the book pull apart a little. Bellotti’s advice is shaped by the fact that rewriting code is expensive. That’s a big part of why parity rewrites are dangerous: years of effort before any payoff. Some of that cost has collapsed. An agent can read 9,000 lines of C# and produce a plausible port to modern .NET in an afternoon. I know because we tried it on a branch, out of curiosity, on day two.

What the afternoon didn’t produce was any confidence that the port did the same thing, and that’s the part the book is really about. So the project changed shape. Billing is one of the areas our autonomy table keeps for people, and this is billing, so engineers are writing the new service. What the agents did instead was the work around it. One read the old code and wrote a list of every behavior it could find, down to the rounding rules and the branches that check for one particular customer, with line numbers. Another built a harness that runs a month of real export jobs through both services and diffs the output byte by byte.

The first full diff run had 22 differences across the forty layouts. Four were bugs in the new code. The other 18 were the old service doing something odd that customers’ import scripts depend on, like rounding tax per line instead of per invoice for one layout, or writing a trailing comma that one accounting package needs and nobody remembers why. That’s Bellotti’s point exactly. The code was never the hard part. The undocumented behavior was, and it was always going to be.

So the agents made the cheap part cheaper and the hard part more visible, which I’ll take. They didn’t change her argument about momentum either. We’re shipping the new service one layout at a time, starting with the three that have the most customers, and the old service keeps running the rest until they’re all moved, which is close to what she’d recommend and not what I’d have done in August.

If you run a team that owns anything older than five years, read it. The chapter on how to finish, which is about deciding up front what “done” means so the project actually ends, is the one I’ll reread when the last layout moves.