2025: the year my team stopped writing boilerplate

At the end of 2023 I wrote that I had one decent before-and-after number for the year, the handle time on long support tickets, and that it existed only because our support lead had thought to measure it. The last line of that post was a promise: anything we roll out gets a baseline first.

I kept it, mostly by luck of timing. In January and February, before anyone on the team was using an agent on our code, I pulled two months of numbers from our repos and our ticket tracker, because I’d been meaning to for a year and it was a quiet stretch. Then in March Anthropic got through our vendor review, Claude Code went on the allowed list, and the rest of the year happened. So this year’s post has numbers in it, and I’ve tried to be careful about what they do and don’t show.

We were 20 engineers in January and we’re 21 now, including the two juniors who joined in May and August, and one senior engineer who left in October for a job closer to home. That matters for some of what follows, because nearly everything below is per engineer.

How the code got written

Since June, every PR description has said how the change was made: by hand, by an agent with someone approving each step, or by an agent running unattended in the sandbox. It’s self-reported and I’m sure it’s a little rough at the edges, but nobody has a reason to lie about it, and reviewers would notice.

From the start of June to last Friday, we merged 2,318 PRs:

by hand: 641 (28%)
agent with approvals: 1,402 (60%)
agent unattended: 275 (12%)

The by-hand share has gone down every month, from 41% in June to 19% in November. The unattended share went up slowly through the summer and then jumped in September, which is when I stopped being the only person running long upgrade jobs and wrote up how to do it.

What’s in the by-hand pile is the most interesting part. I read through a sample of 100 of them last week. About a third were tiny (config changes, copy fixes, a one-line bug fix where opening an agent would take longer than typing). Most of the rest were in the areas our autonomy table reserves for people, billing and permissions, plus a handful of hard bug fixes where the engineer said, more or less, that they’d needed to think and the agent was in the way.

The baseline, and what changed

Here’s what I can compare between January–February and October–November, two months each:

PRs merged per engineer per week: 3.1 before, 6.4 after.
Median lines changed per PR: 148 before, 86 after.
Median time from a PR opening to merging: 19 hours before, 27 hours after.
Production incidents: 9 in the first two months, 8 in the last two.

The first two numbers need reading together. We halved the PR size guideline in the spring, on purpose, so more PRs was partly us asking for smaller ones. Multiply them out, roughly, and the lines merged per engineer went up by about 20%, which is much less than the “two to five times” I hear at conferences, and I think it’s the more honest number. Lines are a bad measure of output, as I’ve said before, and I’m only using them because they’re the one measure that’s comparable across the two periods.

The third number is the one I worry about. PRs take longer to get through review now, even though they’re smaller, because there are about twice as many of them, and the reviewers are the same people. Review is the constraint on this team now. It was in 2023 too, and it’s worse.

The incidents didn’t change, as far as I can tell. Of this year’s 47 production incidents, the change that caused it had been made with an agent in 15, which is well below the share of our code that went through agents over the year, though with numbers this small I wouldn’t read much into that. None of the 15 was something an agent did that a person wouldn’t plausibly have done. The two worst days of the year were an AWS region and our CDN, and neither had anything to do with any of this.

The boilerplate, specifically

This is where the title comes from. As far as I can see from the PR labels and from asking people, nobody on the team writes test scaffolding by hand anymore, or the second through twentieth test case, or database migrations and their backfills, or the changes to our API clients after a schema change. And nobody does dependency upgrades by hand, including the annoying ones.

That last one is the clearest number we have. In January there were 138 open Dependabot PRs across our repos, a lot of them months old, because each one was a small risk and nobody wanted to own the breakages. There are 9 now. Most were done by unattended agents in the sandbox, with a spec that says “upgrade this, fix what breaks, don’t change behavior, write down anything that wasn’t obvious”. Two of our .NET services that were on unsupported versions in January aren’t anymore. Since September those upgrades all wait out a seven-day cooldown first, which hasn’t slowed anything down that I can see.

Support

The drafted-reply pilot went to the whole support team in October, on GPT-5, after we’d worked out why it had regressed on the trial-extension case in August (an instruction in our prompt that read, to the new model, as permission). This time we measured properly from the start, with a month of baseline. Median handle time for tickets where the agent used the draft is about 30% lower than for comparable tickets before. I don’t fully trust “comparable” there, since agents choose when to use the draft and probably use it on the easier ones. The citation rule from the spring is still in, and the eval set is up to 171 cases.

Money

Agent spend for engineering, mostly Claude Code on API billing, averaged about $210 per engineer per month in the second half of the year. The range is wide. The heaviest user, who runs most of the long upgrade jobs, was over $700 in October, and a couple of people are under $50. Against what an engineer costs, it’s small. It’s also a line in the budget that didn’t exist in January, and the CFO asked me in November to forecast it for 2026, which I did with less confidence than anything else in the plan.

We cut Copilot from 22 seats to 5 in September. The five are people who mostly work in the editor and like the agent mode there. Nobody else objected, which is a long way from 2023, when 16 of our 19 engineers said they’d object to cancelling it, most of them strongly.

What didn’t change, and what got worse

Design, again. The big decisions this year, like the webhook idempotency question after the AWS outage or what to do about the login page after the CDN one, were people talking and writing. I use a model to argue with my design docs now, and it’s better at it than in 2023, but it still doesn’t know which consideration is the one that matters.

The hard bugs, also again. The handful that took days still took days. Agents helped write the scripts to reproduce them and nothing with the part where someone has to understand what’s happening.

What got worse is reading. Everyone on the team reads far more code than they write now, and the code they read was written quickly by something that never gets tired. I said in the summer that the risk was people approving diffs they hadn’t read, and I’ve seen it happen, including to me on a four-pane afternoon. The walkthroughs we started for the juniors, where the author explains every line before opening the PR, have spread to about half the team voluntarily, which I take as a sign that people feel it too.

The juniors are doing well, as far as I can judge after a few months. They ship. Our May hire is now on the normal autonomy table for everything except billing. Whether they’re building the instinct for when something’s wrong as fast as the rest of us did, I can’t measure, and it’s the question I’m least sure about going into next year.

My own numbers

My text file has 734 agent tasks in it for the year, starting in late February. I accepted about 71% with at most small edits, against 84% in that first week in February, which sounds like going backwards until you remember that the first week was my own small side projects and most of this year was our real code, on much harder tasks. Most of what kept the rate up was me writing better specs, though the models got better too. I wrote a little under a quarter of my own merged PRs by hand this year, which for someone who still thinks of himself as a programmer first is a strange number to type.

For 2026, the obvious prediction is more of the same, faster, and I’m wary of it, because two years ago I didn’t predict this year at all. What I’m fairly sure of is that the constraint on my team has moved from writing code to deciding what it should do and checking that it does, and that we’re short of the people who are good at the second part. That’s what I’ll be hiring for.