Software in the era of AI, for teams

Our product manager built a working prototype of a reporting feature in an afternoon last month. She described it to an agent, iterated on the result for a few hours, and showed it in our Friday demo, with real-looking charts, filters that worked, and our seed data behind it. The room was impressed, me included. Turning it into something we could ship took two engineers a little under three weeks, because the prototype had no permissions model, ran its queries straight against the main database, and fell over on any account with more than a few thousand rows.

I thought about that a lot while watching Andrej Karpathy’s talk from YC’s AI Startup School, which went up last week. It’s aimed at founders, and most of it is about what kinds of products to build now. But two of his ideas describe what’s been going on inside my team this year better than anything I’ve managed to say myself, so this is my attempt to translate them from products to teams.

The two ideas

The first is the autonomy slider. Karpathy’s argument is that the useful AI products right now are “partial autonomy” apps, and the good ones let the user decide how much to hand over, from autocomplete to “change this file” to “go do the whole thing”. His metaphor was an Iron Man suit as opposed to an Iron Man robot: something that makes a person stronger and that they can take more or less control of, depending on the job.

The second is the generation-verification loop. The model generates and a human verifies, and the speed of the whole system depends on how fast that loop goes around. So you make verification fast, with good diffs and visual review, and you keep the model “on the leash”, working in chunks small enough to check. He said he doesn’t want a 10,000-line diff, because he’s still the bottleneck, and I laughed out loud, because I’d said almost the same thing in a one-on-one the day before.

He also talked about how big the gap is between a demo and a product, using self-driving cars as the example, and said this is more likely to be a decade of agents than a year of them. That’s the prototype story above, in one line.

The slider, as a team policy

From March, when Claude Code was cleared for our code, until a few weeks ago, how much autonomy an agent got was whatever the person using it felt like. Some people approved every edit. One engineer was letting it run for twenty minutes at a time on a service that handles billing. Neither of them was wrong, exactly, but there was no shared answer, and that showed up in review, where nobody knew how much of a PR the author had actually looked at.

So we wrote the slider down. It’s a short table in the engineering handbook:

Area	How far an agent goes
Billing, auth, anything touching money or permissions	Suggestions only. A person writes the change.
Product services	Agent edits with approvals, and the author reads every line before opening a PR.
Internal tools, tests, scripts, docs	Agent can run on its own, in the sandbox, against seed data. Normal review.
Upgrades and mechanical migrations	Long unattended runs, in the sandbox, with a spec and a notes file.

That last row is from the experiment I wrote about in May. The first row is the one people argued about. Two engineers thought it was too cautious, and they may be right. I’d rather loosen it in six months with some evidence than tighten it after an incident.

Making verification the thing we invest in

The generation side is solved, as far as a team lead is concerned. Anyone can produce more code than they could last year. What’s changed on our team is that review is now clearly the constraint, and I’ve been treating it that way.

We cut the PR size guideline in half. I’ve been asking people to keep PRs small for about ten years with mixed success, and the difference now is that it’s cheap to comply. Asking an agent to split a change into four reviewable PRs is a sentence. Asking a person to do it used to be an afternoon.

Every PR now says, in the description, how it was made: written by hand, agent with approvals, or agent unattended. In 2023 I argued against requiring people to label AI-generated code, because it was unenforceable and fuzzy. With completions, it was. With agents, the line is much clearer, and it changes how the reviewer reads the PR, which is the whole reason to know.

And tests got promoted. A ticket isn’t ready to start until it has acceptance criteria someone could turn into a test, because a spec plus a test is what lets the agent’s loop and the reviewer’s loop both go faster. The engineers who write good tickets have become noticeably more productive with agents than the ones who don’t, which I didn’t expect and which says something about what the job is turning into.

Team shape

We have about twenty engineers, and the obvious question from the rest of the company is whether we need that many, given all of the above. My honest answer is that our output went up and our headcount needs didn’t go down, because what we’re short of is people who can verify, who know the system well enough to look at a passing diff and say “this is wrong”, and that’s the scarcest skill we have. If the work shifts from writing code to reviewing it, I need more senior judgment per engineer, not less. We hired a junior engineer in May anyway, and a second one starts later in the summer, partly because of that: if judgment is the scarce thing, somebody has to be building it. How they build it, when the tedious code that taught the rest of us is now written by an agent, is something we’re working out with the first one as we go.

Then there’s the prototype. Our product manager has asked if she can keep building them, and I’ve said yes, enthusiastically, as long as everyone understands that a prototype is a very good spec and nothing more. She’s taken that well. I’m less sure the rest of the company has, and the next time someone demos a working feature built in an afternoon, I’ll have to explain the three weeks again.