Why commit counts lie

Dan North’s The Worst Programmer I Know went around at the start of the month, and I’ve sent it to three people since. The story: his company introduced individual productivity metrics, story points delivered per person, tracked in Jira, and one programmer, Tim Mackinnon, scored zero. Every week. Because Tim almost never picked up a story himself. He spent his days pairing with everyone else, letting juniors drive and asking them questions, arguing with the seniors until they found a better design. The manager wanted him gone, North refused, and the line everyone’s quoting is “Tim wasn’t delivering software; Tim was delivering a team that was delivering software.”

I’ve had a Tim. I nearly did the manager’s version.

This was a few years ago. We’d started pulling a dashboard out of GitHub for a quarterly review, PRs merged and commits per engineer, mostly because it was easy to build and I wanted a number to look at. One engineer was at the bottom of it by a long way. Something like a fifth of the median, two quarters running. I had a one-on-one with him on the calendar and the dashboard open in another tab, and I’m not proud of the conversation I’d planned.

What I hadn’t put on the dashboard was review comments. When I went and looked (because he asked me, a bit coldly, whether I’d looked), he’d reviewed more PRs than anyone on the team, and not with a thumbs-up either. His comments were the ones that started with “what happens if this runs twice?” He was the person new hires got pointed at in their first week. Two of our better engineers at the time said, separately, that they’d learned more from his reviews than from anything else at the company. He wasn’t low output. His output just landed in other people’s commits.

We dropped the per-person dashboard after that quarter. I still look at the team-level numbers, cycle time and how long PRs wait for review, because those tell you something about the system. A per-person commit count mostly tells you who likes small PRs and who’s stuck on the gnarly migration that’s been one branch for a month, and it puts the people making everybody else faster dead last.

The thing that actually worked was asking people. At review time I now ask each engineer who on the team they go to when they’re stuck, and the same three or four names come up again and again. That’s not a metric either, and it can be gamed, but it has never once pointed at the wrong person, which is more than I can say for the dashboard.

(He left a couple of years later, for a staff role somewhere bigger. I wrote him a very long reference.)