Offboarding 30 people in an afternoon

The list had 30 names on it and 17 columns. I’d built it over the previous week with our IT lead and one person from HR, and for that week the three of us, plus the leadership team, were the only people in the company who knew what it was for.

Two weeks ago, on a Wednesday, we laid off 30 people, a bit under a fifth of the company. I’m not going to write about the decision. I was in the room for it and I agreed with it, and it’s still the worst week I’ve had in this job. What I want to write about is the narrow part I was responsible for, which was the access. Thirty people had accounts in dozens of systems, some of them with production access, a couple of them with admin rights, and at the end of the day every one of those accounts had to be closed, without anyone learning they’d been laid off from a login screen.

That last part was the requirement I cared most about, because of what we’d all just watched. When Twitter cut about half its staff on November 4th, a lot of people there seem to have found out because their laptop stopped working or Slack logged them out, in some cases before the email arrived. Whatever you think of the reasons, that’s a cruel way to learn you’ve lost your job, and it’s also bad security practice, because it means the access was cut on a guess about timing, not on a confirmed step.

The list

We had more of a head start than I expected. Our offboarding checklist goes back to 2015, when we found an ex-employee whose VPN account still worked months after he’d left, and it’s been extended every time we found another place an account could hide. Most systems now sign in through our identity provider, so suspending someone there turns off most of their access in one step. That was the first column.

The other sixteen columns were the things that one step doesn’t cover. Sessions that are already open, in apps that don’t check back with the identity provider for hours or days, so a suspended account stays logged in to them until the session expires unless you revoke it. GitHub, where we remove people from the organization, which also kills any personal access tokens they’d authorized for our repos. The handful of services that still have local accounts, like the error tracker and the status page. The shared folders in our password vault, which four of the 30 could see. Laptops, which our IT provider manages. Email, where suspending isn’t the same as deleting, because deleting an account in our email system can cancel every meeting that person organized, including customer calls their manager now needs to take over. And, new since our forgotten cloud account in the summer, which cards and billing contacts were in their names. Two were, and we moved both, quietly, the week before.

The day

Every person was told by their own manager, one on one, on a video call, with someone from HR on the call too. The calls ran from 10 a.m. to about 1:30, in parallel across managers. Each manager had a private channel with me and the IT lead, and when their call ended, they posted the person’s name, and we ran the steps for that person. Not before. A person who was in a call being told should still have been able to read their email, because they might have needed to.

Most of the steps were one script against the identity provider and the apps that have APIs for it: suspend, revoke sessions, remove from GitHub, remove from the vault’s shared folders. The rest was a checklist, by hand, two of us reading it aloud to each other. The vault’s shared entries were the slow part. Four people’s worth of shared credentials meant rotating every entry in the folders they’d had access to, 31 of them, which took the rest of the afternoon and well into the evening, and touched the domain registrar, two payment dashboards and a couple of AWS break-glass accounts.

People kept their personal phones, obviously, and we’d asked HR to collect personal email addresses in advance so the paperwork and the severance details went somewhere they could read it. Laptops came back in prepaid boxes that the IT provider sent out the next day. If anyone wanted personal files off them, the IT provider copied those off before the wipe. Several people asked, mostly for photos.

The next morning we ran the full list again, every system against every name, from the access-review sheet rather than from memory. That found three more accounts, all local logins in tools we’d added to the sheet but not to the checklist, and none of them used since the day before. They’re on the checklist now.

The call that ran long

The part I got wrong was at the end. For the last few calls, which were all scheduled to end at 1:30, I’d set the script to run on a timer at 1:35, because I wanted to get started on the vault rotation and I assumed the calls would end on time. One of them didn’t. The person had questions, which was entirely reasonable, and his manager was answering them, and at 1:35 Slack and his email logged him out in the middle of a sentence. The video call stayed up, because it was a link his manager had sent and he’d joined it without signing in. He told his manager what had just happened, his manager apologized, and then I called him later that afternoon and apologized too.

It was exactly the thing I’d built the whole process to avoid, and I did it to save about ten minutes. Nothing about the security outcome changed. The account would have been closed ten minutes later either way.

I still don’t know whether the ten minutes he spent being told, with his access still open, was a risk worth taking for everyone else. The standard advice from security people is to cut access at the moment the conversation starts, and I understand why. In a company our size, where I know most of these people by name, I chose the other way, and I’d choose it again. But it’s a judgment call, and I’d like to hear from anyone who’s done this with a different answer and thinks theirs was better.