Uninvited guests at our all-hands

Our CFO was about two slides from the quarterly numbers when one of our engineers sent me a direct message: “who is iPhone?”

This was the first all-hands since we went remote, a Thursday at the start of April, on Zoom. The participant list said 87. I scrolled through it. There were three phone numbers, which turned out to be people who’d dialed in from the car, there was “iPhone”, camera off and muted, and there was a name I recognized but couldn’t place for a second, until I did: a contractor who’d worked with our marketing team last year and whose contract ended in the fall.

I messaged the CEO, who was hosting, and he removed both of them from the meeting. The CFO, who hadn’t noticed anything, skipped the slide with the bank balance on it, and we carried on.

How they got in

The contractor was easy to find, and entirely innocent. When he was working with us he’d been invited to the all-hands as a courtesy, back when it was a monthly meeting in the big conference room with a Zoom link for anyone remote. The invite was a recurring event. When his contract ended, nobody took him off it, and since our CEO had always used his Personal Meeting ID for it, the link never changed. So on that Thursday a calendar reminder popped up on his laptop, he clicked it out of habit, and he found himself in our all-hands. He told us later he’d figured he was still welcome and didn’t want to be rude by leaving. I believe him. I’d probably have done the same.

“iPhone” was the one that worried me. We went through everything we could think of. Not one of our people recognized the device name or admitted to joining from a second phone. Zoom’s participant report gave us an IP address that geolocated to a mobile carrier somewhere in the right country, which narrows it down to about 300 million people.

What we did find, while looking, was worse than either guest. The all-hands event lived on a shared company calendar in G Suite. Years ago someone had made that calendar public, with full event details, so the careers page could embed our office holiday schedule. The careers page doesn’t embed it anymore and hasn’t for a long time, but the calendar was still public, which meant anyone who found its address could see every event on it, including the all-hands, with the Zoom link and its meeting ID in the description. And the same Personal Meeting ID was in the CEO’s email signature, because he used it for sales calls too.

So the link to our all-hands had been, in effect, published for over a year. That was fine while the all-hands was a conference room with a laptop at the end of the table. It wasn’t fine once it was 80-odd people on a video call talking about the bank balance.

What we changed, quickly

This was the same week the rest of the internet discovered that random people could wander into Zoom calls. The FBI put out a warning about “Zoom-bombing” at the end of March, after meetings at schools in Massachusetts were interrupted by strangers, and Eric Yuan announced on April 1st that Zoom was freezing new features for 90 days to work on security and privacy. On the 5th Zoom turned on passwords and waiting rooms by default for everybody. Most of the hijacked meetings in the news had been public links posted on social media. Ours just hadn’t been hijacked by anyone loud.

We made the calendar private the same afternoon, and then went through every other shared calendar in the domain and found two more public ones, neither with anything sensitive on them. Then the meeting itself. The all-hands now gets a fresh meeting ID every time, generated by Zoom rather than the CEO’s personal room, with a passcode. Only people signed in with a company account can join, which Zoom supports through our Google login, and everyone else goes to the waiting room. Screen sharing is host-only. Once the meeting’s started and the participant count matches what we expect, the host locks it. Zoom added a Security button to the toolbar on April 8th that puts most of this in one place, and that’s made it a lot easier for people who aren’t me to run meetings this way.

We also told everyone to stop putting their Personal Meeting ID in email signatures and calendar descriptions, which got some grumbling from sales, since the PMI is convenient and they use it for every prospect call. A recurring meeting with a fixed link is fine for a customer call. It’s not fine for anything you wouldn’t want a stranger to sit in on.

The Windows link bug

There was one other Zoom item that week that was more interesting to me technically. Zoom’s Windows client turned UNC paths in chat, the \\server\share kind, into clickable links. If you clicked one that pointed at a server on the internet, Windows would helpfully try to authenticate to it over SMB, which sends your Windows username and an NTLM hash of your password to whoever runs that server, and weak passwords can be cracked from those offline. This is a very old Windows behavior and I’ve seen it used in CTFs more than once. Zoom fixed it in a client update on April 1st.

We have maybe a dozen people on Windows laptops. Our firewall config at the office would have blocked outbound SMB, but nobody’s at the office anymore, and home routers don’t block it. I pushed the Zoom update, checked versions across our managed machines (all but two had updated themselves, the other two got a nudge), and also set the Windows policy that restricts outgoing NTLM to remote servers, which we should have had anyway.

The rest of the Zoom news

I’ll admit I went back and forth on whether we should keep using Zoom at all. In the space of two weeks we learned that its iOS app had been sending data to Facebook through the Facebook SDK (fixed within days), that its “end-to-end encryption” wasn’t end-to-end in the sense everyone else means it, and from Citizen Lab that it uses AES-128 in ECB mode and that some meeting keys had been issued through servers in China even for calls with no participants in China. That last one in particular isn’t a mistake I’d expect from a company that takes crypto seriously.

We’re staying for now. The things that actually exposed us were a public calendar and a reused link, both our own doing, and both would have exposed us on any other video service just as well. Zoom’s response has been fast, and the 90-day freeze is the right move if they mean it. But I’ve put a reminder in my calendar for July to look at it again.

The loose end is “iPhone”. The contractor joined because of a stale invite. The other person joined from a link that had been sitting in public for a year, and we have no idea how they found it, what they were looking for, or whether that was the first meeting they’d sat in on. Zoom’s logs for our account only go back so far, and the earlier all-hands were in a conference room where nobody was looking at the participant list at all.