How Tailscale works

Last week I wrote that if I put my agents on a box of their own, I’d want it to have no public SSH at all, and to be reachable only from my own devices over a private network. Everyone I asked said “just use Tailscale”, in the tone people use for things they’ve stopped thinking about. I don’t like using a network I can’t explain, so I spent a few evenings reading how it works, and this is what I came away with. Some of it I’m sure of. Where I’m not, I’ve said so.

There are two separate problems in a VPN like this, and Tailscale solves them with two separate pieces. One is the tunnel itself: how two devices encrypt traffic between them and know they’re talking to the right peer. That’s WireGuard. The other is everything around the tunnel: how each device finds out the other exists, what its key is, and how to get a packet to it when both are behind routers that drop anything they didn’t ask for. That’s most of what Tailscale actually is.

WireGuard, the tunnel

WireGuard is a VPN protocol by Jason Donenfeld, and it’s been in the mainline Linux kernel since 5.6, in March 2020. It’s small on purpose. The kernel implementation is around 4,000 lines, which is a rounding error next to IPsec or OpenVPN, and the small size was a large part of the argument for it. It picks one set of modern cryptography (Curve25519 for key exchange, ChaCha20-Poly1305 for encryption, BLAKE2s for hashing) and doesn’t negotiate. There’s no list of cipher suites, so there’s no downgrade attack against a list of cipher suites.

A WireGuard peer is identified by its public key, and that’s all. There are no usernames or certificates. Each side has a short config listing the peers it’ll talk to: their public key, and the private IP addresses that peer is allowed to use. The WireGuard paper calls this cryptokey routing. When a packet goes out to 100.64.0.5, the interface looks up which peer owns that address and encrypts to that peer’s key. When a packet comes in, it decrypts it with whichever peer’s key works, and then checks that the inner source address is one that peer is allowed to have. If not, it’s dropped. So an address on a WireGuard network actually means something: a packet from 100.64.0.5 came from whoever holds that key.

It’s also quiet. WireGuard runs over UDP, and it doesn’t answer anything that isn’t a correctly authenticated handshake. If you scan a WireGuard port you get the same nothing you’d get from a closed one. I like that a lot for my use.

What WireGuard doesn’t do is any of the logistics. Each side’s config has to contain the other’s public key, and at least one side needs to know an IP and port it can reach the other on. For two servers with static addresses that’s fine. For a laptop that’s on home Wi-Fi this morning and a coffee shop this afternoon, a phone on cellular, and a server somewhere, you’d be editing config files all day. And if both ends are behind NAT, neither one has an address the other can reach.

The coordination server

Tailscale’s answer to the first half of that is a coordination server, run by Tailscale, that acts as a shared phone book.

When you install Tailscale on a device and sign in, the device generates its own WireGuard key pair. The private key stays on the device. The public key goes to the coordination server, along with the addresses the device thinks it might be reachable on. The server gives each device an address from 100.64.0.0/10, a range reserved for carrier-grade NAT that’s unlikely to clash with anything on your home or office network, and then hands every device the list of the peers it’s allowed to talk to, with their public keys and their candidate addresses. When something changes, like a laptop moving networks, the new information gets pushed to everyone.

The access rules live there too. You write a policy in the admin console that says which devices or users can reach which others on which ports, and the server compiles it into packet filters that it sends to each device, which enforces them locally. That’s the part that would replace firewall rules for me: the server can say “my laptop and phone can reach the agent box, and the agent box can’t start a connection to anything”.

The important property is what the coordination server never sees. It has the public keys and the metadata, which devices exist and where they’ve been connecting from. It doesn’t have any private keys, so it can’t decrypt traffic, and the traffic doesn’t go through it anyway. If Tailscale’s servers went down, existing connections would keep working, though new devices couldn’t join and changes wouldn’t propagate. If they were compromised, an attacker could in principle hand out a bad public key and add a device of their own to your network, which is why there’s a feature called Tailnet Lock where your own devices have to sign any new node’s key. I’d turn that on for anything important, and I haven’t yet read enough about it to explain it properly.

The client is open source. The coordination server isn’t, though there’s an open-source reimplementation called Headscale if you’d rather run the phone book yourself.

Getting through NAT

This is the clever part, and the part I understood least before I started. Tailscale published a long post about it in 2020, How NAT traversal works, by David Anderson, which I’d recommend over anything I’ll say here. This is my short version.

Almost every device I own is behind something that drops unsolicited inbound packets. My laptop is behind a home router doing NAT. The phone is behind a carrier’s NAT. A VPS usually has a public address, but I’m going to firewall it so that nothing gets in. In the normal case, nothing can start a connection to any of them.

But a stateful firewall or NAT lets in replies. When my laptop sends a UDP packet from port 41000 to some address and port, the router remembers that and lets packets from that address and port back in, to port 41000, for a while. So if both devices send to each other at the same time, each one’s router sees outgoing traffic first, and then the other side’s packet arrives looking like a reply. Both doors are open, and neither side ever accepted an unsolicited packet. This is called hole punching, and it’s been around for a long time; it’s how a lot of voice and games traffic gets through.

To do that, each side needs to know what address and port it appears to come from on the public internet, which it can’t see from the inside. So it asks. Tailscale’s clients send a packet to a server on the internet whose only job is to reply with the source address it saw, the same idea as the STUN protocol that WebRTC uses. Each device reports what it learned to the coordination server, which passes it on to its peers, and then both sides start sending to each other’s public address. Usually within a second or two, one gets through each way, and the connection is direct.

That works when the NAT is the polite kind, which uses the same public port for a given inside port no matter where the traffic is going. The paper’s term is endpoint-independent mapping, and most home routers do it. The impolite kind picks a new public port for each destination. Then the port the address-check server saw is useless, because the NAT will use a different one for packets going to my server.

The trick for that case is my favorite thing in the whole article, because it’s a cousin of the birthday problem, and I don’t get to use probability on networking very often. Say one side is behind a NAT like that, and the other side’s address is known. The difficult side opens 256 ports and sends from all of them to the other side, which creates 256 mappings on its NAT at unknown public ports. The other side then sends probes to random ports on the difficult NAT’s public address, hoping to hit one of the 256. Each probe has a 256-in-65,536 chance, so the chance of at least one hit after n probes is 1 − (1 − 256/65536)ⁿ. That’s about even odds at 177 probes, 63% at 256, and 98% at 1,024. A thousand small UDP packets takes well under a second. When both sides are difficult, the numbers get much worse, and that’s where the next piece comes in.

DERP, for when nothing works

Sometimes the direct path never opens. Corporate networks that block outbound UDP, hotel Wi-Fi, two impolite NATs at once. For those, Tailscale runs relay servers around the world, which it calls DERP (Designated Encrypted Relay for Packets). A DERP relay accepts connections over HTTPS, which gets through almost any network that lets you browse the web, and forwards packets between devices by their public key.

The relay doesn’t weaken anything. The packets it forwards are already WireGuard-encrypted to the destination’s key, so the relay sees which keys are talking and how much, and nothing else. Connections usually start over DERP straight away, so traffic flows immediately, and then switch to the direct path once hole punching succeeds. If you run tailscale status you can see which peers are direct and which are relayed. It’s slower over a relay, but it works.

What I’d use it for

For the agent box, this gives me what I wanted. The server runs Tailscale and a firewall that drops everything on its public address. SSH listens, but only my devices can reach it, and only over the tunnel. My phone can get to it from anywhere. And the box never needs a public port open, which is the whole point.

The things I’d still want to think about are the keys. Tailscale node keys expire, 180 days by default, and you can turn that off per device, which is convenient for a server and exactly the kind of convenience I’d want to think twice about. Tailnet Lock, as I said, I need to read more about.

At work, this is the third time in a year someone’s suggested we replace our VPN, the cloud endpoint we set up after we went remote in 2020, with something like this. I used to say no on the grounds that I didn’t understand it well enough. I’ve lost that excuse now, and I’ll have to come up with a better one, or agree.