Is open-weights AI the path forward?

Last November, the week OpenAI’s board fired and then un-fired its CEO, we built a 60-ticket eval set so we could tell whether some other model could do our support ticket summaries if we ever had to switch. We’ve used it a handful of times since. On Thursday I ran it again, against Llama 3.1, which Meta released on the 23rd, and for the first time an open-weights model came close enough that switching wouldn’t be a downgrade we’d have to apologize for.

The tickets in the set are scrubbed of names and anything identifying, and the model runs on my laptop, so they never leave it. The 70B ran on our existing AWS account, under the agreement we already have with AWS, so it didn’t need a new vendor review, which for anything that might eventually see real tickets is the slow part. The numbers, scored by our support lead against the rubric, out of 60: the hosted OpenAI model we use in production got 52 summaries she marked as good. Llama 3.1 70B got 49. The 8B, which I also ran on my laptop for fun, got 40, and most of its misses were the same kind of miss, a summary that was accurate about everything except what the customer was actually asking for. I didn’t run the 405B through the whole set. I tried a dozen tickets with it and it wasn’t obviously better than the 70B on this task, which is not a task that needs a giant model.

That 49 is the interesting number. A year ago, when I ran Llama 2 13B on my laptop, the honest description was “close to GPT-3.5 if you squint”. Three points behind on our own eval, on a task we actually ship, is a different situation.

The letter

Zuckerberg published a letter alongside the release called “Open Source AI Is the Path Forward”, and it’s worth reading even if you’re cynical about the author, which I am a little. The argument is that AI will go the way Unix and Linux went. Closed versions come first and are better for a while, then the open one catches up and becomes the standard everyone builds on, because it’s cheaper and nobody wants to depend on a single vendor. He makes the case that it’s good for developers (you can fine-tune it, run it where your data is, and nobody can take it away), good for Meta, and safer for everyone, the last of which is the part I’m least qualified to judge and least convinced by.

He’s quite open about why it’s good for Meta: he says selling access to AI models isn’t Meta’s business model, so giving the models away costs them little and keeps them from being locked out of the platform by a competitor, the way he feels Apple has locked them out of mobile. I find that more convincing than the idealistic parts, honestly. A company giving something away because it makes strategic sense for them is a company that will keep doing it for as long as it makes strategic sense. That’s not forever, but it’s something you can reason about.

The word “open”

I said last year that Llama 2 wasn’t open source by the usual definition, and Llama 3.1 still isn’t. The weights are downloadable. The training data isn’t published, and the license is still Meta’s own. The 700-million-monthly-users clause is still there. So is an acceptable use policy.

What did change is the one I called a real restriction in the Llama 2 post: you couldn’t use the model or its output to improve any other large language model. The 3.1 license now allows it, as long as you credit Llama and name derived models accordingly. Meta’s announcement explicitly mentions synthetic data generation and distillation, which is to say: use the 405B to train something smaller. That’s a meaningful change, and it’s the kind of thing a company does when it wants to be the base layer for everybody.

I’d still call it “open weights” and not “open source”, and I’ll keep doing that, though I’ve accepted that I’m on the losing side of that vocabulary fight. In the 2000s I spent a fair amount of energy explaining to people that freeware and open source weren’t the same thing. They weren’t, and it didn’t matter, and most people still say “open source” when they mean “free download”.

Would we actually run it?

This is where I part company with a lot of the excitement. Running a 70B model ourselves means GPU instances, which are expensive and still hard to get at a size that makes sense for us, plus somebody on a team of about twenty engineers owning the serving setup, the upgrades, the monitoring, and the 3 a.m. page when it falls over. For one feature that summarizes tickets, that’s a bad trade. The 405B is out of the question; Meta’s own guidance is that it takes a full server of top-end GPUs even at reduced precision.

What we’d actually do is call Llama 3.1 through a hosted provider. It was on the cloud we already use the day it came out, and there are several other companies serving it. That’s still a vendor, and I’m aware of the irony of “open weights, rented from a cloud”. But it’s a different kind of dependency. If the provider raises prices or changes terms, the same model is available from someone else, or from us, with the same weights and the same behavior on our eval. When I wrote last November about the thin interface we put between our code and the model, I said the second implementation was “waiting”. It’s been written now, and it’s about 40 lines.

So my answer to the title is: for us, yes, partly, but not for the reason in the letter. I don’t need the freedom to fine-tune or to run it in my own data center. What I want is a second source. Open weights are the first time we’ve had one for an LLM that I’d trust with something customers see.

The one thing I’d want before switching anything over is to understand that 52 versus 49 better. Three tickets out of sixty is within the range where I’d want to rerun it with a fresh set before believing it in either direction, and our support lead has agreed to mark up another forty in August.