cat posts/2025-01-28-deepseek-r1.md
DeepSeek-R1, and what a cheap open reasoning model changes
The first thing the 14B model printed was <think>, and then it argued with itself for about forty seconds. I’d asked it a scheduling question we use as an interview exercise, three people, overlapping availability, one of them in a different time zone, and I watched it try an answer, notice that it had treated 9 a.m. in two zones as the same hour, say “wait, that’s not right”, and start over. It got there in the end. That’s a distilled 14-billion-parameter model, quantized to about 9 GB, running on the same M1 MacBook Air I ran Llama 2 on eighteen months ago, with the Wi-Fi off.
What came out
DeepSeek, a Chinese lab spun out of a quant hedge fund, released R1 on January 20th. The full model is a 671-billion-parameter mixture-of-experts (about 37 billion active per token), built on the DeepSeek-V3 base they published at Christmas. On the reasoning benchmarks in their paper it’s roughly level with OpenAI’s o1: 79.8% on AIME 2024 against o1’s 79.2%, and close on the math and coding sets. It’s under the MIT license, which is a much less complicated license than Meta’s, and the reasoning trace is right there in the output, where o1 hides its chain of thought and bills you for it anyway.
The API price is the other number. DeepSeek is charging $2.19 per million output tokens for R1, where o1 is $60. Input is similarly cheap. Even if you don’t trust their API with your data (I’ll get to that), the weights are on Hugging Face, and anyone with enough GPUs can host it.
They also released six smaller models, from 1.5B to 70B, made by fine-tuning existing Qwen and Llama models on R1’s outputs. Those are what I’ve been running. The 14B is the largest that’s comfortable in 16 GB of RAM, and it’s a different thing from Llama 2 13B. On the handful of tasks I’ve been using to compare models since 2023, it beat the old 13B easily on anything with steps in it, and it’s slow, because thinking out loud costs tokens and the Air produces them at reading speed at best. It also thinks out loud about things that don’t need it. Asking it to rephrase an email produced a page of deliberation about tone.
The part of the paper I found most interesting is R1-Zero, a version trained with reinforcement learning on the base model and no supervised examples of reasoning at all. It’s rewarded only for getting answers right and putting the reasoning in the right tags, and it learned on its own to write longer chains, check itself and backtrack. The authors include a transcript where the model writes, in the middle of a math problem, “Wait, wait. Wait. That’s an aha moment I can flag here.” I don’t know what to make of that sentence yet, beyond noting that the recipe is in a public paper now and isn’t a trade secret anymore.
Monday
The market reaction is the part everyone’s talking about, so briefly: DeepSeek’s app went to the top of the US App Store over the weekend, and on Monday Nvidia’s stock fell about 17%, roughly $590 billion of market value, apparently on the theory that if a capable model can be trained cheaply, people will buy fewer chips. That theory leans on the V3 paper’s figure of about $5.6 million in GPU time for the final training run. That number is real as far as it goes, but it’s the cost of one run on hardware they already had, and it leaves out the research, the failed runs and the cluster itself. Comparing it to what OpenAI spends is comparing a final exam to a degree. I’d also point out that it’s a week since OpenAI, SoftBank and Oracle stood in the White House announcing a $500 billion data center project, so the market is holding two very different stories at once.
I don’t have a view on Nvidia. I have a view on prices, which is that the cost of a given level of capability keeps falling much faster than I’d have guessed, and this is the steepest drop yet.
What it changes for us
Two things, I think.
The first is about what open models are good for. We already have a second source for our ticket summaries. Last summer Llama 3.1 70B scored 49 to production’s 52 on the 60-ticket eval set we built after the OpenAI weekend, and the 40-line second implementation behind our model interface calls a hosted Llama. Summarizing was never the hard case, though. What we didn’t have was an open model for the work where reasoning models like o1 are clearly better than the rest, the multi-step debugging and planning questions people bring to the internal chat page, and now there’s one at roughly o1’s level, under a license with none of Meta’s clauses in it. I ran the 32B distill against our eval on Sunday, out of curiosity, on a GPU instance in our own AWS account so the tickets never left it. It’s a bit worse than production at summarizing, partly because its summaries come with a page of reasoning attached that we’d have to strip out, which is about what I expected from a reasoning model on a writing task. The more interesting test is a set of reasoning questions we don’t have yet, and I’ve started writing one. I said last summer that running a big model ourselves is a bad trade for a team our size, and I still think that. What I’m watching for is R1 turning up at a hosting provider we already have a contract with.
The second is the reverse. Our AI usage policy says new tools get asked about before anyone uses them, and three people asked about DeepSeek’s chat app yesterday. The answer is no, for anything to do with work. Its privacy policy says data is stored on servers in China, and it’d have to go through the same vendor review as anyone else, which it wouldn’t pass for us. There are also lots of screenshots this week of it declining to talk about Tiananmen Square, which is a reminder that a model is trained with someone’s priorities, and that’s true of every model, not only this one. The weights are a different question. A model running on our own hardware doesn’t send anything anywhere, and the MIT license doesn’t ask us to agree to anything. It’s an odd situation where the same model is a no on the website and a maybe through a provider we already trust, and I’ve had to explain that distinction to people twice already.
When the “no moat” memo leaked in 2023, I thought it was half right: open models were closing the gap quickly, but they were still clearly behind on quality. On summaries, the gap closed last summer. I’d assumed reasoning would stay closed-model territory for a good while longer, and on the things that are easy to measure, the gap there is now a few months. I don’t know if that lasts. The labs with the biggest budgets will presumably ship something better soon, and it’s possible the distills are good at benchmarks and worse at the messy stuff in ways I haven’t found yet. But it happened a lot faster than I expected, and that’s worth writing down.