What's a model factory, and why is it worth $6 billion?

Nvidia has just paid $6 billion for a machine it can neither touch, photograph nor load onto a truck. Better: the company selling it continues to use it every day, quietly, as if nothing had happened.

It's called the Model Factory. The model factory. It belongs to Poolside, a company that makes AI models specialized in code, and the transaction itself can be summed up in three sentences: Nvidia pays $6 billion for the right to use it, hires 109 people who know how to run it, and puts another billion into the company along the way. Poolside keeps its model, its customers and its three founders. There. It's done.

Except that we don't really care about the transaction. What is worth sitting down for five minutes over is the machine. Because we talk about these models all day long without ever asking ourselves how you make one, and the answer is much more concrete, much dirtier and much funnier than you might imagine.

Five stations, a conveyor belt, and at the very end, a file. All this hardware to give birth to a thing that fits on a USB stick.

First, is it software or a real machine?

The question is worth asking right away, because the word factory gets everyone picturing the wrong thing. No, it's not a machine for engraving chips like the ones made by ASML, those contraptions delivered in spare parts on several cargo planes and reassembled on site over several months.

It's software. Dozens of programs talking to each other.

But if you stop there, you completely miss the point, because this software doesn't do anything on its own. It runs a room. A real one, with tens of thousands of graphics cards inside, miles of cables, air conditioning that screams and an electricity bill measured in megawatts. The factory is the software plus the room plus the people who know what they're doing. All three. Remove one and there's nothing left at all.

And to understand why it's worth so much, you have to look at what really happens in the room while a model is being made.

A breakdown every three hours for two months

Here I'm lucky, because there's a public and precise document on the subject: when Meta released its big Llama 3 model, they published a technical report where they honestly explain how the training went. And it's enlightening.

The model was trained on 16,384 graphics cards for 54 days straight. Over that period, they counted 466 interruptions to the work. Forty-seven were planned, maintenance. The other 419 were not. That's one breakdown every three hours, day and night, for two months.

The graphics card is by far the component that fails most often. Which works out well, seeing as there are sixteen thousand of them.

And the worst part isn't there. The worst part is that training of this kind is synchronous: the sixteen thousand cards advance in lockstep, together, and Meta explicitly writes that a single card failing can force the entire job to be restarted. One card out of sixteen thousand. Every three hours.

Anyone who has already launched a six-hour process that crashes at five and a half hours knows exactly what I'm talking about. Now multiply that by two months, and by a room with sixteen thousand graphics cards consuming power the whole time.

So here's the number I've been wanting to show you from the start, and it's the one that answers the question "what is this machine for". Out of these 419 breakdowns, how many times did a human have to intervene seriously?

Three times.

Three. Everything else was detected, diagnosed, worked around and recovered automatically, without waking anyone up. That's what a model factory is. That's what makes a two-month project on equipment that breaks constantly reach the finish line anyway. Without it, you just have sixteen thousand very expensive graphics cards and a team spending their nights restarting scripts.

Little bonus from the same report, because it's delicious: they observed that the room's output dropped by 1 to 2% in the middle of the day, quite simply because it gets hotter and the cards throttle themselves. And when the tens of thousands of cards start waiting at the same time, the building's power consumption jumps by several dozen megawatts all at once, which, I quote, puts the local electrical grid at its limit. We are a long way from software.

Now, let's open the hood

Poolside did something quite rare in this business: they publicly explained, station by station, how their factory is built. A series of technical articles on their own site. So what follows isn't a reconstruction on my part, it's their blueprint.

The production line has five stations.

Station 1, the raw material. Text and code are collected from everywhere: open datasets, web scraping, code repositories sorted by quality and license, plus text made specifically for the purpose. Then it's cleaned up, and I'll come back to that in two minutes because it's the most underestimated station in the entire factory. Everything is served to the machines by an in-house system they call Blender, the mixer, which decides in real time what proportion of what goes into the grinder.

Station 2, the cooking. The program that distributes the computation across all the cards at the same time and that has to survive what I told you about above. They named it Titan, and in the title of their own article they call it "the factory's oven". I'm not inventing the metaphor, it comes from them, and it's accurate.

Station 3, the test bench. A platform that really runs the code the model writes. This one deserves its own section, it's coming.

Station 4, the finishing touches. Where a raw model becomes a usable model. Same thing, dedicated section, because that's exactly the question everyone is asking themselves without daring to put it into words.

Station 5, the output. A file. A big file of numbers, but a file. That's all. This whole factory for that.

Why a good factory produces a better model

That's the real question, and the answer fits into a sentence that Poolside writes on its site: experiments that used to take weeks to program now run in less than an hour.

You have to measure what that means.

No one, today, knows how to calculate in advance what will make a good model. No one. We have intuitions, habits, past results, but the exact recipe, the proportion of code compared to text, the size of the model compared to the amount of data, the way to organize the training, all that is found by trying. Not by thinking: by trying, measuring, and starting over.

So the person who can test a hundred ideas while their neighbor tests three isn't going to win because they're smarter. They're going to win because they've tried a hundred things. It's a race to learning speed, and the factory is what sets that speed.

That's actually why the race for the biggest model has always seemed a little silly to me, and I wrote thirteen days ago about a Chinese model with ten trillion parameters that the number of parameters was the least interesting figure in the whole affair. The number of parameters is the engine size. The speed at which you can try a new idea is the driver.

The first cleaning: the data

This station is the most thankless one and it's the one that pays the most.

When you scrape the web to feed a model, you don't collect a library. You collect a dump. Ads, navigation menus, cookie banners, insults, entire pages generated automatically that don't mean anything, broken code, code copied fifteen times, code under a license you're not allowed to touch. Poolside puts it very politely in their article: existing datasets contain considerable amounts of content that is simply useless, such as advertisements.

So they filter it in several passes. First, the crude stuff, with a machete: insults, hate, ads. Then something finer, based on the characteristics of the document. Then, and this is where it gets interesting, they automatically group documents by subject and throw out entire batches when the model is visibly having trouble learning anything from them.

The announced result: about 5% better on average in the tests, and up to 150% on some.

One hundred and fifty percent. Without touching the model, without adding a single graphics card, without changing one line of the algorithm. Just by cleaning up what it's given to read.

That's the kind of thing every developer knows by heart in another field. You can optimize your code for three days, if your database is full of crappy duplicates, you're wasting your time. Garbage in, garbage out. The rule hasn't changed since punched cards, it just costs a lot more to ignore now.

The station I find the cleverest in the whole factory

Poolside makes code models. And for a code model, there is one judge the other fields don't have: the machine.

When a model writes a poem, nobody can objectively say whether it's good. When it writes a function, they can. You run it. It compiles or it doesn't compile. The tests pass or they don't pass. There is no debate, no opinion, no subjective score: it works or it doesn't work.

So they built a platform that does exactly that, on a very large scale. The model proposes code, the machine runs it for real, the tests decide, and the result goes back to the model in the form of a score. Well done, you start again. Failed, you change it.

The only score that can't be argued with. The compiler doesn't do pleasantries, that's for sure.

And this loop keeps running continuously, without ever stopping. The model doesn't learn to write code that looks correct, it learns to write code that passes. The difference is huge, and that's precisely where the difference lies between an assistant that gives you something plausible and an assistant that gives you something that runs.

To build that, you need to know how to run unknown, potentially dangerous code, continuously and in parallel, without it setting your infrastructure on fire. It's not a small engineering task. It's even, in my opinion, the most expensive piece in the whole lot.

The second cleanup: the model itself

Second question we rarely ask ourselves: why do you have to finish a model before releasing it? It's trained, it's read everything, it should work, right?

No. And the explanation is almost comical.

What comes out of the oven, what we call the base model, knows an astonishing amount of things and knows how to do nothing. Poolside puts it very well: base models are learned, but they aren't good at solving problems. Because what it was taught wasn't to answer. It was to continue.

So you write it “how do you sort a list?”, and it, very seriously, answers you “how do you sort an array? how do you sort a file? how do you sort...”. It's not making fun of you. It has seen billions of web pages go by, it has understood that in real life a question is very often followed by other questions, and it does exactly what it was trained for: the continuation.

On the left, it has read everything. Really everything. It just can't do anything with it for you.

The finishing station is for that. First, you show it thousands of examples of the right way to behave: you ask a question, you give an answer, you stop. That's what gives it predictable behavior and, incidentally, what people call its personality. Then you bring out the big guns, the loop from station 3, the one that executes and scores, to teach it to maintain a long line of reasoning and not give up halfway through a complicated problem.

The first pass serves as a warm start for the second. We rough it out by hand, then refine it at execution time. And at the end, you have something you can give to a human being without them looking at you funny.

So why Nvidia? Are they preparing their own models?

That's the question that naturally comes up, and the answer may surprise you: they're not preparing them, they're already putting them out.

Nvidia has been publishing a family of open models called Nemotron for a while now, in three sizes, and they don't just provide the final file: they also publish the datasets and methods used to make it. They even set up a coalition with other players to build high-level models in public. So the debate over "is Nvidia going to make its own models" was settled a long time ago. What they were missing wasn't the desire, it was speed.

And then there's a more down-to-earth reason. Nvidia sells the cards. All the cards, or nearly all of them. A toolchain that really knows how to take advantage of these cards, that keeps them running at full capacity instead of leaving them waiting, that recovers from failures on its own, mechanically increases the value of every card sold. They've been calling their data centers AI factories for years. They've just bought a factory. It makes sense, you have to give them that.

The third point is more discreet. It's neither an acquisition nor poaching, the letter to investors takes care to spell that out in black and white, because an acquisition goes before the competition authorities and can sit there for months, whereas a license plus hires takes much less time. It's neither new nor illegal, it's just become the usual form of major operations in the sector.

And the other side of the story, which is considerably less glorious

Because you can wonder why a company that has just built a six-billion-dollar machine agrees to license it and let the team that ran it leave.

The answer is brutal, and its co-founder said it himself. They had a six-week window to raise two billion dollars intended to pay for a set of 40,000 cards that was supposed to start up in January. They didn't close it in time. They lost the whole set.

There you have it. They had the recipe, the team, the factory, and no oven left.

The same co-founder adds that the constraint at the moment isn't just money, but physical space in data centers and computing capacity that has already been reserved. In other words: even with the check, someone still has to have built the building. And he specifies that fewer than 70 people built this model, fewer than 115 counting all the engineering and research.

Look closely at that number, then look at the 109 people hired by Nvidia. That's practically the whole team. Which confirms what I was saying at the beginning: this factory doesn't exist without the people who know how to run it. You don't buy a manual, you buy a skill that lives in a hundred heads.

Okay, and what about you?

Let's be honest: today, nothing. You'll never see this machine, you'll never use it, and it won't change your day tomorrow.

But there are two very real consequences, and the first one is rather good news.

The free models you can download and run at home come straight out of factories like these. Poolside's code model has open weights, meaning that the final file is public and installs on your machine. Meta did the same thing eleven days ago with a model that runs on a gamer's graphics card. Every advance in these manufacturing chains ends up, a few months late, in a file that you can put on your hard drive without an account, without a subscription and without sending a line of your work to anyone. For code under contract or just code you don't want to see leave, that's not a minor detail.

The hangar costs two billion. What comes out of it ends up on your desk, and it's free. Spot the mistake.

The second consequence is less cheerful. If the value is now in the factory, and a factory costs six billion, then the number of companies capable of housing one will soon be countable on the fingers of one hand. Poolside had the best possible team and got knocked out by a banking schedule problem. That gives you an idea of the price of admission.

As for when all this shows up in your everyday life: it is already the case, actually. The assistant that completes your sentences, the one that sorts your photos, the one that answers you when you call customer service, they all came out of a production line like this one. You just never saw the workshop.

What to take with a pinch of salt

The amounts come from a letter to investors revealed by the Newcomer newsletter, not from a document filed with a regulator. Bloomberg, The Next Web and The Decoder all confirm the six billion, but a republished report announced six hundred million: I am keeping the figure from the three matching sources and telling you where it comes from.

Next, nobody has published the exact inventory of what the license covers. Poolside described its factory publicly, which is already remarkable, but there is a lot of room between a technical blog post and a contractual list. We know what Nvidia is paying. We do not know exactly what it is receiving.

Finally, the failure figures I showed you come from Meta and concern the training of Llama 3, not Poolside's. They are orders of magnitude for the industry, not measurements from this particular factory. Poolside has not published its own.

What I think

What I like about this story is that it puts things back where they belong. For three years we have been sold models like cars are sold, with rankings, names that pack a punch and curves that keep going up. And it turns out that the real value was behind them, in the workshop, in the scripts that restart a job at three in the morning because a card has given up, in the filters that throw out the ads, in the platform that executes the code instead of taking its word for it.

It is the kind of lesson every developer ends up learning the hard way. The value of a project has never been in the deliverable, it is in the ability to do it again. Lose your build chain and you have nothing left, even with the binary in your hands.

And then there is something I find quite beautiful, all the same. All this lavish spending, this room as big as a hangar, these sixteen thousand cards breaking down every three hours, these filters, these tests, these millions of executions, all of it to end up with a file that someone will download for free and run on the tower they built themselves in their living room.

So if you are one of those who download these models to run them at home, know that you are the end of a two-billion-dollar chain. Make the most of it, and think from time to time about the three guys who got up in the night!

Sources

• Poolside : The Model Factory, the description of the platform by its authors
• Poolside : Gathering and processing raw materials for the Model Factory, data cleaning and measured gains
• Poolside : The finishing touches, what post-training does
• Meta : The Llama 3 Herd of Models, section 3.3.4 and table 5, the 466 interruptions and the 3 human interventions
• Newcomer : Poolside Strikes 6 Billion Licensing Deal with Nvidia, the letter to investors, August 21, 2026
• The Decoder : Nvidia is acquiring Poolside's Model Factory and 109 employees
• Latent Space, Eiso Kant's comments on the lost cluster and the size of the team
• Nvidia : Nemotron Labs, How Open Models Give Enterprises and Nations AI They Can Trust