🚀 Breaking Down DeepSeek’s $5.57M Training Cost – What’s Missing?

Over the past 2 weeks, I have got messages and comments to deep dive into the training costs of Deepseek. This topic garnered so much interest that markets reacted, discussions exploded, and some even claimed the AI bubble had burst.

Now that the dust has settled, let’s take a step back and look at what the technical paper actually says.

🎯 Here’s what we do know from the DeepSeek-V3 paper:
👉 Pre-training: 2,664K H800 GPU hours (~$5.328M)
👉Context Length Extension: 119K H800 GPU hours (~$0.238M)
👉Post-Training (SFT + RLHF): 5K H800 GPU hours (~$0.01M)
👉Total Training Cost: 2.788M GPU hours (~$5.576M)

The paper clearly states that these costs only cover the official training of DeepSeek-V3. It does not go deeper into costs associated with prior research, ablation experiments on architectures, algorithms, or data. And that’s a big gap because these costs are tough to estimate—they depend on the number of experiments, dataset curation, and model variations tested.

If you’re building an LLM from scratch, without using pre-trained components, these costs can be massive.
1️⃣ Compute Needs – Training on raw, unoptimized data takes huge GPU hours.
2️⃣ Longer Development Cycles – Teams must experiment with different architectures, optimizers, and data processing techniques.
3️⃣ No Transfer Learning – Without leveraging existing models, everything must be learned from the ground up, requiring more data and iterations.

That’s why training a next-gen model—one that pushes us closer to AGI or ASI—demands huge budgets. If a paper doesn’t provide details on these steps, it’s tough to make an apple-to-apple comparison with other models.

As a technologist, I’d be the happiest person to see a leading model trained at such a low cost. But to truly assess this claim, we need more data. Without it, we’re just blindly following a headline.

Read the entire article here: