Unless You Work For A Rich LLM Company
Sigh.
Here’s yet another impressive piece of technology that we are unlikely to get our hands on until the bubble bursts. Micron have moved their PCIe 6.0 9650 Series controller and drives into mass production, and as you should expect by now they’ve been specifically designed for LLM workloads. They offer twice the bandwidth of PCIe 5.0 drives, hovering around sequential read speeds of 28 GB/s and writes at 14 GB/s. When it comes to random reads and writes it seems that PCIe 6.0 is worthy of a new acronym.
You could call the random read performance 5.5 million IOPS, or make it simpler and just say 5.5 MIOPS, which makes the writes 0.9 MIOPS as KIOPS is taking things a bit too far. The Micron 9650 PCIe 6.0 drives eat a massive 25W, almost double most PCIe 5.0 drives and it is why these drives can be watercooled.
All of this is wonderful, but we won’t be seeing PCIe 6.0 drives in enthusiast machines for quite some time.
