Cerebras WSE-3

Cerebras' WSE-3 chip, unveiled in 2024, holds four trillion transistors and 900,000 cores on a single wafer. It is about the size of a dinner plate.

Context

Most chips are cut from a silicon wafer in dozens or hundreds of small pieces. Cerebras instead keeps almost the whole wafer as one processor, which is why its chip is so much larger than any ordinary one, around 46,000 square millimetres by the company's figures. The WSE-3 is made on a 5-nanometre process and was announced in March 2024.

The idea is to keep data moving on the chip itself instead of between many separate processors, which can speed up training and running large AI models. The company says the chip also carries tens of gigabytes of fast on-chip memory. Those performance claims come from the manufacturer, so independent benchmarks are the better guide to real speed.

Because small defects are unavoidable in silicon, the design routes around faulty cores so the whole wafer can still work. The transistor count is stated by the company rather than counted independently, and the record applies to a single chip, not to a system built from many chips. Wafer-scale chips need special cooling and power delivery because so much circuitry sits in one place, and Cerebras sells them inside purpose-built systems rather than as parts for ordinary servers.