HBM2E: 460 gigabytes a second per stack
On 2 July 2020 SK hynix began full-scale mass production of HBM2E memory: eight 16-gigabit dies joined by through-silicon vias, 16 gigabytes per stack and over 460 gigabytes a second across 1,024 lines at 3.6 gigabits a second per pin.
Why it matters
The bottleneck in training stops being arithmetic and moves to feeding. The atlas holds the accelerator generations - the A100 arrived in May of the same year - and not what feeds them; without memory of this bandwidth the tensor cores stand idle most of the time.
SK hynix gives the arithmetic in the release itself: 3.6 gigabits a second per pin across 1,024 I/O lines is 3,686.4 gigabits a second, which is 460.8 gigabytes a second. Capacity is more than doubled against the previous generation, HBM2. The release puts the speed as an image: 124 Full-HD movies of 3.7 gigabytes each, every second. Through-silicon via technology is described separately, at up to 30 per cent less size and up to 50 per cent less power than existing packaging. Ten months separate the August 2019 development announcement from mass production, and the atlas stands on the second of those dates: availability, not a working sample. The record names no buyer. The release mentions deep learning accelerators and high-performance computing as a class rather than any system; SK hynix's first customer named on the record arrives two years later, with HBM3.