Residual connections (ResNet)
An architecture with skip connections was described that made it possible to train much deeper networks.
Why it matters
Residual connections removed a practical limit on depth and are still used in current architectures, transformers included.
The paper addresses image recognition; generalisation to other tasks was shown by later work.