PyTorch
Facebook released a library in which the computation graph is built as it runs, so a model can be debugged with ordinary language tools.
Why it matters
Researchers got back the ability to write an experiment as an ordinary program, and the field moved onto this tool within three years.
TensorFlow required declaring the whole graph and then running it, so an error in the declaration surfaced far from where it was made. PyTorch executes operations immediately, so ordinary debuggers and printing intermediate values work. For research, where a model changes daily, that proved decisive. By 2020 most papers at the main conferences used PyTorch.