Netflix: what went from the Prize into production
On 6 April 2012 Xavier Amatriain and Justin Basilico of Netflix wrote that from the open contest the company had put into production two algorithms from the winning entry of the first Progress Prize in 2007, matrix factorisation and restricted Boltzmann machines. The blend of hundreds of models that won the Grand Prize was not put in: the accuracy gain measured offline did not justify the engineering effort.
Why it matters
The company itself drew the line between a contest record and a production system: what went in was not the most accurate model but two parts of one, adapted to data fifty times larger. The post also records a change of goal, from predicting a rating to ranking what a member will play; by the company's account 75 per cent of what people watched already came from recommendations.
Team Korbell won the first Progress Prize with an 8.43 per cent improvement and a blend of 107 algorithms. Matrix factorisation gave an RMSE of 0.8914, the Boltzmann machines 0.8990, their linear blend 0.88; they had been built for 100 million ratings, and Netflix had more than 5 billion. Cinematch's RMSE was 0.9525 and the Prize's target 0.8572. The post gives more than 23 million subscribers in 47 countries, 2 billion hours streamed in the fourth quarter of 2011 and 4 million ratings a day. Netflix's annual report for 2011 (10-K, 10 February 2012) gives a figure for a different measure, 26,253 thousand unique subscribers including DVD, and lists the recommendation technology among business risks; it does not confirm the 75 per cent. This is the company's own account.