Facebook: boosted trees ahead of logistic regression
On 24 August 2014 eleven Facebook engineers, among them Joaquin Quiñonero Candela, described at the ADKDD workshop a model for predicting clicks on ads: gradient boosted trees transform the features, and a logistic regression updated online gives the probability of a click. The combination beat either method alone by more than 3 per cent in normalized entropy.
Why it matters
On ads served to more than 750 million daily users, the work showed that the right features and fresh data decide the outcome, while the choice of model and other parameters matter less. With Microsoft's paper of 2007 it records how click prediction became the central machine-learning task in advertising, on which a platform's revenue rests.
The paper: 'over 750 million daily active users and over 1 million active advertisers'; Facebook ads are not tied to a query, so there are more candidates than in search, and a cascade of classifiers winnows them, the paper describing its last stage. Real-time training data come from a system called the online joiner. Facebook's annual report for 2013 (10-K, 31 January 2014) gives 757 million daily active users in December 2013 and $6,986 million of advertising revenue for 2013 out of $7,872 million in total; this is reporting under liability to the regulator, though from the same company. The record does not claim how much revenue the model itself brought: the paper does not measure it.