Research · March 1995

AdaBoost

Freund and Schapire showed how to combine classifiers barely better than chance into one strong classifier, raising the weight of mistakes at each step.

Why it matters

The theoretical question of whether a weak learner can be made strong got a practical answer, and it immediately started winning competitions.

Each successive classifier trains on a reweighted sample where examples the earlier ones got wrong count for more. Training error falls exponentially with the number of steps. The method proved oddly resistant to overfitting, which stayed a theoretical puzzle for years. In 2001 the first practical face detector was built on AdaBoost.

Event record

Event date

March 1995

Timeline date

Event date

Verification

Sources gathered automatically · September 17, 2026

Lines

Learning theory

ID

evt-0174

Presented at the second European Conference on Computational Learning Theory in March 1995; the journal version appeared in 1997.

Sources

primary A decision-theoretic generalization of on-line learning and an application to boosting
EuroCOLT 1995 · Published March 1995

Related events

Builds on A Theory of the Learnable
It answers the question posed in learnability theory: whether a weak learner can be made strong.
A decision-theoretic generalization of on-line learning and an application to boosting
Related Induction of Decision Trees
In practice the method was most often applied to decision trees.

Records that link to this one

Builds on Viola-Jones face detection
The cascade of classifiers is trained by boosting itself.
Rapid Object Detection using a Boosted Cascade of Simple Features
Related Conditional random fields
Section 6 of the paper: a CRF can be trained with the same exponential loss that AdaBoost uses.
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Contrasts Random forests
The abstract: random feature selection at each node gives error rates that compare favourably with AdaBoost but are more robust to noise.
Random Forests
Builds on Gradient boosting
The abstract discusses directly how the approach connects to the boosting of Freund and Schapire: AdaBoost becomes a special case of gradient descent in function space.
Greedy function approximation: A gradient boosting machine.