Lossless Online Bayesian Bagging
Herbert K. H. Lee and Merlise A. Clyde
Duke University
July 1, 2002
Bagging frequently improves the predictive performance of a model.
An online version has recently been introduced, which attempts to
gain the benefits of an online algorithm while approximating regular
bagging. However, regular online bagging is an approximation to its
batch counterpart and so is not lossless with respect to the bagging
operation. By operating under the Bayesian paradigm, we introduce
an online Bayesian version of bagging which is exactly equivalent to
the batch Bayesian version, and thus when combined with a lossless
learning algorithm gives a completely lossless online bagging
algorithm. We also note that the Bayesian formulation resolves a
theoretical problem with bagging.
The manuscript is available in postscript and
pdf formats.