February 2009
This paper develops efficient sequential learning methods for the estimation of general mixture models. The approach is distinguished from alternative particle filtering methods in two major ways. First, each iteration begins by resampling particles according to posterior predictive probability, leading to a more efficient set for propagation. Second, each particle tracks only the state of sufficient information for latent mixture components, thus leading to reduced dimensional inference. In addition, we describe how the approach will apply to more general mixture models of current interest in the literature; it is hoped that this will inspire a greater number of researchers to adopt sequential Monte Carlo methods for fitting their sophisticated mixture based models. Finally, we show that this particle learning approach leads to straightforward tools for marginal likelihood calculation and posterior cluster allocation. Specific versions of the algorithm are derived for standard density estimation applications based on both finite mixture models and Dirichlet process mixture models, as well as for the less common settings of latent feature selection through an Indian Buffet process and dependent distribution tracking through a probit stick-breaking model. Three simulation examples are presented: density estimation and model selection for a finite mixture model; a simulation study for Dirichlet process density estimation with as many as 12500 observations of 25 dimensional data, and an example of nonparametric mixture regression that requires learning truncated approximations to the infinite random mixing distribution.