September 2009 (Edited, June 2010)
For the problem of model choice in linear regression, we introduce a Bayesian adaptive sampling algorithm (BAS), that samples models without replacement from the space of models. For problems that permit enumeration of all models BAS is guaranteed to enumerate the model space in 2p iterations where p is the number of potential variables under consideration. For larger problems where sampling is required, we provide conditions under which BAS provides perfect samples without replacement. When the sampling probabilities in the algorithm are the marginal variable inclusion probabilities, BAS may be viewed as sampling models "near" the median probability model of Barbieri and Berger. As marginal inclusion probabilities are not known in advance we discuss several strategies to estimate adaptively the marginal inclusion probabilities within BAS. We illustrate the performance of the algorithm using simulated and real data and show that BAS can outperform Markov chain Monte Carlo methods.The algorithm is implemented in the R package BAS available at CRAN.
Keywords: Bayesian model averaging; Inclusion probability; Markov chain Monte Carlo; Median probability model; Model uncertainty; Sampling without replacement
The manuscript is in available in PDF format and is accepted for publication in the Journal of Computational and Graphical Statistics.
Cite as:
@Article{Clyde:Ghosh:Littman:2010,
Author = "Merlise Clyde and Joyee Ghosh and Michael Littman",
Title = "Bayesian adaptive sampling for variable selection and model averaging",
Year = "2010",
Journal = "Journal of Computational and Graphical Statistics",
Volume = "To appear"
}