Sep 2009
We explore a Bayesian density regression model driven by linear projections of covariates. This offers an alternative to variable selection and provides the best linear compression of covariates for predicting a response in a regression setting. We provide a detailed construction of our probabilistic model based on smooth Gaussian processes and uniformly distributed linear subspaces. We prove posterior consistency properties of the resulting model under mild and easy to assess conditions. We develop a practical model fitting tool based on Markov chain Monte Carlo applied to an efficient interpolation-type approximation of the original model. We discuss model driven choice of the dimensionality of the minimal linear projection. Simulation studies are presented to compare our method against others in estimating the linear projection and to assess the behavior of our dimension selection procedure. Two real data sets are analyzed with the proposed method where we emphasize out-of-sample prediction, dimension selection and calibration of relative importance of variables.
Keywords: Bayesian Inference, Semiparametric Model, Posterior Consistency, Gaussian Process, Markov Chain Monte Carlo, Dimension Reduction.
The manuscript is available in PDF format.