Neural networks are a useful statistical tool for nonparametric regression. In this paper, I develop a methodology for nonparametric regression within the Bayesian paradigm. I address the problem of model selection and model averaging, particularly the problem of searching the model space in terms of both the optimal number of hidden nodes in the network as well as the best subset of explanatory variables. I implement this with a method I call Bayesian Random Searching (BARS). I also demonstrate how to use a noninformative prior for a neural network, which is useful because of the difficulty in interpreting the parameters. Finally, I present results on the asymptotic consistency of the posterior for neural network regression.

Key Words: Bayesian statistics; Model selection; Model averaging; Noninformative prior; Bayesian random searching; Asymptotic consistency

The manuscript is available in postscript and pdf formats