EXPERIMENTS IN STOCHASTIC COMPUTATION FOR HIGH-DIMENSIONAL GRAPHICAL MODELS

Beatrix Jones, Carlos Carvalho, Adrian Dobra, Chris Hans, Chris Carter & Mike West

Duke University

Published in: Statistical Science, 2005

We discuss the implementation, development and performance of methods of stochastic computation in Gaussian graphical models, with a particular interest on the scaleability of MCMC and other stochastic search methods with dimension. Our perspective is that of high-dimensional model search -- we are interested in exploring the complex, high-dimensional spaces of undirected graphical models that arise due to uncertainty about model form. We review the structure and context of undirected graphical models, Gaussian models and model uncertainty (the so-called covariance selection problem). We discuss prior specifications, including new priors over models and hyper-Markov priors on covariance patterns within models, and then explore a number of examples using various methods of stochastic computation. This discussion represents both a review of the theoretical structure of these graphical models and of a number of key aspects and details of existing computational ideas, as well as the point of departure for experimentation with MCMC methods. We then discuss alternative stochastic search ideas, and in examples compare and contrast MCMC methods with a novel stochastic model search approach. We summarize our experiences in trying to use these methods in problems in low (12-20) to moderate (150) dimensions. The examples combine simple synthetic examples with data analysis from gene expression studies. We conclude with comments about scaleability of the approaches studied and the need and potential for new computational methods in far higher dimensions, the need for new theoretical insights, and alternative constructive approaches to Gaussian graphical modeling and computation.

Paper published in Statistical Science (2005), and the original manuscript - that includes broader discussion and also general appendix material on graphical models, is available as a manuscript here.

Some relevant C++ code for implementation of stochastic computational methods discussed in the paper is also available.