Limitations of polynomial chaos in Bayesian parameter estimation

Abstract:

In many science or engineering problems one needs to estimate parameters in a model on the basis of noisy data. In a Bayesian approach, prior information and the likelihood of the model and data are combined to yield a posterior that describes the parameters. The posterior can be represented by Monte Carlo sampling, which requires repeated evaluation of the posterior, which in turn requires repeated evaluation of the model. This is expensive if the model is complex or if the dimension of the parameters is high.

Polynomial chaos expansions (PCE) have been used to reduce the computational cost by providing an approximate representation of the model based on the prior and, hence, creating a surrogate posterior. This surrogate posterior can be evaluated inexpensively and without solving the model.

Here we investigate the accuracy of the surrogate posterior and PCE-based samplers. We show, by analysis of the small noise setting, that the surrogate posterior can be very different from the posterior when the data contains significant information beyond what is assumed in the prior. In this case, the PCE-based parameter estimates are inaccurate. The accuracy can be improved by adaptively increasing the order of the PCE, but the cost may increase too fast for this to be efficient.

We illustrate the theory with an example from subsurface hydrodynamics in which we estimate the permeability on the basis of noisy pressure measurements. Our numerical results confirm what we found in theory and indicate that an advanced MC sampler which uses data to generate effective samples can be be more efficient than a PCE-based sampler.