Particle Filters for Very High-Dimensional Systems

Abstract:

Nonlinear data assimilation for high-dimensional geophysical systems is a rapidly evolving field. Particle filters seem to be the most promising methods as they do not require long chains of model runs to start sampling the posterior probability density function (pdf). Up to very recently developments in particle filtering has been hampered by the ‘curse of dimensionality’, roughly meaning that the number of particles needed to avoid weight collapse growths exponentially with the dimension of the system. However it has been realised that for particle filtering it is not the dimension of the state vector but the number of independent observations that is the problem. Furthermore, proposal densities that ensure better positioning of the particles in state space before observations are encountered lead to much better performance.

Recently particle filters have been proposed that do not suffer from weight collapse by construction. In this talk I will present several of these new filters, including the equivalent-weights particle filters, new combinations with the implicit particle filter, and filters using large-deviation theory. I will present basic ideas and applications to very high-dimensional systems, including a full climate model. Emphasis will be on the fruitful forward directions and on areas that still need attention, as we haven’t solved the problem yet.