Multi-Site Model Benchmarking: Do Land Surface Models Leak Information?

Abstract:

It is widely reported that land surface models (LSMs) are unable to use all of the information available from boundary conditions [1-4]. Evidence for this is that statistical models typically out-perform physics LSMs with the same forcing data. We demonstrate that this conclusion is not necessarily correct. The statistical models don’t consider parameters, and the experiments cannot distinguish between information loss and bad information (disinformation).

Recent work has outlined a rigorous interpretation of model benchmarking that allows us to measure the amount of information provided by model physics and the amount of information lost due to model error [5]. Recognizing that a complete understanding of model adequacy requires treatment across multiple locations [6] allows us to expand benchmarking theory to segregate bad and missing information. The result is a benchmarking method that that can distinguish error due to parameters, forcing data, and model structure - and, unlike other approaches, does not rely on parameter estimation, which can only provide estimates of parameter uncertainty conditional on model physics.

Our new benchmarking methodology was compared with the standard methodology to measure information loss in several LSMs included in the current and developmental generations of the North American Land Data Assimilation System. The classical experiments implied that each of these models lose a significant amount of information from the forcing data; however, the new methodology shows clearly that this information did not actually exist in the boundary conditions in the first place. Almost all model bias can be attributed to incorrect parameters, and that most of the LSMs actually add information (via model physics) to what is available in the boundary conditions.

1 Abramowitz, G., Geophys Res Let 32, (2005).

2 Gupta, H. V., et al., Water Resour Res 48, (2012).

3 Luo, Y. Q. et al., Biogeosciences 9, (2012).

4 Han, E., et al., J Hydromet (2014).

5 Gong, W., et al., Water Resour Res 49, (2013).

6 Gupta, H. V. et al., Hydrol Earth Syst Sci 10, (2013).