Forecast Verification for North American Mesoscale (NAM) Operational Model over Karst/Non-Karst regions

Karst is defined as a landscape that contains especially soluble rocks such as limestone, gypsum, and marble in which caves, underground water systems, over-time sinkholes, vertical shafts, and subterranean river systems form. The cavities and voids within a karst system affect the hydrology of the region and, consequently, can affect the moisture and energy budget at surface, the planetary boundary layer development, convection, and precipitation. Carbonate karst landscapes comprise about 40% of land areas over the continental U.S east of Tulsa, Oklahoma. Currently, due to the lack of knowledge of the effects karst has on the atmosphere, no existing weather model has the capability to represent karst landscapes and to simulate its impact. One way to check the impact of a karst region on the atmosphere is to check the performance of existing weather models over karst and non-karst regions. The North American Mesoscale (NAM) operational forecast is the best example, of which historical forecasts were archived. Variables such as precipitation, maximum/minimum temperature, dew point, evapotranspiration, and surface winds were taken into account when checking the model performance over karst versus non-karst regions.

The forecast verification focused on a five-year period from 2007–2011. Surface station observations, gridded observational dataset, and North American Regional Reanalysis (for certain variables with insufficient observations) were used. Thirteen regions of differing climate, size, and landscape compositions were chosen across the Contiguous United States (CONUS) for the investigation. Equitable threat score (ETS), frequency bias (fBias), and root-mean-square error (RMSE) scores were calculated and analyzed for precipitation. RMSE and mean bias (Bias) were analyzed for other variables.

ETS, fBias, and RMSE scores show generally a pattern of lower forecast skills, a greater magnitude of error, and a greater under prediction of precipitation over karst than non-karst regions. In addition, standardized data was used to eliminate differences from varying climates across CONUS. The metrics derived from the standardized data shows further evidence that the NAM forecast had lower forecast skills and an overall higher magnitude of error over karst than non-karst regions.