Assessing quality of spatial models using the structural similarity index and posterior predictive checks

Abstract

Model assessment is one of the most important aspects of statistical analysis. In geographical analysis, models represent spatial processes, where variability in mapped output results from uncertainty in parameter estimates. Slight spatial misalignments can cause inflated error scores when comparing maps of observed and predicted variables using traditional error metrics at the level of individual spatial units. We conceptualize spatial model assessment as a continuous value map comparison problem and employ methods from image analysis to score model outputs. The structural similarity index, a measure that attempts to replicate the human visual system using a local region approach, is used as an exploratory map comparison statistic. The measure is implemented within a Bayesian spatial modeling framework as a discrepancy measure in a posterior predictive check of model fit. Results are reported for simulation studies representing a variety of spatial processes in a spatial and space–time context. A case study of rainfall mapping in Sri Lanka demonstrates the proposed methodology applied to assessment of Bayesian kriging interpolations. Both simulation studies as well as the case study demonstrate that the approach reveals hidden spatial structure not uncovered by traditional methods. The spatially sensitive assessment methodology provides a diagnostic tool to support spatial modeling and analysis.