The principle also holds independent of both the accuracy of the model and the phase of the LUCC research (original) (raw)

Uncertainty in extrapolations of predictive land-change models

Environment and Planning B: Planning and Design, 2005

This paper gives a technique to extrapolate the anticipated accuracy of a prediction of land-use and land-cover change (LUCC) to any point in the future. The method calibrates a LUCC model with information from the past in order to simulate a map of the present, so that it can compute an objective measure of validation with empirical data. Then it uses that observed measurement of predictive accuracy to anticipate how accurately the model will predict a future landscape. The technique assumes that the accuracy of the model will decay to randomness as the model predicts farther into the future and estimates how fast the decay in accuracy will occur based on prior model performance. Results are presented graphically in terms of percentage of pixels classified correctly so that nonexperts can interpret the accuracy visually. The percentage correct is budgeted by three components: agreement due to chance, agreement due to the predicted quantity of each land category, and agreement due t...

Assessing a predictive model of land change using uncertain data

Environmental Modelling and Software, 2010

This paper presents a method to assess models that predict changes among land categories between two points in time. Cross-tabulation matrices show comparisons among three maps: 1) the reference calibration map of an initial time, 2) the reference validation map of a subsequent time, and 3) the model's predicted map of the same subsequent time. The proposed method analyzes these three maps to evaluate the ability of the model to predict land change vis-à -vis a null model, while accounting for the error in the reference maps. We illustrate this method with a prediction of land change from 1971 to 1999 in Central Massachusetts, USA. Results reveal that the land change model predicts a larger quantity of transition from forest to built than the reference maps indicate, and the model allocates the transition erroneously in space, thus causing substantial error where the model predicts built in 1999 but the reference map shows forest. If the accuracy of each category in the 1971 reference map is greater than 81 percent, then the predicted change is larger than the error in the 1971 reference map. If the accuracy of each category in the 1999 reference map is greater than 82 percent, then the model's prediction disagreement with respect to truth is larger than the error in the 1999 reference map. Partial information concerning the accuracy of the reference maps indicates that the maps are likely to be more accurate than the 82 percent threshold. The method is designed to analyze predictions for the common situation when the levels of accuracy in the reference maps are not known precisely.

Useful techniques of validation for spatially explicit land-change models

Ecological Modelling, 2004

This paper offers techniques of validation that land-use and -cover change (LUCC) modelers should find useful because the methods give information that is useful to improve LUCC models and to set the agenda for future LUCC research. Specifically, the validation technique: (a) budgets sources of agreement and disagreement between the prediction map and the reference map, (b) compares the predictive model to a Null model that predicts pure persistence, (c) compares the predictive model to a Random model that predicts change evenly across the landscape, and (d) evaluates the goodness-of-fit at multiple-resolutions to see how scale influences the assessment. This paper introduces a new criterion called the Null Resolution, which is the spatial resolution at which the predictive model is as accurate as the Null model.

Comparison of the structure and accuracy of two land change models

International Journal of Geographical Information Science, 2005

This paper compares two land change models in terms of appropriateness for various applications and predictive power. Cellular Automata Markov (CA_Markov) and Geomod are the two models, which have similar options to allow for specification of the predicted quantity and location of land categories. The most important structural difference is that CA_Markov has the ability to predict any transition among any number of categories, while Geomod predicts only a one-way transition from one category to one alternative category.

Uncertainty in the difference between maps of future land change scenarios

Sustainability Science, 2010

It is essential to measure whether maps of various scenarios of future land change are meaningfully different, because differences among such maps serve to inform land management. This paper compares the output maps of different scenarios of future land change in a manner that contrasts two different approaches to account for the uncertainty of the simulated projections. The simpler approach interprets the scenario storyline concerning the quantity of each land change transition as assumption, and then considers the range of possibilities concerning the value added by a simulation model that specifies the spatial allocation of land change. The more complex approach estimates the uncertainty of future land maps based on a validation measurement with historic data. The technique is illustrated by a case study that compares two scenarios of future land change in the Plum Island Ecosystems of northeastern Massachusetts, in the United States. Results show that if the model simulates only the spatial allocation of the land changes given the assumed quantity of each transition, then there is a clearly bounded range for the difference between the raw scenario maps; but if the uncertainties are estimated by validation, then the uncertainties can be so great that the output maps do not show meaningful differences. We discuss the implications of these results for a future research agenda of land change modeling. We conclude that a productive approach is to use the simpler method to distinguish clearly between variations in the scenario maps that are due to scenario assumptions versus variations due to the simulation model.

Comparing the input, output, and validation maps for several models of land change

The Annals of Regional …, 2008

This paper applies methods of multiple resolution map comparison to quantify characteristics for 13 applications of 9 different popular peer-reviewed land change models. Each modeling application simulates change of land categories in raster maps from an initial time to a subsequent time. For each modeling application, the statistical methods compare: (1) a reference map of the initial time, (2) a refer-ence map of the subsequent time, and (3) a prediction map of the subsequent time. The three possible two-map comparisons for each application characterize: (1) the dynamics of the landscape, (2) the behavior of the model, and (3) the accuracy of the prediction. The three-map comparison for each application specifies the amount of the prediction's accuracy that is attributable to land persistence versus land change. Results show that the amount of error is larger than the amount of correctly predicted change for 12 of the 13 applications at the resolution of the raw data. The applications are summarized and compared using two statistics: the null resolution and the figure of merit. According to the figure of merit, the more accurate applications are the

A review of current calibration and validation practices in land-change modeling

Environmental Modelling & Software, 2016

Land-change models are increasingly used to explore land-change dynamics, as well as for policy analyses and scenario studies. In this paper we review calibration and validation approaches adopted for recently published applications of land-change models. We found that statistical analyses and automated procedures are the two most common calibration approaches, while expert knowledge, manual calibration, and transfer of parameters from other applications are less frequently used. Validation of model results is predominantly based on locational accuracy assessment, while a small fraction of the applications assessed the accuracy of the generated land-use or land-cover patterns. Of the reviewed model applications, thirty-one percent did not report any validation. We argue that to mature as a scientific tool, and to gain credibility for scenario studies and policy assessments, the validation of land-change models requires consideration of challenges posed by uncertainty, complexity, and non-stationarity of landchange processes, and equifinality and multifinality of land-change models.

Lessons and challenges in land change modeling as revealed by map comparisons

2000

This paper presents the most important lessons from a multi-year collaboration that compared thirteen cases of spatially-explicit land change modeling. A previous paper reports the statistical results of the validation exercise, while this paper offers the broader implications of those findings for the land use and land cover change modeling community. We express the lessons as nine challenges grouped under three themes: mapping, modeling, and learning. The mapping challenges are: to prepare data appropriately, to select relevant resolutions, and to differentiate types of land change. The modeling challenges are: to separate calibration from validation, to predict small amounts of change, and to interpret the influence of quantity error. The learning challenges are: to use appropriate map comparison measurements, to learn about land change processes, and to collaborate openly. The paper elaborates on why these challenges are especially important for the future research agenda in land change science. accuracy, land cover, land use, model, prediction, scale, validation. Lessons and challenges in land change modeling as revealed by map comparisons Page 3, printed on 08/02/07. Conference on the science and education of land use.