Evaluating observed versus predicted forest biomass: R-squared, index of agreement or maximal information coefficient?

Abstract

The accurate prediction of forest above-ground biomass is nowadays key to implementing climate change mitigation policies, such as reducing emissions from deforestation and forest degradation. In this context, the coefficient of determination ($${R^2}$$) is widely used as a means of evaluating the proportion of variance in the dependent variable explained by a model. However, the validity of $${R^2}$$ for comparing observed versus predicted values has been challenged in the presence of bias, for instance in remote sensing predictions of forest biomass. We tested suitable alternatives, e.g. the index of agreement ($$d$$) and the maximal information coefficient ($$MIC$$). Our results show that $$d$$ renders systematically higher values than $${R^2}$$, and may easily lead to regarding as reliable models which included an unrealistic amount of predictors. Results seemed better for $$MIC$$, although $$MIC$$ favoured local clustering of predictions, whether or not they corresponded to the observations. Moreover, $${R^2}$$ was more sensitive to the use of cross-validation than $$d$$ or $$MIC$$, and more robust against overfitted models. Therefore, we discourage the use of statistical measures alternative to $${R^2}$$ for evaluating model predictions versus observed values, at least in the context of assessing the reliability of modelled biomass predictions using remote sensing. For those who consider $$d$$ to be conceptually superior to $${R^2}$$, we suggest using its square $${d^2}$$, in order to be more analogous to $${R^2}$$ and hence facilitate comparison across studies.

Keywords

• model assessment
• overfitting
• biomass
• lidar