Points, cells, or polygons? On the choice of spatial units in forest conservation policy impact evaluation

Abstract

A fast-growing literature uses remotely sensed land-cover data along with quasi-experimental statistical methods to assess the efficacy of forest conservation interventions. A critical modeling choice is the spatial unit of analysis—points, grid cells, and polygons are all commonly used. Yet little is known about the implications of this choice for treatment effect estimates and for their interpretation. We demonstrate that point-level data can generate treatment effect estimates substantially different from those based on polygon-level data when (i) a disproportionate share of sample points is drawn from relatively large, treated polygons as a result of random or quasi-random spatial sampling, and (ii) the intervention analyzed has heterogeneous effects that depend on treatment polygon size. Our paper has four parts. First, using real-world data (on the award of timber extraction permits to forest management units in Mexico) that meet the two aforementioned criteria, we demonstrate that point- and polygon-level data generate qualitatively different results, and we propose a simple method for weighting the point-level data to recover the polygon-level results. Second, we conduct a Monte Carlo simulation to clarify the mechanism that causes this phenomenon and to provide reassurance that it is not driven by unobserved confounding factors. Third, we present new evidence (on Mesoamerican and Dominican protected areas) suggesting this phenomenon is not uncommon. Finally, we discuss the implications of our findings for the design and interpretation of spatial evaluations of forest conservation interventions. Although our analysis focuses on point- versus polygon-level data, the mechanism we describe also applies to grid cell- versus polygon-level data.

Keywords

• impact evaluation
• permit
• deforestation
• sustainable forest management
• community forestry