The ePEStemology database: a meta-analysis of 15 years of research on knowledge claims of Payments for Ecosystem Services (PES)

Introduction

Payments for ecosystem services (PES) are a leading conservation finance tool to encourage land-users to deliver ecosystem services in exchange for financial incentives (Blundo-Canto et al., 2018; Kaiser et al., 2021; Salzman et al., 2018; Schomers and Matzdorf, 2013). They have generated considerable interest over the past two decades, following the publication of the UN’s Millennium Ecosystem Assessment (MEA, 2005). Yet, there has been a dearth of analysis that reveals the outcomes and evolution of the PES research body itself, with most studies having a very specific or limited focus on the theoretical and empirical development of these PES programmes. The ePEStemology database assembles a systematic literature review into a meta-analysis aimed at understanding knowledge generation in PES research. The database allows for an analysis of key trends in how the body of literature on PES has informed scholars and practitioners since the emergence of this conservation tool.

Research questions and main objectives:

The ePEStemology database is the largest review of PES scientific research to date and addresses the following research questions:

• In the history of 15 years of PES research, what are the dominant criteria used to define and assess success of PES in practice? Are there geographical patterns in defining those criteria? Where does research take place? And by whom?
• What does 15 years of PES research illustrate in terms of the social and political consequences that arise in the implementation of PES? To what extent does literature engage with territorial dynamics and conflicting frameworks around nature’s values?
• How has PES research evolved? Do research recommendations feed into future research objectives over time?
• How does the epistemological position, including methods used, geographic location and disciplinary focus of authors among other variables, influence PES research outcomes?

A core objective of the database is to ensure it remains a living compendium of PES research. While the database does not account for every peer-reviewed published article on PES, it facilitates the possibility of iteratively adding new publications or retroactively adding in missing articles as well as those that use cognate terms to define PES (e.g. as rewards or compensation for ecosystem services).  The database also aims to serve as an important basis for future research questions on overall or regional trends emerging from PES research.

Search strategy

Using these key objectives and questions, we defined a set of variables (see attached) and search strategy to construct the ‘ePEStemology’ database. The database is populated by International Scientific Indexed (ISI) peer-reviewed journal articles. It includes Anglophone articles in Scopus and Web of Science (WoS), using all of the search terms “PES”, “Payments for Ecosystem Services,” “Payment for Ecosystem Services,” “Payment for Environmental Service,” or Payments for Environmental Services” either in the title, abstract, or keywords of queried articles.  

Articles were excluded if they were produced in a language other than English or if they were book chapters, books, conference papers, reviews, or webpages. While recognizing the caveats of excluding research articles in other languages as well as in other media (e.g. as books or conference papers), we justify our approach in order to ensure consistency and comparability of the body of research literature.

Following the different steps prescribed by the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) protocol (Moher et al., 2009), we identified a total of 1,067 published articles between 2005 and 2019. We took the year 2005 as the starting point for published research on PES, following the Millennium Ecosystem Assessment (Millennium Ecosystem Assessment, 2005) of the same year as well as an initial seminal publication on the subject (Wunder, 2005). Initial screening of articles began in September, 2018 with a total of 1,215 ISI-peer reviewed research articles on PES identified between both the Scopus and WoS queries. Additional articles were included to account for the year 2019 in October, 2020 to total 1,439 articles.

It is also worth noting that Scopus providing the greatest coverage of published articles on PES, accounting for 78% of the total identified. WoS accounted for 66% of the total, but also included 305 articles that did not fall within the Scopus search.

The screening procedure for selecting articles was contingent on the time in which the search in SCOPUS and WoS were conducted. It should be noted that this systematic review does not account for every ISI-research article published on PES during the time period considered. Since the database search tools are continuously updated retroactively, the query date may alter the number of articles retrieved in the search, increasing them over time despite being limited to specific dates. However, even if the database gets adapted over time, we can safely assume that the corpus represents a representative account of peer-reviewed published research on PES. In addition to the language exclusions made (i.e. our database only includes Anglophone publications), there are various permutations on the PES terminology, including “payments for hydrological services,” “payments for watershed services”, “conservation payments,” “rewards for ecosystem services,” or “agro-ecological incentives,” and many others that make it challenging to fully account for every relevant article. However, the protocol does not include all these possible variations, but can easily allow these variations to be included and brought into the analysis at a later stage.

Coding strategy

For the initial database (2005-2019) four independent reviewers to review each abstract and full text of each article were selected (BT, CJ, GVH, VK).  Each reviewer had experience in (empirical) PES research and have published peer-reviewed research on PES or other market-like transactions for ecosystem services (e.g. Tabaichount et al., 2019; Jacob et al., 2016; Van Hecken et al., 2015; Kolinjivadi and Sunderland, 2012). Broad variables of interest that guide our analysis (e.g. research objectives, disciplines, methods, author affiliations and positionality, geographical base, research outcomes, and others) served as the basis for coding. However, in order to establish a foundation of codes for each of these categories of interest and further refine each of those broader categories, the research team randomly selected 100 articles of the final 1,067 articles identified and applied a grounded theory open-coding process (e.g. Strauss and Corbin, 1990; Thornberg and Charmaz, 2014). The latter implies a process in which any preconceived identification for each category of interest was open to be further adapted prior to the final coding process. The iterative process of initial coding determined the most relevant sets of codes for each identified variable of interest, as well as possible values for each of them. Additional codes were added to the list as more articles were reviewed and until saturation was reached.

After discussing and deciding on the final categories/variables with the whole group,  all remaining  articles in the corpus were evenly and randomly divided among all reviewers. On average, each of the four reviewers coded between 500 and 600 articles.   

Subsequently, and for over a period of 15 months, the coding team revised and coded all articles based on the abstracts and a review of the full text in case the abstract did not allow to accurately code some of the variables. Each article was independently (blindly) coded by two different reviewers. To ensure robustness of coding, pairs of reviewers for each set compared results of the coding process for each article through a triangulation process and resolved discrepancies through collective deliberation and cross-checked consensus.

Variable list

The variable list and description of variables and their corresponding values can be found in the attached Word-document