Quality evaluation of citizen-observed data to the LandSense demonstration cases II

Citizen observations have the potential to revolutionise the field of Land Use and Land Cover (LULC) monitoring, greatly increasing reporting capacity and enabling near real-time response to emerging environmental hazards (e.g., deforestation, flooding). However, accuracy and other data quality issues are a key concern when utilising citizen observations. A suite of Quality Assurance (QA) tools suitable for LandSense were identified (D5.1) and implemented in phase I of the project (D5.4).

This deliverable reports on the wider implementation of all QA tools with the data collected in phases I and II of the project and provides detailed results from this work. The use and performance of eight QA tools is discussed across all three LandSense themes (urban landscape dynamics, agricultural land use and forest and habitat monitoring) using heterogeneous datasets from six pilot studies. The QA platform performed as designed and no notable operational errors were encountered. Modifications and additions were made to some of the QA tools in light of the findings of D5.4 and are described here. Quality and privacy checks on photographic data collected performed well (e.g. 90%+ accuracy in detecting privacy features) and correlations between photo quality and feature detection were investigated and described. Image blur was not found to be a significant problem and only detected in specific instances (i.e. low light conditions and images taken from moving vehicles).

QA checks were used to assess hundreds of user observations of LULC features and demonstrated the ability to identify areas of both high and low agreement between multiple contributors. Links between contributor agreement and the type of LULC feature are also described. Building on this work, QA checks on the categorical accuracy of user contributions were performed and found to be very promising. It was found that Volunteered Geographic Information (VGI) was of sufficiently good quality for identifying key types of LULC, such as residential land use change or detecting and identifying the urban fabric. However, some specific LULC features were harder for VGI to identify accurately, e.g., distinguishing between different types of agricultural land use. The results outlined in this deliverable will form the basis for development of the LandSense QA good practice guide (D5.7).