Plot-level semantically labelled terrestrial laser scanning point clouds

Abstract

Point clouds from Terrestrial Laser Scanning (TLS) are an increasingly popular source of data for studying plant structure and function. However, unlocking their full potential currently requires extensive manual processing to extract ecologically important information. One key task is the accurate semantic segmentation of different plant material within point clouds, particularly wood and leaves, which is required to understand plant productivity, architecture, competition, space optimisation and physiology, and is a key step in common approaches to individual tree extraction. Existing automated semantic segmentation methods are primarily developed for single ecosystem types, and whilst they show good accuracy for biomass assessment from the trunk and large branches, often perform less well within the crown. In this study, we demonstrate a new framework that uses a deep learning architecture developed from PointNet++ and pointNEXT for processing 3D point clouds to provide a reliable semantic segmentation of wood and leaf in TLS point clouds from the tree base to branch tips, applied to diverse natural European forests. Our model combines meticulously labelled data with voxel-based sampling and a novel gated reflectance integration module embedded throughout the feature extraction layers. We evaluate its performance across an extensive dataset, encompassing diverse ecosystem types and sensor characteristics. Our results show consistent outperformance against the most widely used PointNet++-based approach for leaf/wood segmentation on a high-density TLS dataset collected across diverse mixed forest plots across all major biomes in Europe. We tested our model against others’ open data from China, Eastern Cameroon, Germany and Finland, collected using both time-of-flight and phase-shift sensors, finding consistently strong performance, showcasing the transferability of our model to a wide range of ecosystems and sensors. Our newly developed evaluation metric for assessing performance in the outer parts of the canopy, such as in twigs and small branches, found our model to clearly outperform the most widely used approach.

Methods

Within each country, we scanned a subset of the 30 m x 30 m FUNDIV plots using a Riegl VZ400i TLS instrument (RIEGL Gmbh, Horn, Austria), scanning at 600MHz and with an angular resolution of 0.04 mrad. All plots were scanned following a 10m grid system with a minimum of 16 upright and 16 tilt scans (following Wilkes et al. 2017), with additional scans to minimise occlusion in dense areas, and on the plot perimeter. To ensure high-quality data with minimal noise, scanning was paused when wind conditions rose above 5 m/s (measured with an anemometer on the ground) or when gusts were visually evident. In order to create our labelled dataset, we used a semi-automated approach informed by existing approaches followed by significant manual cleaning. Vicari et al. (2019) found anisotropy, verticality and linearity to be informative features for leaf-wood separation, so we created these geometric features at spatial scales of approx. 5 cm - 0.5 m (using CloudCompare, 2023). Alongside these, we used reflectance and xyz information for each point and labelled leaf-wood by thresholding these features. We followed this with intensive manual checking and cleaning to ensure high label quality, especially in the smaller branches and twigs. Our dense scanning and labelling strategy means that this dataset is an ideal candidate for training and testing the capabilities of processing algorithms. Our dataset comprises nine 10 m x 10 m blocks of x, y, z coordinates with corresponding reflectance values and labels, all at 1 cm resolution achieved through voxel downsampling. This data was used for training and validation. For comprehensive evaluation, we incorporated additional openly available datasets, which we cropped to reduce size and cleaned to rectify erroneous labels. The original, unmodified versions of these datasets can be accessed as follows:

*For licensing reasons these data re not included in this repository but can be downladed from the doi provided.