NEON Tree Species Predictions

# Individual Tree Predictions for 100 million trees in the National Ecological Observatory Network

Preprint: https://www.biorxiv.org/content/10.1101/2023.10.25.563626v1

## Manuscript Abstract

The ecology of forest ecosystems depends on the composition of trees. Capturing fine-grained information on individual trees at broad scales allows an unprecedented view of forest ecosystems, forest restoration and responses to disturbance. To create detailed maps of tree species, airborne remote sensing can cover areas containing millions of trees at high spatial resolution. Individual tree data at wide extents promises to increase the scale of forest analysis, biogeographic research, and ecosystem monitoring without losing details on individual species composition and abundance. Computer vision using deep neural networks can convert raw sensor data into predictions of individual tree species using ground truthed data collected by field researchers. Using over 40,000 individual tree stems as training data, we create landscape-level species predictions for over 100 million individual trees for 24 sites in the National Ecological Observatory Network. Using hierarchical multi-temporal models fine-tuned for each geographic area, we produce open-source data available as 1km^2 shapefiles with individual tree species prediction, as well as crown location, crown area and height of 81 canopy tree species. Site-specific models had an average performance of 79% accuracy covering an average of six species per site, ranging from 3 to 15 species. All predictions were uploaded to Google Earth Engine to benefit the ecology community and overlay with other remote sensing assets. These data can be used to study forest macro-ecology, functional ecology, and responses to anthropogenic change.

## Data Summary

Each NEON site is a single zip archive with tree predictions for all available data. For site abbreviations see: https://www.neonscience.org/field-sites/explore-field-sites. For each site, there is a .zip and .csv. The .zip is a set 1km .shp tiles. The .csv is all trees in a single file.

## Prediction metadata

*Geometry*

A four pointed bounding box location in utm coordinates.

*indiv_id*

A unique crown identifier that combines the year, site and geoindex of the NEON airborne tile (e.g. 732000_4707000) is the utm coordinate of the top left of the tile. 

*sci_name*

The full latin name of predicted species aligned with NEON's taxonomic nomenclature. 

*ens_score*

The confidence score of the species prediction. This score is the output of the multi-temporal model for the ensemble hierarchical model. 

*bleaf_taxa*

Highest predicted category for the broadleaf submodel

*bleaf_score*

The confidence score for the broadleaf taxa submodel 

*oak_taxa*

Highest predicted category for the oak model 

*dead_label*

A two class alive/dead classification based on the RGB data. 0=Alive/1=Dead.

*dead_score*

The confidence score of the Alive/Dead prediction. 

*site_id*

The four letter code for the NEON site. See https://www.neonscience.org/field-sites/explore-field-sites for site locations.

*conif_taxa*

Highest predicted category for the conifer model

*conif_score*

The confidence score for the conifer taxa submodel

*dom_taxa*

Highest predicted category for the dominant taxa mode submodel

*dom_score*

The confidence score for the dominant taxa submodel

## Training data

The crops.zip contains pre-cropped files. 369 band hyperspectral files are numpy arrays. RGB crops are .tif files. Naming format is 
<individualID>_<year>_<sensor>, for example. "NEON.PLA.D07.GRSM.00583_2022_RGB.tif" is RGB crop of the predicted crown of NEON data from Great Smoky Mountain National Park (GRSM), flown in 2022.
Along with the crops are .csv files for various train-test split experiments for the manuscript.

### Crop metadata

There are 30,042 individuals in the annotations.csv file. We keep all data, but we recommend a filtering step of atleast 20 records per species to reduce chance of taxonomic or data cleaning errors. This leaves 132 species.

*score*

This was the DeepForest crown score for the crop.

*taxonID*
For letter species code, see NEON plant taxonomy for scientific name: https://data.neonscience.org/taxonomic-lists

*individual*
unique individual identifier for a given field record and crown crop

*siteID*
The four letter code for the NEON site. See https://www.neonscience.org/field-sites/explore-field-sites for site locations.

*plotID*

NEON plot ID within the site. For more information on NEON sampling see: https://www.neonscience.org/data-samples/data-collection/observational-sampling/site-level-sampling-design

*CHM_height*

The LiDAR derived height for the field sampling point.

*image_path*

Relative pathname for the hyperspectral array, can be read by numpy.load -> format of 369 bands * Height * Weight

*tile_year* 

Flight year of the sensor data

*RGB_image_path*

Relative pathname for the RGB array, can be read by rasterio.open()

# Code repository

The predictions were made using the DeepTreeAttention repo: https://github.com/weecology/DeepTreeAttention
Key files include model definition for a [single year model](https://github.com/weecology/DeepTreeAttention/blob/main/src/models/Hang2020.py) and [Data preprocessing](https://github.com/weecology/DeepTreeAttention/blob/cae13f1e4271b5386e2379068f8239de3033ec40/src/utils.py#L59).