SapFlower: An automated tool for sap flow data preprocessing, gap filling, and analysis using deep learning
Description
SapFlower is an interactive and deep learning powered tool for processing sap flux data from thermal dissipation probes (TDP; Granier 1985, 1987). It is designed for (1) automated data cleaning, (2) state-of-the-art machine learning modeling for gap-filling, and (3) estimating of sap flux from raw TDP data. SapFlower integrates some important features from Baseliner 4.
Please note that some functions, such as data smoothing (for some cases), haven't been enabled.
Here supplementaryData contains raw sap flow data (96 sensors/trees), auto cleaned data, manually cleaned data, and matched cleaning percentage of auto cleaning and manual cleaning for users to explore the functions of SapFlower. Users need to download, install SapFlower and MATLAB Runtime, and run SapFlower. Users can click the Open Project button in Projection Configuration Tab, define the DataPath to create and load project. If you run it in Matlab, please use Matlab2024b.
Please report bugs and submit suggestions on GitHub page. For questions and requests, please email: jiaxinwang[at]vt.edu / jiaxinwang362[at]gmail.com.
When using SapFlower, we kindly request that you cite these articles, recognizing the hard work that went into collecting the data, developing the software, and the authors' willingness to make them publicly available.
Citation:
Wang, J., Renninger, H. J. (2025). SapFlower: An automated tool for sap flow data preprocessing, gap filling, and analysis using deep learning. New Phytologist, 70107. https://dx.doi.org/10.1111/nph.70107.
Notes
- Fixed bugs of printing out text when validating trained RNN models.
- Added feature scaling for modeling. Note: please use with caution.
- Users can modify Random Forest number of training cycles through changing EpochForTraining.
- Changed AutoClean to enable users to preview autoClean for current sensor.
- Added SapwoodArea Analysis tab to allow users scale and model sapwood area across the growing season.
- Added features to support SAPFLUXNET data.
- Added AutoClean to let users auto clean and save the cleaned data.
- Added more non-linear models for gap-filling.
- Added water use analysis using gap-filled sap flux data and scaled sapwood area data.
- Allowed users to view split training and validation data.
- Added scatter plot for visualizing predicted and true data.
- Fixed issues that users may not able to plot the original data and gap-fill for SAPFLUXNET data.
- Added functionality to plot and view water use data.
- Allowed users to enable or disenable ploting after model training.
- Disabled auto-clean before training for SAPFLUXNET data to ensure user can gap-fill.
- Added update check, users will be notified once new version is available.
- Added SapFlower Tutorial video.
- When working on SAPFLUXNET data, you may encounter below error: "Error reading the data file: Undefined function 'times' for input arguments of type 'cell'," which is because the calculation of baseline, and it will not affect the model training and gap-filling. (in debugging).
- Fixed bug: "Error reading the data file: Undefined function 'times' for input arguments of type 'cell'."
- Fixed bug that users may not be able to stop drawing lines in Sapwood area analysis tab after gap-filling.
- Added AIC and BIC for model evaludation.
- Added data loading monitoring process bar.
- Added sources of setting hyperparameters for deep learning models.
- Note: If you can successfully load data in, but it does not give you dTmax, then you need to either change your PAR threshold to determine nighttime or check your VPD data to see if they are reasonable. Because dTmax are identified based on nighttime and VPD.
Files
reFormatedSapFluxNetData.zip
Files
(7.8 GB)
| Name | Size | Download all |
|---|---|---|
|
md5:5da18af3439c778fb33a5d27e26ec268
|
808.6 MB | Preview Download |
|
md5:21b6710ced9c016064221bc4b44ad8db
|
1.0 MB | Download |
|
md5:640821824a51f39ed11cb4c52db4e1d8
|
3.5 GB | Preview Download |
|
md5:41b510b36168de787321de536d1e4d6b
|
3.1 MB | Download |
|
md5:d120e9f134ef614474a25c923961af05
|
2.8 GB | Preview Download |
|
md5:3ece5fe50e5d6e147c419af466f98a6d
|
2.0 MB | Preview Download |
|
md5:d4a577c46c46ef59a3ca1681d224fada
|
632.6 MB | Preview Download |
|
md5:a84c7f8bba6957ade609b1d592781d85
|
3.9 MB | Preview Download |
Additional details
Related works
- Is published in
- Journal article: 10.1111/nph.70107 (DOI)
Dates
- Updated
-
2025-04-01Uploaded all files for Mac version
Software
- Repository URL
- https://github.com/JiaxinWang123/SapFlower
- Programming language
- MATLAB
- Development Status
- Active