LSDTopoTools: open-source software for topographic analysis

Introduction

LSDTopoTools is software for performing topographic analysis of landscapes, with applications in hydrology, ecology, soil science, and geomorphology.

Firstly, we should tell you that LSD stands for Land Surface Dynamics, and is named after Land Surface Dynamics research cluster in the School of GeoSciences at the University of Edinburgh.

Today we’ll use the code to (hopefully) extract channel networks, get maps of slope and curvature across the landscape, and do some chi profile analysis.

The philosophy behind LSDTopoTools

LSDTopoTools is a software package designed to analyse landscapes for applications in geomorphology, hydrology, ecology and allied fields. It is not intended as a substitute for a GIS, but rather is designed to be a research and analysis tool that produces reproducible data. The motivations behind its development were:

To serve as a framework for implementing the latest developments in topographic analysis.
To serve as a framework for developing new topographic analysis techniques.
To serve as a framework for numerical modelling of landscapes (for hydrology, geomorphology and ecology).
To improve the speed and performance of topographic analysis versus other tools (e.g., commercial GIS software).
To enable reproducible topographic analysis in the research context.

The toolbox is organised around objects, which are used to store and manipulate specific kinds of data, and driver functions, which users write to interface with the objects.

Getting started

Installing LSDTopoTools

Hopefully everyone managed to get the code installed either on Windows using vagrant, Mac with docker, or Linux natively. If you had problems getting it to install, then just let me know and I will help out!

You can also find detailed installation instructions here.

Basic code structure and organisation

Our analyses are packaged into separate programs. They are run from the Linux shell. All of our programs operate the same way:

You keep the program and the data in two different directories.
The directory with the data also has a parameter file, which contains instructions for the program.
You have to tell the program where the data directory is, as well as the name of the parameter file. You do this from the command line.

Figure 1. Basic setup of LSDTopoTools. There are two main directories: the read directory where the data is stored, and the driver directory where the code is stored

We have already set up this directory structure for you with LSDTopoToolsSetup.py. The folder /LSDTopoTools/ has the structure:

/LSDTopoTools/
--Git_projects
--Topographic_projects

All of the code is stored in the Git_projects folder, and all of the DEMs can be stored in the Topographic_projects folder.

Get the example datasets

I have transformed the Santa Cruz dataset into the correct format for our code. All the DEMs need to be in the ENVI BIL format (see our instructions if you’re interested in how to transform the DEMs). You can get the 1m dataset of the Pozo catchment, and the 1 and 10 m DEMs of the whole Santa Cruz island here.

I suggest we make a new folder in the Topographic_projects directory called Santa_Cruz for today, and put all the DEMs in there. This shared folder also contains an example parameter file for extracting channel networks, getting slopes and curvatures, and doing chi profile analysis on the Pozo catchment. These parameter files also need to be stored in the Santa_Cruz folder.

Channel extraction

Our channel extraction tool bundles four methods of channel extraction. These are:

A rudimentary extraction using a drainage area threshold.
The Dreich method (Clubb et al., 2014).
The Pelletier (2013) method.
A geometric method combining elements of Geonet (Passalacqua et al., 2010) and Pelletier (2013) methods that we developed for Grieve et al. (2016) and Clubb et al. (2016) We call this the "Wiener" method (after the wiener filter used to preprocess the data).

These methods are run based on a common interface via the program channel_extraction_tool.exe.

Figure 2. Example channel network for Indian Creek, Ohio, extracted using our tools

The parameter files

Like most of LSDTopoTools, you run this program by directing it to a parameter file. The parameter file has a series of keywords. Our convention is to place the parameter file in the same directory as your data.

Note	The parameter file has a specific format, but the filename can be anything you want. We tend to use the extensions `.param` and `.driver` for these files, but you could use the extension `.MyDogSpot` if that tickled your fancy.

The parameter file has keywords followed by the : character. After that there is a space and the value.

Channel extraction parameter file format

Lines beginning with # are comments.
Keywords or phrases are followed by a colon (:).
The order of the keywords do not matter.
Keywords are not sensitive, but must match expected keywords.
If a keyword is not found, a default value is assigned.

Example channel extraction parameter file

Below is an example parameter file. This file is included in the shared folder with the DEM data.

# Parameters for channel extraction
# Comments are preceeded by the hash symbol
# Documentation can be found here:
# TBA

# These are parameters for the file i/o
# IMPORTANT: You MUST make the write directory: the code will not work if it doesn't exist.
read path: /opt/LSDTopoTools/LSDTopoTools/Topographic_projects/Santa_Cruz
write path: /opt/LSDTopoTools/LSDTopoTools/Topographic_projects/Santa_Cruz
read fname: Pozo_DTM
write fname: Pozo_DTM
channel heads fname: NULL

# Parameter for filling the DEM
min_slope_for_fill: 0.0001

# Parameters for selecting channels and basins

# threshold_contributing_pixels: 2500
print_area_threshold_channels: false
print_wiener_channels: true
print_pelletier_channels: false
print_dreich_channels: false

# write hillshade: true
print_stream_order_raster: true

For more information on all the different options available see the full channel extraction documentation.

Running the code

To run the channel_extraction_tool, first navigate to the channel extraction repository:

$ cd /opt/LSDTopoTools/LSDTopoTools/Git_projects/LSDTopoTools_ChannelExtraction

The objects which host the main functionality are stored here. We keep the driver functions, used to actually run the code, in the folder driver_functions_ChannelExtraction.

$ cd driver_functions_ChannelExtraction

Once you’re in the right folder, you can run the code with:

$ ./channel_extraction_tool.exe /path/to/data/folder name_of_parameter_file.param

For our example, it would be:

$ ./channel_extraction_tool.exe /opt/LSDTopoTools/LSDTopoTools/Topographic_projects/Santa_Cruz/ LSDTT_channel_extraction.param

Getting slope and curvature maps

Now we’ll do some basic topographic analysis using our program LSDTT_BasicMetrics. This program can extract topographic metrics like slope, aspect and curvature. The program is intended for everyday data processing. A number of these operations are available in GIS software, but we tend to prefer LSDTopoTools because the slope and curvature metrics in typical GIS software only uses information from the nearest pixels. We fit a polynomial surface from a neighbourhood of pixels. See Hurst et al., 2012, DOI: 10.1029/2011JF002057 and Grieve et al., 2016, doi:10.5194/esurf-4-627-2016 for the rationale.

Parameter files

Similar to the channel extraction, we also need to define a parameter file for running the slope and curvature analysis. An example for our DEM is shown below, which again needs to be in the same directory as the DEM:

# Parameters for extracting simple surface metrics
# Comments are preceded by the hash symbol

# These are parameters for the file i/o
# IMPORTANT: You MUST make the write directory: the code will not work if it doesn't exist.
read path: /opt/LSDTopoTools/LSDTopoTools/Topographic_projects/Santa_Cruz/
write path: /opt/LSDTopoTools/LSDTopoTools/Topographic_projects/Santa_Cruz/
read fname: Pozo_DTM
write fname: Pozo_DTM
channel heads fname: NULL

# Parameters for surface metrics
surface_fitting_radius: 7
print_slope: true
print_aspect: true
print_curvature: true
print_tangential_curvature: true

Run the code

To get the slope and curvature, we’re going to use a different repository than for the channel extraction. Navigate to the folder:

$ cd /opt/LSDTopoTools/LSDTopoTools/Git_projects/LSDTopoTools_AnalysisDriver/Analysis_driver/

You can then run the code with:

$ ./LSDTT_BasicMetrics.out /path/to/input/data/ name_of_parameter_file.param

For our example, we would have:

$ ./LSDTT_BasicMetrics.out /opt/LSDTopoTools/LSDTopoTools/Topographic_projects/Santa_Cruz/ LSDTT_BasicMetrics.param

Channel steepness analysis

The final thing we’ll do today is to do some channel steepness analysis. For background into channel profile analysis using both slope-area plots and \(\chi\), you can refer to our website which has a lot of detail on this. Here, I’ll just go through how to actually run the code.

Using the chi mapping tool

This runs in the same way as the channel extraction and the slope and curvature methods, using the LSDTopoTools_ChiMudd2014 repository. We’ll select a series of basins to perform the analysis on, and then print out some csv files and rasters which can be used to visualise channel steepness.

Again, we use a parameter file to run the chi mapping tool. It also has a series of keywords which are used to set which analyses we perform.

The important elements of the parameter file are:
1. test_drainage_boundaries: true This is actually the default. We put it in the parameter file here to highlight its importance. The chi coordinate is calculated integrating drainage area along the channel so if your basin is truncated by the edge of the DEM it means that your chi coordinate will be incorrect.
2. write hillshade: true. Does what is says on the tin. It means that you will print a hillshade raster. You really need to do this the first time you analyse a DEM (but you only need to do it once). The reason is that all your figures will look much nicer with a hillshade!
We are going to visualise the output using python.

Some comments on basin selection

In the chi mapping tool, we have several ways to select basins. We feel the default method is best (find_complete_basins_in_window). The options are:

find_complete_basins_in_window: This goes through several refining steps. If first checks every basin in the raster and selects basins within a size window between minimum_basin_size_pixels and maximum_basin_size_pixels. It then takes the resulting list of basins and removes any that are influenced by the edge of the DEM (to ensure drainage area of the basin is correct). Finally, it removes nested basins, so that in the end you have basins of approximately the same size, not influenced by the edge of the DEM, and with no nested basins.
find_largest_complete_basins: This is a somewhat old version that takes all basins draining to edge and works upstream from their outlets to find the largest sub-basin that is not influenced by the edge. To get this to work you MUST ALSO set find_complete_basins_in_window: false.
test_drainage_boundaries: If either find_complete_basins_in_window or find_largest_complete_basins are true then this is ignored. If not, then it eliminates any basin that is influenced by the edge of the DEM.
BaselevelJunctions_file: If this points to a file that includes a series of integers that refer to junction indices, it will load these indices. If the file doesn’t contain a series of integers the most likely result is that it will crash!

Figure 3. Example basins extracted for Xi’an province, China

Running the code

You run the chi_mapping_tool the same way as the other programs. Navigate to the driver directory for chi analysis:

$ cd /opt/LSDTopoTools/LSDTopoTools/Git_projects/LSDTopoTools_ChiMudd2014/driver_functions_MuddChi2014

Then run the code with the path to the input data and the parameter file:

$ ./chi_mapping_tool.exe /opt/LSDTopoTools/LSDTopoTools/Topographic_projects/Santa_Cruz/ LSDTT_chi_analysis.param

Example parameter file

Below is an example parameter file for the Santa Cruz dataset that will extract all the basins and produce chi data for each channel in each basin.

# Parameters for performing chi analysis
# Comments are preceded by the hash symbol
# Documentation can be found here:
# https://lsdtopotools.github.io/LSDTopoTools_ChiMudd2014/

# These are parameters for the file i/o
# IMPORTANT: You MUST make the write directory: the code will not work if it doesn't exist.
read path: /opt/LSDTopoTools/LSDTopoTools/Topographic_projects/Santa_Cruz/
write path: /opt/LSDTopoTools/LSDTopoTools/Topographic_projects/Santa_Cruz/
read fname: Pozo_DTM
write fname: Pozo_DTM
channel heads fname: Pozo_DTM_Dsources

# Parameter for filling the DEM
min_slope_for_fill: 0.0001

# Parameters for selecting channels and basins
threshold_contributing_pixels: 5000
minimum_basin_size_pixels: 50000
maximum_basin_size_pixels: 600000
test_drainage_boundaries: false
find_largest_complete_basins: false
find_complete_basins_in_window: true

# The data that you want printed to file
write hillshade: false
print_basin_raster: true

# Chi analysis options
print_chi_data_maps: true

Visualisation

If you look in the Santa_Cruz folder you’ll now see a lot of different rasters and CSV files that have been produced as a result of the analysis. We can load these into QGIS for a first look at the data. Some interesting things to look at might be:

Channel extraction: we output the results of the channel extraction as a stream order raster (with the extension _SO), a CSV of the full channel network (extension _CN), and a CSV with all of the channel heads (extension _sources). Each channel extraction algorithm has a different letter associated with it, so area threshold is AT, DrEICh is D, Wiener is W, and Pelletier is P.
Basic metrics: this is pretty self explanatory, but we printed out maps of curvature, slope, aspect, and tangential curvature from our polyfitting surface routines. Each one has the appropriate extension.
Chi profile analysis: we output a csv file which has all of the chi data for each channel, with the extension _chi_data_map.csv. You can load this into QGIS and see all of the data. We also output a raster of all the extracted basins which is useful for checking if you got the minimum and maximum basin sizes correct, with the extension _AllBasins.bil.

Python scripts

We’re also working on some python scripts for visualising the software. This repository is messy and not very well ordered yet, but can be cloned here.

You can read about our python tools here: link:LSDTT_visualisation.html