mbm_makedatalist
Section:  MB-System 5.0  (1)
Updated:  22 June 2022 
Index


 
NAME

mbm_makedatalist - Macro to generate a datalist of the swath
files in a specified directory.

 
VERSION

Version 5.0

 
SYNOPSIS

mbm_makedatalist [-Bsize -Fformat -Idirectory -L
-Odatalist -P -Ssuffix -T -H -V]

 
DESCRIPTION

Mbm_makedatalist is a macro to generate an MB-System datalist file
referencing all identifiable swath files in the specified target directory.

Datalists are fundamental structures in MB-System workflows because they
allow programs to operate on sets of swath data files.
Datalist files are text lists of swath data files and their format ids with each
file entry taking up a single line. These lists may contain references to other
datalists, making them recursive. Datalists may also contain comments and parsing
directives that, for example, determine whether parsing returns references to
raw or processed data files. See the MB-System manual page for details
on the format and structure of datalists.

This macro is used to easily construct a datalist referencing swath files in a
directory, often as part of setting up the MB-System processing environment
for a new seafloor mapping dataset. By default, mbm_makedatalist tests
all of the filenames in the local directory for adherence to the MB-System
swath file naming convention (see the mbsystem manual page for details),
and makes an output datalist including all of the files that can be recognized
as swath data. This can include files with suffixes such as ".mb88", that are
recognized as format 88 (Reson s7k) files, and files with recognized vendor
format suffixes, such as "_raw.all" or ".all" for old and recent Kongsberg
multibeam data, respectively. The suffix of the files to be considered can be
specified with the -Ssuffix option; under most circumstances this is
the best approach as it prevents mbm_makedatalist from including files that
are not intended for inclusion by the user yet satisfy one of the many recognized
naming conventions. By default, mbm_makedatalist infers the format id
of included files using the naming conventions. The format id can be specified
using the -Fformat option.

MB-System differentiates between raw and processed files by inserting the
letter "p" into the processed file name immediately before the filename suffix.
For instance, for Reson 7k data the raw files will have file suffix of *.mb88,
and the processed files generated by mbprocess will end with *p.mb88.
When using mbm_makedatalist to generate a datalist by specifying a suffix,
processed files can be excluded from the datalist by also specifying the -P
option. Thus it is possible to generate a datalist of raw files in a directory
containing both raw and processed file.

The output datalist will be placed in the current working
directory. The output datalist is named datalist.mb-1
by default, but can be specified with the -O option.
If no target directory is specified with the -I
option, then the current working directory is used. In order
for swath files to be recognized and included in the datalist,
they must conform to the MB-System swath file naming convention
(see the mbsystem manual page for details).

Most seafloor mapping sonars and datalogging packages produce original swath
data files with names constructed so that standard directory listings are in
time order. However, Kongsberg multibeam data filenames usually start with a survey
file number that restarts at 0001 whenever datalogging is stopped and restarted.
Consequently, when data from multiple instances of logging are present in a
single directory, the simple listing of files will not in general be in time
order. If the file suffix is declared to be ".all", ".ALL", ".kmall", or ".KMALL" using the -S
option, then mbm_makedatalist will attempt to sort the swath files into
time order before generating the final datalist. This behavior can be disabled
by specifying the -T option. Kongsberg dataloggers often create junk data
files with names such as "9999.all" or "9999.ALL"; these filenames are ignored.

 
MB-SYSTEM AUTHORSHIP

David W. Caress




  Monterey Bay Aquarium Research Institute



Dale N. Chayes




  Center for Coastal and Ocean Mapping




  University of New Hampshire



Christian do Santos Ferreira




  MARUM - Center for Marine Environmental Sciences




  University of Bremen

 
OPTIONS


-B


size



This option sets a size threshold in kilobytes below which a file will be
ignored and not included in the output datalist file. 
-F


format



By default, mbm_makedatalist infers the format id
of included files using the naming conventions. This option specifies the
format id that is assigned.
-H


This "help" flag cause the program to print out a description
of its operation and then exit immediately.
-I


directory



Sets the path of the target directory.
-L





Causes the last file in a directory listing to be omitted from the output datalist.
The intent of this option is to exclude the most recent file in an actively data
logging context, since the most recent file is still being logged. This presumes
that filenames sort in time order as listed by the ls command, an assumption
that does not hold for all file naming conditions or if the -T option has
been used to disable time ordering.
-O


datalist



Sets the name of the output datalist file. Default: datalist.mb-1
-P





Suppresses the inclusion of processed files in the output datalist (e.g.
*p.mb88).
-S


suffix



Sets the filename suffix of swath files considered for inclusion in the output
datalist. Under most circumstances using this option explicitly is the best
approach as it prevents mbm_makedatalist from including files that
are not intended for inclusion by the user yet satisfy one of the many recognized
naming conventions.
If the file suffix declared with this option corresponds to one of the formats
associated with Kongsberg multibeam sonar data, then mbm_makedatalist will
attempt to sort the swath files into time order before generating the final datalist.
This behavior applies to files with the ".all" or ".ALL" suffix used for raw logged
Kongsberg files, and also to files with suffixes following the MB-System
convention (e.g. ".mb56", ".mb57", ".mb58", ".mb59" suffixes). This behavior is
disabled by the -T option.
-T





By default, when the -S option is used to specify a file suffix corresponding
to Kongsberg multibeam data, mbm_makedatalist attempts to sort the data files
into time order before outputting the datalist. This option disables that behavior.
-V


The -V option causes mbm_makedatalist to print out status messages.


 
EXAMPLES

Suppose we have a set of swath files in a directory and we
want to construct a datalist referring to those files. A simple
directory listing shows four files:




        20051019_154030.mb88


        20051019_162540.mb88


        20051019_171756.mb88


        junk.txt




of which three are identifiable as swath files because of the
".mb88" file suffix. Running mbm_makedatalist in this
directory with no arguments other than verbosity:




        mbm_makedatalist -V




produces a datalist file named datalist.mb-1. The contents of
this file are:




        20051019_154030.mb88 88


        20051019_162540.mb88 88


        20051019_171756.mb88 88




where the file junk.txt has been ignored.

 
SEE ALSO

mbsystem(1), mbdatalist(1), mbinfo(1)

 
BUGS

Only fake bugs here.



 
Index

NAME

VERSION

SYNOPSIS

DESCRIPTION

MB-SYSTEM AUTHORSHIP

OPTIONS

EXAMPLES

SEE ALSO

BUGS