Moored acoustic Doppler current profiler data reduction and database applications

One problem inherent in collecting vast amounts of oceanographic data such as acoustic Doppler current profiler (ADCP) data is processing, editing, and storing these data so that they may be retrieved in a useful form. In this paper the authors outline the processing phase and discuss the underlying problems associated with preparing a final moored ADCP data set for end users. In this case the end user is the U.S. Navy, and the data may be used in various ways. For most purposes, edited raw data yield vital information on tides, storm surges, and seasonal variations. Another use for these data is to develop and constrain models of basin circulation. For this purpose the data must be further refined by the end user. Processing of raw ADCP data has been useful in obtaining ground truth for ocean circulation models in certain areas of the Yellow Sea and East China Sea. Examples in these areas use processed ADCP data to study movement of pollutants with the motion of the water mass.


INTRODUCTION
A typical deployment of Self-Contained Trawl-Resistant Moored Acoustic Doppler Current Profiler (ADCP) lasts approximately six months. Data are collected in binary format to optimize memory use and battery duration. After the instrument is recovered, data are downloaded to a laptop computer and reviewed visually with proprietary software, TRANSECT, 1 for completeness and anomalous occurrences such as rotations by trawls, obstructions by nets, debris, or biofouling, and sensor damage caused by salt water encroachment, battery corrosion, or direct impact. In this initial editing phase determinations such as start/end time, depth corrections, bin trimming, and magnetic variation are made and applied in the same step in which the data are converted to ASCII output.

DATA REDUCTION
First, the magnetic variation at the meter location is found by running a program, MAGMODEL, which uses coefficients from the World Magnetic Model 1995. Then, the depth at the meter deployment location is checked for accuracy. This depth should be from the best possible source (i.e., pressureaveraged value, sonar centerbeam depth, or charts). The depth and magnetic variation information is saved in a configuration file.
Next, extraneous data at the beginning and end of the data set must be truncated unless the meter is set up to turn on/off at a predetermined interval when submerged. Otherwise, initialization occurs on deck, when the meter functionality is tested. Temperature, roll, pitch, and heading will change while the meter awaits deployment on deck. After the meter is deployed on the bottom, these parameters will stabilize. This first time increment corresponds to the first ensemble of depth bins which have useful velocity data. This ensemble is used in a processing step called subsectioning, which truncates the data. Unless battery power was depleted, the recovery time of the meter is found by looking for the same parameters to destabilize as the meter rises off the bottom after acoustic release. The corresponding ensemble number before these parameters destabilize is also used in the subsectioning step.
At this point, bad bins or depth cells are trimmed in the scaling step. During deployment the bin size, typically 2-8 meters, is configured based on desired resolution, water column depth, frequency range, and battery time constraints. Determination of bad bins is made by visual inspection of graphical or tabular data. Bins near the surface loose coherent velocity structure due to overloading the Doppler returns in side-lobe reflections off the surface. Typically the upper 15% of the data is bad with a 30 o beam configuration. If the acoustic energy does not reach the surface, the data will diffuse, and undulating returns dependent on current velocity (higher velocity equals more energy return) in the upper bins will be observed. The surface bin truncation is based on the depth to which coherent data structure extends. This correction is applied by changing the upper depth in the scale function and observing the effects on the data.
When we are satisfied that only coherent bins are kept, an ASCII output file is created. Bin truncation is a subjective process and a good look at a representative sample of the data should be made before "chopping." The binary data are processed from the first chosen ensemble to last chosen ensemble which is input in the subsectioning step. The depth interval is chosen for bin chopping in the scaling step. The total depth and magnetic variation from the configuration file are then used to create a reduced output file.

DATA PROCESSING
Once the data have been reduced under TRANSECT, the ASCII output file is generated, and then transferred to local networks via LapLink. The output file contains information in a header and array format that is repetitive for each ensemble. This header section contains readings for pitch and roll, corrected heading, bottom temperature, and bottom and water track velocities. There are also depth readings for each beam, values for distance traveled north and east, and position data for each ensemble. Values for each corresponding depth cell/bin are arranged under the header rows (information such as velocity magnitude; velocity direction; east, north, and vertical velocity components; and echo intensity by beam).
This file becomes the input file for several programs that are intending to reduce erroneous data. The first program, PASS1, scans the data by ensemble searching for values in the %GOOD field. The %GOOD is a statistical representation of goodness based on the signal to noise threshold. PASS1 flags any values under 25% or other predetermined percentage and produces a file containing a listing of the current velocities U=east, V=north, and W=vertical that are suspected to be erroneous. The vector components of the velocities in this listing of possible rejections are compared to neighboring bins' velocities within the same ensemble. By inspecting this listing we determine whether velocities of these bins have anomalous values and if there is more than one consecutive bad bin.
When reviewing this time-series data, it is recommended that velocities from one ensemble to the next be compared.
To minimize data gaps a second filtering program, PASS2, interpolates the velocities if there is only one or no more than two consecutive bad bins. The interpolation is done using a Spline-Method; the new value is substituted into the bad field, and an editing flag is set. When the condition of more than two consecutive bad bins arises, the program will then keep the initial flag and introduce a data gap. At this point the PASS2 data are groomed, filtered and ready to be format converted for the final editing stage using a conversion routine, BBREFORM.

FINAL EDIT
For the final editing we have developed a program in an X-Windows environment written in PV-WAVE. 2 This program, called BBEDIT, displays 3-D images of the east and north current velocity (U and V, respectively) of the water column. The output data file from BBREFORM becomes the new input file and is executed once BBEDIT is running. The initial display in Fig. 1 is the Graphical User Interface (GUI), which prompts the users for the proposed naming convention based on the meter identification, instrument serial number, input data file, and desired output data file. Once the desired meter is selected, input parameters are configured. These parameters consist of maximum speed for plotting scale, number of ensembles per window, number of ensembles per zoom window, and the percentage of the plot overlapped.
After GUI inputs are applied, three new windows are generated. The first two are graphical displays of the velocity components U and V, and the third is the menu that drives the editing. The X-component of U and V windows represents the ensemble in a time-series projection; each ensemble represents a 15-minute interval. The Y-component indicates the number of bins which in this case, equals 8 meters. The Z-component represents the velocity in centimeters/sec of the vector current. A graphical illustration in Fig. 2a and Fig. 2b demonstrates the U and V components of a moored current meter survey that was collected in the Yellow Sea.
The menu window is designed to have full control of the display images of the U and V components. Initially the images are set to a defined angle of rotation. Through the GUI, the 3-D image can be rotated about its X and Z axis independently. Its angle of rotation is indicated in the menu screen, allowing the scientist to control the fields of view and edit the questionable data.
The displayed data in the 3-D graph show ensemble numbers along the X-axis, bin numbers along the Y-axis, and negative and positive velocities from zero in cm/sec along the Z-axis. This data set starts at ensemble number ten on the X-axis. The previous ensembles were removed using the TRANSECT program. The bin numbers, represented on the Y-axis, are illustrated in negative numbers, where zero represents the surface of the ocean, and moving down the negative scale indicates increasing depth toward the seafloor. In this case the data start at bin number -15, where the previous 14 bins were also removed by the TRANSECT routine. Bad data are identified by simply observing the formed contours for anomalies. The U and V components can be displayed in a shaded color pattern, where the color code corresponds to the velocity intensity along the water column. In Fig. 2b an example of erroneous velocities along the entire water column is shown in the 95 th ensemble on V component display. All the bins within the ensemble spike out of the normal. From the menu screen the next step is to truncate within the V component. After toggling "TRUNCATE V" and "ENSEMBLES," a screen opens that allows the user to select with the mouse the entire ensemble and flag it for rejection. The data are then smoothed with the Splines routine, generating a new display with these errors removed. The same can be done for individual or small groups of bins within an ensemble. Fig. 3 illustrates a zoom area of the Vcomponent, where a select group of bins has an excessive variation in velocity. Using the same procedure followed with the ensembles, bins can also be truncated, affecting only the intended region. When satisfied with the edits in the display window, we can toggle to the next window, displaying the next series of ensembles. Once we have reviewed the entire data set, it is then exported to be loaded into databases with C-routines embedded with Structured Query Language (SQL).

APPLICATIONS
A useful tool in the study of ocean currents is to model the movement of a tracer with the motion of the water mass. A hot topic recently has been the problem of pollution and its effect on fisheries in the Yellow Sea and East China Sea. Also of importance is the current debate on whether the source of the Yellow Sea Warm Current (YSWC) and the Tsushima Warm Current (TSWC) is the Kuroshio or the Taiwan Warm Current (TWC). Traditional thought has the source as the Kuroshio (Uda, 1934;Nitani, 1972), but recently a group of scientists have suggested that the source of YSWC and TSWC is the TWC (Beardsley et al., 1985;Fang et al., 1991).
The Naval Research Laboratory developed the Yellow Sea Model (YSModel), including the East China Sea in the model domain. Presently the model outputs temperature, salinity, and velocity data. With the YSModel velocity fields and ADCP processed data as our gridded velocity field, we have applied a tracer experiment using a numerical scheme (Hur et al., 1996). The scheme we have chosen solves the continuity equation for the distribution of a tracer quantity in order to investigate the flow pattern of the water mass at a choke point such as the Taiwan Strait. This tracer is an arbitrary constant quantity representing any physical material which is conservative, that is, not changing with time.
ADCP DATA ASSIMILATION ADCP data are used in two ways in this application. The processed data are used to confirm and to check the YSModel output velocity values at the grid point which corresponds to the location of a current meter deployed during the same time period. In this study the ADCP processed data have been assimilated into the model data at the available grid point of the velocity field. In order to get the relatively long-term velocity for assimilating ADCP data into our model, the tidal signal is removed from the processed ADCP data. This is accomplished by discarding the tidal signal with low-pass filtering of the 15-minute time-series components of velocity. To generate weight functions for the low-pass digital filter, methods are adapted from Martin (1957) and Davis (1971). An example of the east velocity component of the raw data is displayed in Fig. 4. Fig. 5 shows the results of the filtered data, where longer term changes in velocity are now evident. The periods of change in velocity which occur over a few days are probably attributable to storm surges. The north component is processed in a similar manner. Filtered data are resampled at an hourly rate and compared to model data at the meter deployment locations. Fig. 6 shows a comparison of correlation in direction is evident from this plot, since a majority of the velocity values plot in the positive region. Storm surges which pass through the filter can explain the variation in magnitude in the comparison. In the tracer experiment the real processed ADCP data are injected into the model data field at the given locations and used to and used to drive the numerical model. (1) ∂t Density (ρ) in the governing continuity equation represents any tracer which is a conservative quantity.
Physical processes affecting trace spreading are diffusion and advection. Vertical motion, which is relatively small compared to horizontal motion as shown by ADCP data results, is ignored. Once our velocity field is known, we can solve for the density (ρ).
Various numerical schemes are available for this kind of hyperbolic equation. If the velocity field is given by constants whose values are less than one, this equation can be regarded as a kind of advection equation. Forward in Time and Backward in Space (FTBS) is the simplest and easiest scheme to apply on the advection equation, which is conditionally stable, but is limited by an accuracy of the order O(∆t). In order to determine the movement of the tracer in the model domain, we solve Equation (1) using Central in Time and Central in Space (CTCS), which has an accuracy of the order O(∆t 2 ). The tracer is introduced in a desired location of interest as an initial condition shown in Fig. 8. The CTCS model then computes how the tracer moves in time. Using a ∆t of 1 hour, a ∆x of 20 to 25 km, and a ∆y of 27 km, we calculate our solutions of the finite difference approximations to the continuity equation. Equation (1) can be rewritten as follows, (2) ∆y

TRACER EXPERIMENT RESULTS
Once the model is set up correctly, various interesting oceanographic issues can be explored. The example here examines examines the origin of the YSWC and TWC as the major source of the warm saline water in the Yellow Sea and Japan Sea. The tracer is set at the point of bifurcation between the Kuroshio and TWC , which is near the mouth of the Taiwan Strait. After 9 days the solution for Equation 2 has the tracer distribution moving generally northeastward, as shown in Fig. 8, which is coincident with the new theory.
After 18 days the tracer shows a slight drift toward the north in the upper part of the distribution as shown in the next case in Fig. 8. After 27 days the model velocity data is complete and the final solution for the tracer distribution is shown as the last case in Fig. 8. Current efforts are being made to increase model data availability to about a year and to assimilate more ADCP data into the model field. This will make the water mass movement experiment more meaningful and therefore more useful.

SUMMARY
As the requirement for acquiring large amounts of ADCP data grows the necessity for processing, editing, and storing the data will become more important. The methods presented here prove to be a useful and complete way in which to meet the requirements. Future applications involving these data will determine modifications to these processes. Use of the final stored products can satisfy a variety of users needs.