The BAMS/EDSS/Models-3 I/O API: User Manual

Contents

This web page is under construction.
As a matter of fact ALL web pages are under construction whether they admit it or not.
--Found on the web page of Do Wong Chu

Best Viewed With Any Browser

These pages are All Browser Enhancedtm. There are absolutely no <BLINK>ing text, unnecessary and ill-considered <TABLE>s nor <FRAME>s, nor obnoxious <FONT> COLORs, SIZEs, or FACEs at this <CENTER>, nor are there in-line <IMAGE>s that take forever to load.

Please report any problems to the author Carlie Coats (cjcoats@email.unc.edu).

This hyperdocument describes recent (second and third) versions of the EDSS./Models-3/BAMS I/O API. Copyright © 1992-2002 MCNC, © 1992-2013 Carlie J. Coats, Jr., and © 2003-2013 Baron Advanced Meteorological Systems, LLC. Distributed under the GNU Library Public License (LGPL) Version 2.1.

Both the original working prototype and this version are implemented on top of netCDF , (c) 1993 University Corporation for Atmospheric Research for persistent storage and PVM from Oak Ridge National Laborator for parallel model coupling.

NetCDF provides an application- and machine-independent interface to self-describing, multidimensional scientific data. It supports an abstract view of scientific data as a collection of named variables and their attributes, and provides access to data that is faithful to the abstraction.

PVM likewise provides an application- and machine-independent interfacethat permits a heterogeneous collection of computers hooked together by a network to be used as a single system for the support of coupled modeling.

Back to Contents

I/O API Tutorial

Back to Contents

Availability/Download of the BAMS EDSS/EPA&nbs EDSS/ Models-3 date-time manipulation routines
datshift
gregdate
greg2jul
juldate
jul2greg
juldiff
julshift
timeshift
p;Models-3 I/O API

Back to Contents

Changes from I/O API Version 1

Back to Contents

Conventions

Logical Names: environment variables and "setenv"
Standard environment variables: LOGFILE, SCENFILE, EXECUTION_ID, GRIDDESC, PROMPTFLAG
Grids and Coordinate Systems: both horizontal and vertical
Dates and Times and Time-increments: integers YYYYDDD:HHMMSS
Calling from Fortran: Just do it.
Calling from C: (but use f77 to link the ".o" files)
File Structure: Variables, Layers, and Time Steps
Files as Data Sets: NetCDF and native-binary disk files, PVM-mailbox Coupling-Mode virtual "files", and memory-resident buffered virtual "files".
Back to Contents

Data Types Supported

DICTIONARY: file structure repository
CUSTOM: user-defined structure
GRIDDED: 2D or 3D gridded data
BOUNDARY: boundary values for 2D or 3D gridded data
ID-REFERENCED: e.g., observations, county-aggregations,...
PROFILE: special type for rawinsonde data
GRID-NEST: nest of 2D or 3D grids of data
SPARSE MATRIX: uses "skyline-transpose" representation
Back to Contents

Object Libraries and Supported Machines

MODULEs and Public INCLUDE Files

MODULE M3UTILIO: parameters, data structures, INTERFACE blocks, and declarations.
MODULE MATXATTS: parameters and routines for map-transform metadata.
FDESC3: file description data structures
FDESC3: file description data structures
IODECL3: declarations and usage comments for I/O routines
PARMS3: "magic number "tokens and dimensioning parameters

New !! MODULE M3UTILIO: an f9x alternative/extension to the public INCLUDE files above.
Back to Contents

I/O API: Public Routines

Primary I/O Routines

CHECK3: is <timestep> available for <variable> from <file>?
CLOSE3: close and flush <file>
DDTVAR3: compute time derivative of <variable> from <file>
DESC3: describe <file>
INIT3: start up I/O API; return log-file unit
INTERP3 and INTERPX: interpolate <variable> from <file> to <time>, with windowing to user-selected column, row, and layer ranges for INTERPX()
OPEN3: open <file> for <mode / status>, using <file description> to create it, if necessary
READ3: read <timestep> of <layer> of <variable> from <file>
SHUT3: shut down I/O API; flush files to disk
SYNC3: Synchronize <file> with disk: flush output; re-read header from disk.
("Volatile" files auto-synch themselves...)
WRITE3: write <timestep> of <variable> to <file>
XTRACT3: read <window> of <timestep> of <variable> from <file>

Special (Convective-Event, Extra-Metadata) I/O Routines

KFINDX: reads the event index for the indicated cell from the indicated <KF event file>.
KFOPEN: open the indicated <KF event file> for <mode / status>, using <file description> to create it, if necessary.
KFREAD: reads data for the specified variable at the specified event, along with its grid location, starting date, starting time, and duration from the indicated <KF event file>.
KFWRITE: writes the specified event data for the specified cell, starting date, starting time, and duration to the specified <KF event file>, and return the corresponding event number.

INQATT3: Inquire the names, types, and sizes of netCDF attributes for <variable> in <file>.
RDATT3 and RDATTC: Read value of <attribute> of <variable> in <file>. Numeric attributes use RDATT3() and character-string attributes use RDATTC()
WRATT3 and WRATTC: write value of <attribute> for <variable> in <file>. Numeric attributes use WRATT3() and character-string attributes use WRATTC()

GETMTXATT(): get input or output grid description attributes for matrix files and return them as arguments.
SETMTXATT(): set or check input and output grid description attributes from arguments for matrix-files: if file is NEW, set the attributes; else check them.
CHKMTXATT(): check input or output grid description attributes for matrix-files against attributes from the argument list. Return .TRUE. if they match, .FALSE. otherwise.

Utility and Misc I/O Routines (consistency checking, etc.)

FILCHK3: check a file's file type and dimensions for consistency with values supplied by the caller.
GRDCHK3: check a file's map projection, horizontal and vertical grid definition parameters for consistency with values supplied by the caller.
IOPARMS3: return dimensioning parameters from INCLUDE-file PARMS3.EXT to allow consistency checks in model-level code.
READ4D: read <time step sequence > of <layer> of <variable> from <file>
WRITE4D: write <time step sequence > of <variable> to <file>

Back to Contents

Coordinate and Grid Related Routines

See also programs bcwndw, m3wndw, and projtool.

NOTE: Longitudes are represented in REAL or DOUBLE PRECISION within the range [-180,180], as consistent with duly-enacted ISO Standard 6709, treaty obligations since 1878, and several centuries of common usage. Beware that the WMO with its GRIB "standards" violates all of these. When dealing with WMO "longitudes", special-purpose modeling-code hacks will be required.

BILIN and BMATVEC: apply 4-band sparse matrix to vector (e.g., for bilinear interpolation; BMATVEC also does horizontal::vertical transpose).
CBARNES1 and CBARNESN: Use Barnes analysis to interpolate scattered data to a grid
DSCOORD: get description of <named coordinate system>
DSCGRID: get description of <named grid>
GRDCHK3: check a file's map projection, horizontal and vertical grid definition parameters for consistency with values supplied by the caller.
GTP0: driver routine GTPZ0 from US Geological Survey's official GCTP coordinate system manipulation library.
LAMBERT: convert among various map projections:
LL2UTM and UTM2LL: convert between LAT-LON and UTM coordinates
SETSPHERE and SPHEREDAT: select non-default geodetic spheroids for LAMBERT, LL2UT etc.
UNGRIDB and UNGRIDI: compute "ungridding"matrices for bilinear interpolation (e.g., with BILIN or BMATVEC) and for incidence-matrix re-gridding, respectively.
Back to Contents

Date-Time Manipulation Routines

See also programs DATSHIFT, gregdate and juldate for date-related calculations, and m3tproc and m3tshift for time-related file operations.
CURRSTEP and CURREC: find starting date and time, or record-number respectively, of the timestep containing <jdate-&-time>
DAYMON: find month and day-of-month for <jdate>
DT2STR: Construct string "HH:MM:SS Month DD, YYYY" for <jdate-&-time>
GETDATE: Prompt the user for a date; return the result according to EDSS/Models3 standard conventions for dates and times
GETDTTIME: get current wall-clock date and time
HHMMSS: construct string "HH:MM:SS" for <time>
ISDSTIME: TRUE if Daylight Savings Time in effect for <jdate>
JSTEP3: Compute record number (1,2,...) for <jdate:jtime> with respect to the time step sequence <sdate:stime:tstep>, or (-1) if <jdate:jtime> is not on that sequence.
JULIAN: find Julian day number (1...365,366) for <month> <day> <year>
LASTTIME: Find ending date & time for a specified time step sequence <SDATE:STIME:TSTEP:NSTEPS>
MMDDYY: construct string "Month DD, YYYY" for <jdate>
NEXTIME: update <jdate-&-time> by <timestep>
SEC2TIME: get Models-3 time representation for <seconds>
SECSDIFF: find time difference (seconds) between two <jdate-&-time> pairs
TIME2SEC: get number of seconds for <time>
WKDAY: get day-of-week (1...7) for <jdate>
YEAR4: Return 4-digit year for the given 2-digit year (e.g., YEAR4(97)==1997)
YR2DAY: get the year-to-day conversion factor (1/365 or 1/366) for the given year.
Back to Contents

Utility Routines

BILIN and BMATVEC: apply 4-band sparse matrix to vector (e.g., for bilinear interpolation). For layered data, BMATVEC performs interpolate-and-transpose, transforming data from I/O API layer order to (e.g.) SMOKE LAYPOINT computational order.
C/Fortran String Conversion: name2cstr(), fstr2cstr(), and cstr2fstr(): conversion back and forth between Fortran CHARACTER strings and C char * strings.
CBARNES1 and CBARNESN: Use Barnes analysis to interpolate scattered data to a grid
CHARACTER*2 FUNCTION CRLF(): returns character string "<CR><LF>" (DOS/Windows line terminator)
DMATVEC: applies diagonal (sparse) matrix to vector
DSCOORD: get description of <named coordinate system>
DSCGRID: get description of <named grid>
ENVDBLE: get DOUBLE PRECISION <value of logical name> from the environment
ENVINT: get INTEGER <value of logical name> from the environment
ENVREAL: get REAL <value of logical name> from the environment
ENVSTR: get CHARACTER-STRING <value of logical name> from the environment
ENVYN: get LOGICAL <value of logical name> from the environment
FINDC, FIND1, FIND2, FIND3, FIND4, FINDR1, FINDR2, FINDR3, FINDR4: find character-string <key>, INTEGER, or REAL <key-tuple> in sorted <key-table> or <keytuple-table>
GCD: greatest common divisor function
GET_ENDIAN: get host-machine byte order.
GETDFILE: open direct access Fortran file with <logical name> and <record length>, <formatting> and <read-write status>
GETEFILE: open sequential Fortran file with <logical name>, <formatting> and <read-write status>
GETDBLE: prompt user for DOUBLE and get response, with default and range checking
GETMENU: prompt user for menu choice, etc.
GETNUM: prompt user for INTEGER, etc.
GETREAL: prompt user for REAL, etc.
GETSTR: prompt user for CHARACTER STRING, etc.
GETYN: prompt user for "Yes-No" answer, etc.
GRIDOPS: select and compute various comparison operations
INDEX1: <unsorted-name-table> lookup for <character-string key>
JUNIT: return a "safe" <Fortran unit number>
LBLANK: number of leading blanks in <string>
LEN2: number of leading blanks in <string>; superseded by (more-generic) LBLANK().
INTLIST, REALIST, and STRLIST: Read in (and count) environment variables that are comma delimited lists of the indicated type.
LOCATC, LOCAT1, LOCAT2, LOCAT3, LOCAT4, LOCATR1, LOCATR2, LOCATR3, LOCATR4: find insertion-point for character-string <key> or INTEGER or REAL <key-tuple> into sorted <key-table> or <keytuple-table>, or -1 if the key is already present
LUSTR: left-justify and upcase contents of <string>;
M3ERR: <warning or error message>; with SHUT3() and EXIT(2) if fatal error; deprecated: superseded by M3EXIT() and M3WARN()
M3EXIT: <exit message> with <date and time>, SHUT3() and EXIT(<status>)
M3MESG, M3MSG2, M3PARAG, M3PROMPT, and M3FLUSH: Write 1-line or multi-line <message> to Fortran-program log; or do prompt-and-response processing.
M3WARN: <warning message> with <date and time>
NAMEVAL: get (CHARACTER-STRING) value of <environment variable> (for Fortran)
PCOEF: coefficients for the polynomial fitting given data
PMATVEC: applies sparse incidence matrix to vector
POLY: degree-<d> polynomial interpolation
PROMPTDFILE: Prompt user for <logical name> and open the corresponding file for Fortran direct access with the indicated <read-write status>, <formatting>, and <record length>
PROMPTFFILE: Prompt user for <logical name> and open the corresponding file for Fortran sequential access with the indicated <read-write status> and <formatting>
PROMPTMFILE: Prompt user for <logical name> and open the corresponding I/O API file for sequential access with the indicated <read-write status> and <formatting>
READSMET: read 1 hour's data window from surface met file.
SETENVVAR: Subroutine-call equivalent to csh command setenv name value
SMATVEC: applies sparse matrix to vector
SORTC, SORTI1, SORTI2, SORTI3, SORTI4, SORTR1, SORTR2, SORTR3, SORTR4 : sort <index table> acording to a character string key-table, or a <keytuple-table> of INTEGERs or REALs.
SCANINT, STR2INT, STR2REAL, and STR2DBLE: Convert a numeric <string> into an INTEGER, REAL, or DOUBLE-PRECISION value, with error-checking.
SYSTEMF(): Fortran wrapper around standard system() C-library call: execute shell-commands and other programs from within Fortran code.
TRIMLEN: length of <string>, not counting trailing blanks
UNGRIDB: computes "ungridding"matrices for bilinear interpolation
UPCASE: make <string> into ALL CAPS
Back to Contents

I/O API Related Programs and Examples

Some Sample Programs using the I/O API:

ncdump: print file as structured ASCII; from NCAR

MCPL I/O API output module for MM5: Generate I/O API output files directly from within MM5, for analysis, emissions modeling, air quality modeling, and model coupling.

airs2m3: Import air quality monitor data from an AIRS AMP350-format ASCII file and put it into an I/O API "observational data" file.

bcwndw: extract data data from a gridded file to the boundary of a subgrid window (see m3wndw for extracting to the window itself)

m3cple: copy to the same grid, or interpolate to another grid a time sequence of all variables from a source file to a target file, under the optional control of an I/O API coupling-mode "synch file".
m3combo: Compute linear combinations of sets of variables from an I/O API input file, and write the resulting variables to an I/O API output file.
m3diff: compute statistics for pairs of variables and for the results of applying various comparison ("differencing") operations to those variables in a pair of files.
m3edhdr: edit header attributes/file descriptive parameters
m3merge: Merges selected variables from a set of input files for a specified time period, and writes them to a single output file, with optional variable-renaming in the process.
m3pair: build ASCII file of paired values for two variables from two files, within a user-selected window into the grid, according to user specifications,
m3stat: compute statistics for variables in a file
m3tproc: compute time period aggregates (e.g., 08:00-16:00 gridded daily maxima) and write them to an output file
m3tshift: copy/time shift data from a file
m3wndw: window data from a gridded file to a subgrid (see bcwndw for extracting to the boundary of the subgrdi window)
m3xtract: extract a subset of variables from a file for <time interval>
m3fake: build a file according to user specifications, filled with dummy data, or data read in from a set of user-supplied files.
m4filter: convert first-edition Models-3 files to current version
mtxblend: Use a sparse-matrix file to interpolate/transform data from an input file to the grid of a "base" file and to merge it with data from the "base" file.
mtxbuild: Build a sparse-matrix transform file from user-supplied ASCII coefficient inputs.
mtxcalc: Build a grid-to-grid sparse-matrix transform file using a sub-sampling algorithm.
mtxcple: Use a sparse-matrix file to interpolate a time sequence of all variables from a source file to a target file, under the optional control of an I/O API coupling-mode "synch file".
presterp: Interpolate from a 3-D sigma-coordinate file to a new 3-D pressure-coordinate file, using coefficients from PRES_CRO_3D.
selmrg2d: select multiple 2D layer/variable combinations from multiple gridded input files, and write result to merged 2D gridded output file

vertot: compute vertical-column totals of variables in a file

datshift: Takes calendar date (form YYYYMMDD) and a number of days D, and reports the date D days later.
gregdate: compute calendar style date "Month DD, YYYY", day-of-week (Sunday, Monday, ..., Saturday) and whether or not daylight savings time is in effect from Julian date YYYYDDD, or "yesterday", "today", or "tomorrow".
juldate: compute Julian date YYYYDDD and day-of-week (Sunday, Monday, ..., Saturday) and whether or not daylight savings time is in effect from calendar style date "Month DD, YYYY", or "yesterday", "today", or "tomorrow".

projtool: Perform coordinate conversions and grid-related computations for LAT-LON, Lambert Conformal Conic, UTM, Polar Stereographic, General Transverse Mercator, Equatorial Mercator, and Albers Conic Equal Area coordinate systems.
Supersedes utmtool (below).
utmtool: Perform coordinate conversions and grid-related computations for Lat-Lon, Lambert, and UTM coordinate systems.
vertintegral: Compute vertical-column integral for each CMAQ conc-file variable, and write the output to a 1-layer GRIDDED file.

Programs for date-manipulation
These programs echo a single number to standard output for use in scripting, or if invoked with command line argument --help display a usage-screen.

greg2jul: Convert Gregorian-style dates to Julian-style and echo the result.
jul2greg: Convert Julian-style dates to Gregorian-style and echo the result.
juldiff: Compute the number of days between two Julian dates and echo the result.
julshift: Shift a Julian date by a specified number of days and echo the result.
Back to Contents

Python ioapiTools

Alex Zubrow of the University of Chicago just announced release of a beta version of his ioapiTools. For those of you who didn't get a chance to hear his talk at the last CMAS conference, ioapiTools is a python interface to I/O API data. It allows you to easily import the data into python, have full access to the I/O API metadata, use the full breadth of python's scientific libraries, create plots, and write out your data to new I/O API or netCDF files.

See the ioapiTools site:

http://www-pcmdi.llnl.gov/software-portal/Members/azubrow/ioapiTools/index_html

Back to Contents

I/O API Troubleshooting

Back to Contents

I/O API Requirements Document

Back to Contents

Notices and Acknowledgements

Back to Contents

netCDF User's Guide (at NCAR)

Back to Contents

PVM: Parallel VirtualMachine (at Oak Ridge)

Back to Contents

UP: I/O API home page

Back to Contents

Send comments to

Carlie J. Coats, Jr.
cjcoats@email.unc.edu 

    $Id: AA.html 1925 2012-06-12 14:39:17Z coats@borel $