There are versions of the I/O API callable from both
Fortran and from
C . This document describes the Fortran
interface; the C interface is very
similar. The major difference between the two are that Fortran LOGICAL
functions returning .TRUE. or .FALSE. correspond to C functions returning
1 or 0, Fortran ".EXT"
include-files correspond one-to-one to
C ".h"
include files, and the C calls
look much like the Fortran calls, except that file descriptions are
passed via pointers to data structures typedeffed in fdesc3.h
instead of a COMMON
found in FDESC3.EXT
There are 9 routines you'll need to know:
INIT3() and SHUT3()
to start up and shut down
the I/O API;
OPEN3() to open files;
DESC3() to get file descriptions;
READ3(), INTERP3(),
XTRACT3(), and DDTVAR3()
to access data values; and WRITE3()
to store data to files. Additionally,
M3EXIT() is a useful utility routine which works
with the I/O API to generate exit (or error) messages to the
program log,
call SHUT3(), and then terminate the program with a user-supplied
exit status (which should be 0 for success and nonzero for failure).
There are three INCLUDE files you'll have to worry about.
Each has extensive in-line documentation describing how it is used.
PARMS3.EXT
contains the dimensioning parameters and the "magic number"
parameters used to control the operation of various routines in the I/O API.
FDESC3.EXT has
commons that hold file descriptions (more about that later); it needs
PARMS3.EXT
for its own dimensioning. Finally,
IODECL3.EXT
has declarations and usage comments for the various functions in the
I/O API (it's really a short manual on the I/O API in its own right).
csh setenv
command. Additionally, there are four
standard logical names:
LOGFILE, SCENFILE, and EXECUTION_ID, which may be used for the
program log file, scenario-description file, and execution identifier,
and GRIDDESC , for ASCII grid and
coordinate system databases used by utility routine
DSCGRID().
For programming purposes, the significant facts are that names should
not contain blanks (except at the end: "foo "
is OK;
"f oo"
is not), and are at most 16 characters long.
When you run a program that uses the I/O API, you begin with a
sequence of setenv
commands that set the values for the
program's logical names, much as you begin a (normal) Cray Fortran
program with a sequence of ASSIGN
commands for its files.
For example, if "myprogram
" has logical names
"foo
" and "bar
" that I
want to connect up to files "somedata.mymodel
"
and "otherdata.whatever
" from directory
"/tmp/mydir
", the script for the program would
look something like:
... setenv foo /tmp/mydir/somedata.mymodel setenv bar /tmp/mydir/otherdata.whatever setenv qux "/tmp/mydir/volatilestuff.mymodel -v" setenv LOGFILE /tmp/mydir/mymodel.log setenv SCENFILE /tmp/mydir/test17a.description setenv EXECUTION_ID TEST17A /user/mydir/myprogram ...VOLATILE files are indicated by a trailing
-v
in the setenv command, as
above, in order to tell the I/O API to perform disk-synch
operations before every input and after every output operation on that
file. Such files can be accessed by other programs while the
generating program is still running, and are readable even if it fails
to do a SHUT3() or
M3EXIT() (or if it crashes unexpectedly).
BUFFERED virtual files can be used to
provide safe, structured exchange of data -- of "gridded",
"boundary", or "custom" types only -- between
different modules in the same program. If you setenv
the value of a logical name to the value BUFFERED
, as
given below:
... setenv qux BUFFERED ... /user/mydir/myprogram ...then the I/O API will establish in-memory buffers and time indexing for "qux" instead of creating a physical file on disk. One module can then use WRITE3() (see below) to export data for sharing, which other modules would then use READ3() or INTERP3() to import. Note that since these routines associate the data with its simulation date-and-time, the system will notice the error (and warn the user) if you attempt to get and use data before it has been produced. Note also that by changing the setenv in the script between "BUFFERED" and a physical file-name, you can change between efficient data sharing between modules and high-resolution instrumentation of the data being shared, without changing the underlying program at all.
COUPLING-MODE virtual files can be used to provide PVM-based data exchange between cooperating programs using exactly the unchanged I/O&API programming interface, with the kind of name-based direct-access semantics that provides, with the extra scheduling condition that requests for data that has not yet been written put the requester to sleep until it becomes available (at which time the requester is awaked and given the requested data). The decision of which files are disk-based and which are COUPLING-MODE virtual files is also made by setenv commands at program-launch, the value being of the form "virtual <communications-channel-name>":
... setenv zok "virtual CHEM_CONC_3D_G3" /user/mydir/myprogram ...
Except for INIT3(), all of the I/O API routines are LOGICAL functions returning TRUE for success and FALSE for failure.
There are a number of dimensioning parameters and "magic number" token values for the I/O API. Throughout the I/O API, names (logical file names, variable names, and units) are character strings with maximum length NAMLEN3 = 16; descriptions are either one or MXDESC3 = 60 lines of length at most MXDLIN3 = 80. The I/O API currently supports up to MXFILE3 = 50 open files, each with up to MXVARS3 = 120 variables.
NUMIDS
in the
ID-referenced data structure, below) . Where such system-defined
variables are present, the operations READ3() and WRITE3() act on entire
time steps (all variables) at once; otherwise, they can be used to store
or retrieve time steps of individual variables one at a time. There are
moderate performance advantages to writing the variables for a time step
in the same order that they appear in the file description, and for writing
the time steps in consecutive order; however, this is not required by the
I/O API (which permits any access order to the data, for both read and
write operations). The structural types are as follows, together with
declarations for sample time step records of these data types:
(In the examples, declarations are given for M3REAL variables in terms
of REAL*4, etc, instead of merely REAL, to protect you in the cases that
your compiler has a "-r8" flag, etc., which silently changes all
REALs from 4-byte to 8-byte -- and causes you accidentally to be linked
with an incompatible version of the library; if you never use this flag,
don't worry.) The structure-types are:
...(SIZE is a fixed, user-defined dimension:) REAL*4 ARRAY( SIZE, NLAYS, NVARS )
FTYPE3D, TSTEP3D, NCOLS3D, NROWS3D, NLAYS3D, NVARS3D, NTHIK3D, GDTYP3D, P_ALP3D, P_BET3D, P_GAM3D, XORIG3D, YORIG3D, XCELL3D, YCELL3D, GDNAM3D, XCENT3D, YCENT3D, VNAME3D, UNITS3D, VDESC3D
REAL*4 ARRAY( NCOLS, NROWS, NLAYS, NVARS )
NTHIK
cells wide (where you may use a
negative NTHIK
to indicate an internal perimeter such as
is used by ROM and RADM). The boundary array is dimensioned
as follows in terms of the dimensions NCOLS and NROWS for the array
it surrounds:
...(SIZE = ABS( NTHIK )*(2*NCOLS + 2*NROWS +4*NTHIK) REAL*4 ARRAY( SIZE, NLAYS, NVARS )There are accompanying diagrams illustrating the data layout for various cases of NTHIK:
... INTEGER*4 MAXID ! max permitted # of sites PARAMETER ( MAXID = 100 ) ... INTEGER*4 NUMIDS ! number of actual sites INTEGER*4 IDLIST( MAXID ) ! list of site ID's REAL*4 XLON ( MAXID ) ! first variable in file REAL*4 YLAT ( MAXID ) ! second variable REAL*4 TK ( MAXID ) ! third variable REAL*4 PRES ( MAXID ) ! fourth variable REAL*4 RH ( MAXID ) ! fifth (last) variable COMMON /FOO/ NUMIDS, IDLIST, XLON, YLAT, TK, PRES, RHNVARS3D is 5. The dimension
MAXID
maps into the NROWS3D
dimension in the file description data structure FDESC3.EXT,
for use by OPEN3() or DESC3(). To read or write this data, put the
first element, NUMIDS,
of this common in the
array
spot of READ3(),
WRITE3(), etc.:
IF ( .NOT. WRITE3( 'myfile', 'ALL', JDATE, JTIME, NUMIDS ) ) THEN ...(some kind of error happened--deal with it here) END IF
ELEV,
TA,
and QV
given at up to 50 stations, each of
which may have up to 100 observation levels, is given by the following.
Note that ELEV
= "height of the level above ground"
is user-specified as one of the variables rather than supplied by the
system as for ROM Type 1 files.
... INTEGER*4 MXIDP ! max # of sites INTEGER*4 MXLVL ! max # of levels PARAMETER ( MAXID = 50, MXLVL = 100 ) ... INTEGER*4 NPROF ! # of actual sites INTEGER*4 PLIST( MXIDP ) ! list of site ID's INTEGER*4 NLVLS( MXIDP ) ! # of actual levels at site REAL*8 X ( MXIDP ) ! array of site X-locations REAL*8 Y ( MXIDP ) ! array of site Y-locations REAL*8 Z ( MXIDP ) ! array of site Z-locations REAL*4 ELEV ( MXLVL, MXIDP ) ! height of lvl a.g.l. REAL*4 TA ( MXLVL, MXIDP ) ! variable "TA" REAL*4 QV ( MXLVL, MXIDP ) ! variable "QV" COMMON /BAR/ NPROF, PLIST, NLVLS, X, Y, Z, ELEV, TA, QV ...The site dimension
MXIDP
maps into the NROWS3D
dimension , and the levels dimension MXLVL
maps into the
NCOLS3D
dimension in the file description (FDESC3) data
structures. To read or write this data, put the first element,
NPROF,
of the common BAR in the
array
spot of READ3(), WRITE3(), etc.:
IF ( .NOT. WRITE3( 'myfile', 'ALL', JDATE, JTIME, NPROF ) ) THEN ...(some kind of error happened--deal with it here) END IF
... INTEGER*4 MXNEST ! max # of nests INTEGER*4 MXGRID ! max # of cells (total, all grids) INTEGER*4 MXLAYS ! max # of levels PARAMETER ( MXNEST = 10, MXGRID = 10000, MXLAYS = 25 ) ... INTEGER*4 NNEST ! # of actual nests INTEGER*4 NLIST( MXNEST ) ! list of nest ID's INTEGER*4 NCOLS( MXNEST ) ! # of actual cols of nest INTEGER*4 NROWS( MXNEST ) ! # of actual rows of nest INTEGER*4 NLAYS( MXNEST ) ! # of actual lays of nest REAL*8 XN ( MXNEST ) ! array of nest X-locations REAL*8 YN ( MXNEST ) ! array of nest Y-locations REAL*8 DX ( MXNEST ) ! array of nest cell-size DX's REAL*8 DY ( MXNEST ) ! array of nest cell-size DY's REAL*4 NO2 ( MXGRID, MXLAYS, MXNEST ) ! variable "NO2" REAL*4 O3 ( MXGRID, MXLAYS, MXNEST ) ! variable "O3" COMMON /QUX/ NNEST, NLIST, NCOLS, NROWS, NLAYS, & XN, YN, DX, DY, NO2, O3 ...The nest dimension
MXNEST
maps into the NROWS3D
dimension, the cells dimension MXGRID
maps onto
NCOLS3D,
and the layers dimension MXLAYS
maps onto NLAYS3D
in the file description (FDESC3) data
structures. To read or write this data, put the first element,
NNEST,
of this common QUX
in the
ARRAY
spot of READ3(), WRITE3(), etc.:
IF ( .NOT. WRITE3( 'nfile', 'ALL', JDATE, JTIME, NNEST ) ) THEN ...(some kind of error happened--deal with it here) END IF
... INTEGER NASRC ! max # of active cols per row INTEGER NGRID ! number of rows in the matrix PARAMETER ( NASRC = 3978, NGRID = 5400 ) ... INTEGER NS( NGRID ) ! # of actual cols per row INTEGER IS( NACEL , NGRID ) ! column pointers REAL CS( NACEL , NGRID ) ! col-coefficients COMMON / GRIDMAT / NS, IS, CSIn this case, NVARS3D = 1. In the case of NVARS3D > 1, multiple coefficient matrices would share the same NS and IS arrays. The active-columns dimension NASRC maps into the NCOLS3D dimension and the matrix-rows dimension NGRID maps into the NROWS3D dimension in the file description (FDESC3) data structures. To read or write this data, put the first element,
NS
of the common
GRIDMAT
in the array
spot of READ3(),
WRITE3(),etc.:
IF ( .NOT. WRITE3( 'mfile', 'ALL', JDATE, JTIME, NS ) ) THEN ...(some kind of error happened--deal with it here) END IF
.. IF ( .NOT. DESC3( 'myfile' ) ) THEN ...(error: probably the file hasn't been opened yet) END IFSome of the items in a file description, such as the dates and times for file creation and update, and the name of the program which created the file, are maintained automatically by the system. Others describe the variables in the file: the file type (as described above), the number of variables, their names, unit designations, and descriptions, as well as the description of the file as a whole. Still others dimension the data: the number of layers and the grid dimensions (where for ID and profile files, the number of sites is mapped onto the rows dimension; for profile files, the number of vertical levels is mapped onto the columns dimension). Still other parts of the file description specify the geometry of the grid: the map projection used, its projection parameters, and the grid's location and cell-size relative to that map projection; the vertical-grid-coordinate type and the boundary values separating the model layers.
INIT3() is an integer
function. It returns the unit number to be used
for the program's log (if you setenv LOGFILE
, the I/O API's log and
error messages will be written to this unit; otherwise, they go to
standard output, unit 6). INIT3() can be called as many times as you
want, to get the unit number for the program log.
SHUT3() is a logical function that returns TRUE if the system successfully flushed all I/O API files to disk and shut itself down, and FALSE if it failed. If it failed, there probably was a hardware problem -- not much you can do about it, but at least you ought to be able to know. It is legal to call SHUT3() and close down all files currently open, and then to call INIT3() again and open new ones. NOTE that utility routine M3EXIT() calls SHUT3() as the final step of its operation (in addition to generating a log-message with the current simulation date-and-time and the indicated message-text). CLOSE3() is a logical function that returns TRUE if the system successfully flushed the indicated file to disk and closed it, and FALSE if it failed.
setenv SCENFILE
to that
file before you run the program, then OPEN3 will copy the
SCENFILE
information into the headers of any output files
for that program. Also, if you setenv EXECUTION_ID
to your
own identifier for the program execution, it will automate the storage
and the logging of that identifier. Finally, if you setenv
IOAPI_CHECK_HEADERS YES
, then the I/O API will perform a sanity
check on internal file descriptions -- checking that grid description
parameters are in range, for example, or that vertical levels are
either systematically increasing or systematically decreasing.
The arguments to OPEN3 are the name
of the file, an INTEGER
"magic number" indicating
the type of open operation, and the caller's name for logging and
audit-trail purposes. You can call OPEN3 many times for the same file
without hurting anything, if you want -- as long as you don't first open
it read-only and then try to change your mind, or try to open it as a
NEW file after it is already open. Names and values for the
mode-of-opening magic number argument are defined in
PARMS3.EXT as
the following:
INCLUDE
file
FDESC3.EXT
to define the structure for the file, and then call OPEN3(). If the
file doesn't exist in either of these cases, OPEN3() will use the
information to create a new file according to your specifications,
and open it for read/write access. In the "unknown"case,
if the file already exists, OPEN3() will perform a consistency check
between your supplied file description and the description found in
the file's own header, and will return TRUE (and leave the file open)
only if the two are consistent. Sample calls to OPEN3() for an
input file 'myfile' and an output file 'my_newfile' might look like
the following:
... IF ( .NOT OPEN3( 'myfile', FSREAD3, 'my program') ) THEN ...(some kind of error happened--deal with it here) END IF ... ... (First, fill in the file's description for 'my_newfile'. ... Then open it:) IF ( .NOT. OPEN3( 'my_newfile', FSNEW3, 'my program' ) ) THEN ...(some kind of error happened--deal with it here) END IFThere are also three sample programs that demonstrate how to use the I/O API to create various kinds of files -- gridded, boundary, and ID-referenced, with one or multiple layers, and either time-stepped or time-independent.
NOTE: Joan Novak (EPA) and Ed Bilicki (MCNC) have declared as a software standard that modeling programs may not use FSCREA3 as the mode for opening files. FSCREA3 is reserved for use by analysis/data extraction programs only.
To get a file's description, you use the
DESC3() function.
When you call DESC3(), it puts the file's complete description in the
standard file description data structures in
FDESC3.EXT .
Note that the file must have been opened prior to calling DESC3().
A typical call might look like:
... IF ( .NOT. DESC3( ' myfile' ) ) THEN ...(some kind of error happened--deal with it here) ELSE ...(the FDESC3 commons now contain the file description: ... data type, dimensions, starting date&time, timestep, ... list of variables and their descriptions, etc.) END IF ...
size
argument -- you tell it how much data you expect, and
it checks that against how much data the file thinks you ought to get,
for error checking purposes. A typical INTERP3 call to read/interpolate
the variable HNO3 to 12:30 PM on February 4, 1987 might look like:
... CHARACTER*16 FNAME, VNAME REAL*4 ARRAY( NCOLS, NROWS, NLAYS ) ... IF ( .NOT. INTERP3( 'myfile', 'HNO3', 1987035, 123000, & NCOLS*NROWS*NLAYS, ARRAY ) ) THEN ...(some kind of error happened--deal with it here) END IFWith READ3() and XTRACT3(), you can use the "magic values"
ALLVAR3'
(= 'ALL', defined in
PARMS3.EXT
) or
ALLAYS3
(= -1, also defined in PARMS3.EXT
) as
variable name and/or layer number
to read all variables or all layers from the file, respectively.
For time independent files, the date and time arguments are ignored.
ALLVAR3
(= 'ALL', defined in
PARMS3.EXT
) as the
variable-name. For ID-referenced,
profile, and grid-nest files, you must write an entire time step at a time
(i.e., the variable-name must be ALLVAR3
).
WRITE3() is affected by standard
environment variable IOAPI_LOG_WRITE (which has default value
"YES"); normally WRITE3() generates a log message for each
write-operation successfully completed. However, if you
setenv IOAPI_LOG_WRITE NO
then these messages will be
suppressed. Typical WRITE3() calls to write data for date and time
JDATE:JTIME might look like the following:
... REAL*4 ARRAY( NCOLS, NROWS, NLAYS, NVARS ) ... IF ( .NOT. WRITE3( 'myfile', 'HNO3', JDATE, JTIME, ARRAY ) ) THEN ...(some kind of error happened--deal with it here) END IF IF ( .NOT. WRITE3( 'afile', 'ALL', JDATE, JTIME, ARRAYB ) ) THEN ...(some kind of error happened--deal with it here) END IF
HHMMSS = 10000 * hour + 100 * minutes + seconds YYYYDDD = 1000 * year + daywhere the year is 4-digits (1994, say, rather than just 94), and the day is the Julian day-number (1,...,365 or 366). By convention, dates and times are stored in Greenwich Mean Time. There are two utility programs, juldate, for converting calendar dates to Julian dates, and gregdate, for converting Julian dates to calendar dates and reporting the day-of-the-week. Both of these programs also report whether daylight savings time is in effect for the specified date. There are also a number of utility routines available for manipulating dates and times. Note that for these routines, time steps may perfectly well be negative -- just make sure you keep the parts all positive or all negative; a time step of -33000 means to step three and a half hours into the past, for example. This way of representing dates and times is easy to understand and manipulate when you are watching code in the debugger (you don't have to turn "seconds since Jan. 1, 1970" into something meaningful for your model run, nor do you have to remember whether April has 30 days or 31 when your model run crosses over from April to May). The utility routines for manipulating dates and times are the following:
Diagrams showing the relationship of the grid and its layers to the header attributes XORIG3D, YORIG3F, VGLVS3D, etc., are available in Postscript, X bitmap, JPEG, and GIF image formats. Note that Layer 1 is the bottom layer in the modeling grid. Some development work needs yet to be done: we need to do some more work about the definition of grids and map projections , and then to define and write utility routines having to do with map projections, transformation of locations from one map projection to another, and interpolation of data from one grid to another (on possibly different map projections).
Next: Changes from the Previous I/O API Version
To: Models-3/EDSS I/O API: The Help Pages