Availability/Download/Installation of the EDSS/Models-3 I/O API

Please read the copyright, licensing, and acknowledgements page: The I/O API library is available under the Lesser Gnu Public License, version 2.1; the "tool" and example programs are available under the Gnu Public License, version 2.

The What's New page documents bug-fixes, new features, new compiler-support, etc.

Note that a large amount of effort has been expended during the whole of I/O API development to ensure upward compatibility: programs written for the original Version 0.9 pre-release should still work correctly with I/O API Version 3.0, unlike some other software we will refrain from naming here.

Contents


NOTES on gfortran

GNU gfortran seems to be insistent upon incompatibility: not only incompatibility with existing industry standards, but even inconsistency with itself. In some cases, not even Makefile will continue to work from one version to the next. At this time, gfortran should be used only with trepidation.
--Carlie J. Coats, Jr., Ph.D.
I/O API author

Finding MODULE files
Fortran compilers use various ways to find directories containing "module files" such as the I/O API's m3utilio.mod. Suppose the I/O API's OBJDIR is something like /foo/bar/$BIN. Then for most F90 compilers, the way to tell the compiler to search that directory for module files is to use the following as a compiler-flag:
-I/foo/bar/$BIN
. Some other compilers (e.g., Sun, Absoft, older versions of gfortran) use
-M/foo/bar/$BIN
. Recent versions of gfortran use
-J/foo/bar/$BIN
. Needless to say, this makes it more difficult to write Makefiles ;-(

IARGC and COMMAND_ARGUMENT_COUNT, etc.
Recent versions of gfortran break compatibility with older versions, and with standards such as Mil-Spec 1753, in their handling of command line arguments: they no longer support IARGC() and GETARG(), insisting rather on Fortran-2003 constructs COMMAND_ARGUMENT_COUNT() and GET_COMMAND_ARGUMENT().

I/O API file init3.F now has "hacks" to get around this incompatibility: if preprocessor-symbol NEED_ARGS is defined, it generates IARGC() and GETARG() in terms of COMMAND_ARGUMENT_COUNT() and GET_COMMAND_ARGUMENT(). By default, Makeinclude.Linux2_x86, and Makeinclude.Linux2_x86_64 and Makeinclude.Linux2_x86_64gfort (which specify how to build the I/O API (etc.) for 32-bit or 64-bit gfortran/gcc) now define this symbol as part of make-variable ARCHFLAGS.

If you get "symbol not found" errors when building I/O API programs like those in m3tools, remove this -DNEED_ARGS=1 from the relevant Makeinclude file.

Back to Contents


NOTE on NetCDF Version 4

In general, you are best off if you can build the whole modeling system (libnetcdf.a, libpvm3.a, libioapi.a, and your model(s) CMAQ, SMOKE, etc. with a common compiler set and common set of compile-flags.

NetCDF Version 4.x have lots of additional build options, that will require a complex set of additional libraries in Makefiles and all other model-building systems. It is recommended that you disable these options by adding the command-line flags below to your netCDF configure command:

--disable-netcdf4 --disable-dap
NetCDF Version 4.1 and later also split the Fortran interfaces into a separate library libnetcdff.a As a result, much compatiblity with the infrastructure (Makefiles, etc.) is broken: you must modify all Makefiles, replacing -lnetcdf with -lnetcdff -lnetcdf. If you don't build netCDF as indicated above, you will need additional libraries that may be discovered by running the netCDF command
nc-config --libs

Starting with I/O API Version 3.1 of October 11, 2011, the m3tools/Makefiles are configured to use the -lnetcdff -lnetcdf netCDF-v4-style libraries. If using a previous version of netCDF, you must modify them manually.

You must also modify the build-systems for CMAQ. MCIP, SMOKE, and all your other models similarly.

Back to Contents


Other Warnings

Strong Recommendation  1
Don't change any I/O API INCLUDE files. Any changes are likely to result in very obscure, difficult-to-diagnose bugs caused by an inconsistency between standard libioapi.a object-libraries and whatever code is compiled with the resulting modified INCLUDE-file. Note that the distribution license specifies that by making any changes to any of these INCLUDE files, the user explicitly agrees that in the case any assistance is required of BAMS or of the I/O API author, Carlie J. Coats, Jr. as a result of such changes, the user and/or his project or contract agrees to reimburse BAMS and/or the I/O API author, Carlie J. Coats, Jr., at a rate triple the normal contract rate for the services required.

Strong Recommendation  2
You are best off if you can build the whole modeling system (libnetcdf.a, libpvm3.a, libioapi.a, and your model(s) CMAQ, SMOKE, etc. with a common compiler set and common set of compile-flags. In particular, Linux-distribution-vendor supplied libnetcdf.a rarely works with CMAS-supported compiler sets.

Recommendation  3
On at least Sun Solaris and SGI IRIX platforms, f77 and f90 are known not to be link compatible; you will need to keep builds for them separate (which is the original distinction for BIN=IRIX64f77 vs BIN=IRIX64f90, for example). It is recommended that for other systems you likewise keep different compiler systems (and sometimes even different versions of the same compiler--Intel ifort versions 10, 11, and 12, for example) separate.

Back to Contents


Build and installation instructions for I/O API Versions 3.0 and 3.1

To set up the installation:
  1. Download the gzipped tar-file for the version you want: ioapi-3.1.tar.gz or ioapi-3.0.tar.gz. It contains directories ioapi for the I/O API library source code, HTML for the documentation, m3tools for the related tool programs, iotest for various test-programs, and notcdf for the NCEP libnotcdf.a library (a dummy library for installation sites that don't allow netCDF on-site).

  2. cd to the directory under which you wish to build the I/O API. gunzip and untar (gtar xvfz ioapi-3.1.tar.gz does this in one operation).

  3. If you want to disable the coupling-mode configuration (which is enabled by default), cd ioapi and then cp Makefile.nocpl Makefile.

  4. For each desired configuration, setenv BIN <machinetype>, where <machinetype> matches the extension on one of the ioapi/Makeinclude.*. Then do a top-level make dirs to build the object-directories.

    The usual pattern for generating BIN is

    setenv BIN `uname -s``uname -r | cut -d. -f1`
    and then modify the result by appending any compiler specifics, e.g., for F90 or debug compiled configurations.

    Linux versions 2 and 3 are binary-compatible, so Linux2 is used for both of these (and similarly for recent versions of SunOS).

    Note, for example, that for Linux2_x86*, there are distinct (and incompatible) variants for the GNU g77/gcc, Intel ifc/icc, Lahey-Fujitsu lf95/GNU gcc, and Portland Group pgf90/pgcc; moreover, the latter is not compatible with Portland Group pgf90/GNU gcc (when the latter is used to build libnetcdf.a, as in the vendor supplied libraries on RedHat (that incompatibility being the largest single generator of installation-questions in recent years!).

    INTEL and SGI NOTES:

    • At some point in its 8.0.x sequence of compiler versions, Intel has changed the names of its compilers: ecc becomes icc for the Itanium (IA64) platform, and both efc and ifc become ifort; you may need to edit the MAKEINCLUDE.$BIN accordingly.
    • There are problems with the Fortran pre-processing for some versions of the Intel Fortran compilers; these can to some extent alleviated by separating the preprocessing from the compile step for the *.F files. Makefile.cpphack is a sample Makefile for this purpose (as provided, it assumes OpenMP and coupling mode).
    • There are problems with SGI f90 Version 7.4. Use Version 7.4.1 or later instead.
    • There are command line incompatibilities between SGI f90 Version 7.3.x and 7.4.1; for 7.3.x, you need to edit the MAKEINCLUDE.$BIN to replace -TARG:platform=ip27 -TARG:processor=r12000 by -TARG:platform=ip27,processor=r12000, (etc.)

  5. Acquire and build the netCDF library, libnetcdf.a, for each configuration. The netCDF home page is http://www.unidata.ucar.edu/packages/netcdf/ For each configuration,copy or link the resulting libnetcdf.a ( and libnetcdff.a, if you're building netCDF version 4.1 or later) to the appropriate object directory.

    If you plan to build a coupling-mode configuration, do the same with PVM's libpvm3.a. See http://www.csm.ornl.gov/pvm/pvm_home.html (Only the C interface to PVM is used; the resulting library tends to be more compiler-independent than for netCDF, where both C and Fortran interfaces are used.)

  6. Edit the relevant files ioapi/Makeinclude.$BIN for your system specific compiler options and PVM installation location, as appropriate.

  7. Edit the top-level Makefile to set variables BASEDIR, etc., for your system.

    If you are doing a multi-configuration installation, do the same for each */Makefile*.

    If you are doing a single-configuration installation, do make configure

  8. setenv BIN ..., followed by one of the make <target> commands below, as appropriate for your configuration(s),

  9. Note that a top-level make will generate messages about installing the notCDF library. Almost all users can ignore these messages; libnotcdf.a is only required at NCEP, where it is politically forbidden to have any copy of libnetcdf.a on the system.

Top-Level make targets include the following:
make all
Build all libraries and executables (assumes the Fortran compiler is Fortran-90 compliant) for the default mode.
make cpl
Build all libraries and executables for PVM-enabled coupling mode.
make nocpl
... for no-PVM no-coupling mode. This is probably the best target for regulatory users who need multiple configurations.
make relink
Rebuild all the executables, using the existing M3TOOLS object files (e.g.,if you have changed the libraries).
make clean
removes all executable, object (.o), and library (.a) files.
make configure
allows you to set up make-parameters in the top-level Makefile and then run a set of sed commands on the templates */Makefile*.sed in order to generate lower level Makefiles for a particular configuration.
make bins
builds everything for several choices of Linux architectures and compilers.
Note: easily customized for your system.
make binclean
builds everything for several choices of Linux architectures and compilers.
make fix
builds (pre-version-5.0) CMAQ "fixed-132" Fortran source form INCLUDE-files.

Back to Contents


I/O API Version 3.1 Availability

Version 3.1 of the I/O API library code is available here for download in source code form. This version is upwardly source-code compatible with previous versions; however, it is not object-code compatible, nor is it compatible with object files built using INCLUDE files derived from previous versions. New features of this release are documented here, and include
  • Change of MXVARS3 in PARMS3.EXT from 120 to 2048. Consultation with actual modelers indicate that this should be more than adequate; suggestions elsewhere that users should change MXVARS3 to 50000 are extremely irresponsible, quite ridiculous, and dangerous for all but the most sophisticated users..
  • removal of f77 support
  • optional support for "large"time-step records (total timestep extent, including all variables, exceeds 2GB), controlled by environment variable IOAPI_OFFSET_64 (default NO)
  • Fortran-90-ized m3tools programs (demonstrating how MODULE M3UTILIO should be used for all new codes).
  • Changes to Makeinclude files for gfortran, together with replacing g77 with that compiler for some ${BIN} choices.
  • Minor changes to MODULE M3UTILIO:
    • Specification of INTENT where possible in the INTERFACEs
    • Add INTERFACEs for JUNIT(). PCOEF(), PROMPTGRID(), SCANINT(), SETSPHERE(), INITSPHERES(), SPHEREDAT(), SKIPL().
    • New routine LASTTIME() does overflow-safe computation of the ending date&time EDATE,ETIME for a timestep sequence with starting date&time SDATE,STIME, timestep TSTEP and NRECS steps.
  • Default m3tools Makefiles assume netCDF 4.1 or later (which requires -lnetcdff as part of the library flags for linking).

The build procedure is given above..

Version 3.1 can read all files generated by Version 3.0; Version 3.0 can read all files generated by Version 3.1, provided that the number if variables in the file NVARS3D < 121.

Due to the changes in MXVARS3. Version 3.1 is a "Jubilee Day" release: it should completely replace previous versions; object files produced using Version 3.1 should not be mixed with previous versions.

. Note that proper use of MODULE M3UTILIO ensures that the library and user object code use consistent parameter values (as well as ensuring correctness of most call interfaces).

Note also that changing MXVARS3 was never supported, is strongly discouraged, and may lead to invocation of the triple-time clause (see header of PARMS3.EXT) if support of such a modified installation is requested.

This release is current as of September 28, 2011 and includes all bug-fixes up through that date.

Back to Contents


I/O API Version 3.0 Availability

Version 3.0 of the I/O API library code is available here for download in source code form. New features of this release are documented here, and include additional support for WRF, a new native-binary-format lower layer, a file-set lower layer, some new routines, and enhancements for portability and OpenMP thread-safety.

This release is current as of June 11, 2007 and includes all bug-fixes up through that date. It should be considered the production release of Version 3.0 for netCDF files and for PVM-based virtual files, but should probably still be considered "beta" for native-binary layer, and for the multi-file file-list layer, since each of these latter has basically a single-user community rather than widespread beta-test usage.

The build procedure also has been considerably enhanced, by the addition of a top-level Makefile which allows one to build various configurations of the entire (I/O API + M3TOOLS) system with one make command. Depending upon what you want to do, there are two main options for how you will build the system:

To build multiple configurations from one installation directory (e.g., multiple compilers, etc.)
The procedure is much as before, except that one can use just a single make bins command to build all the libraries and all the executables for a configuration.

Before you do so, yhou should edit the ioapi/Makefile and m3tools/Makefile commands so that the make bins and make binclean commands properly reflect the configurations you want to build.

To build a single configuration from one installation directory
Follow the build procedure given above..

To set up the installation:
  1. Download the gzipped tar-file. It contains directories ioapi for the I/O API library source code, HTML for the documentation, m3tools for the related tool programs, iotest for various test-programs, and notcdf for the NCEP libnotcdf.a library (a dummy library for installation sites that don't allow netCDF on-site).

  2. cd to the directory under which you wish to build the I/O API. gunzip and untar (gtar xvfz ioapi-3.0.tar.gz does this in one operation).

  3. If you want to disable the coupling-mode configuration (which is enabled by default), cd ioapi and then cp Makefile.nocpl Makefile.

  4. For each desired configuration, setenv BIN <machinetype>, where <machinetype> matches the extension on one of the ioapi/Makeinclude.*. Then do a top-level make dirs to build the object-directories.

    The usual pattern for generating BIN is

    setenv BIN `uname -s``uname -r | cut -d. -f1`
    and then modify the result by appending any compiler specifics, e.g., for F90 or debug compiled configurations.

    Note, for example, that for Linux2_x86, there are distinct (and incompatible) variants for the GNU g77/gcc, Intel ifc/icc, Lahey-Fujitsu lf95/GNU gcc, and Portland Group pgf90/pgcc; moreover, the latter is not compatible with Portland Group pgf90/GNU gcc (when the latter is used to build libnetcdf.a, as in the vendor supplied libraries on RedHat (that incompatibility being the largest single generator of installation-questions in recent years!).

    INTEL and SGI NOTES:

    • At some point in its 8.0.x sequence of compiler versions, Intel has changed the names of its compilers: ecc becomes icc for the Itanium (IA64) platform, and both efc and ifc become ifort; you may need to edit the MAKEINCLUDE.$BIN accordingly.
    • There are problems with the Fortran pre-processing for some versions of the Intel Fortran compilers; these can to some extent alleviated by separating the preprocessing from the compile step for the *.F files. Makefile.cpphack is a sample Makefile for this purpose (as provided, it assumes OpenMP and coupling mode).
    • There are problems with SGI f90 Version 7.4. Use Version 7.4.1 or later instead.
    • There are command line incompatibilities between SGI f90 Version 7.3.x and 7.4.1; for 7.3.x, you need to edit the MAKEINCLUDE.$BIN to replace -TARG:platform=ip27 -TARG:processor=r12000 by -TARG:platform=ip27,processor=r12000, (etc.)

  5. Acquire and build the netCDF library, libnetcdf.a, for each configuration. The netCDF home page is http://www.unidata.ucar.edu/packages/netcdf/ For each configuration,copy or link the resulting libnetcdf.a ( and libnetcdff.a, if you're building netCDF version 4.1 or later) to the appropriate object directory.

    If you plan to build a coupling-mode configuration, do the same with PVM's libpvm3.a. See http://www.csm.ornl.gov/pvm/pvm_home.html (Only the C interface to PVM is used; the resulting library tends to be more compiler-independent than for netCDF, where both C and Fortran interfaces are used.)

  6. Edit the relevant files ioapi/Makeinclude.$BIN for your system specific compiler options and PVM installation location, as appropriate.

  7. Edit the top-level Makefile to set variables BASEDIR, etc., for your system.

    If you are doing a multi-configuration installation, do the same for each */Makefile*.

    If you are doing a single-configuration installation, do make configure

  8. setenv BIN ..., followed by one of the make <target> commands below, as appropriate for your configuration(s),

  9. Note that a top-level make will generate messages about installing the notCDF library. Almost all users can ignore these messages; libnotcdf.a is only required at NCEP, where it is politically forbidden to have any copy of libnetcdf.a on the system.

Back to Contents


I/O API Version 2.2 Availability

Version 2.2 of the I/O API library code is available here for download in source code form. New features of this release are documented here. The build procedure is the same as that for Version 2.1, documented below, except for the following:
  • The default object library directory is directly under the source code directory. To change where the object directory is located, change the value of shell variable OBJDIR in the Makefile, either by commenting out the current value and un-commenting one of the alternatives, or by setting your own system-dependent value.

  • The default Makefile builds the I/O API with the Coupling Mode Extensions enabled, and therefore requires linking executables with PVM library (version 3 or greater). To disable the Coupling Mode Extensions, copy the provided Makefile.nocpl to Makefile and then build the system.

This download is current as of October 18, 2003 Back to Contents


I/O API Version 2.0 and 2.1 Availability

Updated May 7,2002 for SMOKE release: now produces a library named libioapi_v2.1.a instead of Version 2.0's libioapi_new.a

Release Version 2.1 of the I/O API and the I/O -related tool and sample programs is available here for download in source code form. New features of this release are documented here. Makeinclude files are available for the following platforms (and some debug- and other variants, denoted by dbg i8, and r8 suffixes):

  • AIX (32-bit, with default name-mangling rules)
  • AIX_ (32-bit, with Feldmanish name-mangling rules)
  • HP-UX using f90
  • IRIX5 "old-32" binary object-format, using f77
  • IRIX5f90 "old-32" binary object-format, using f90
  • IRIX6 (same as IRIX6n32)
  • IRIX64 "-64" binary type, using f77
  • IRIX64f90 "-64" binary type, using f90
  • IRIX6n32 "-n32" binary type
  • IRIX6n32f90 "-n32" binary type, using f90
  • IRIX6n32f90dbg "-n32" binary type, using f90, compiled for debug
  • Linux2_alpha using GNU gcc and Compaq fort
  • Linux2_ia64 using gcc and g77
  • Linux2_ia64eifc using the Intel efc and ecc
  • Linux2_x86 using gcc and g77
  • Linux2_x86ifc using the Intel ifc and icc
  • Linux2_x86lf95 using the Lahey-Fujitsu f95 and GNU gcc
  • Linux2_x86pg using the Portland Group f90 and cc
    NOTE: Makeinclude.Linux2_x86pg assumes you are using a gcc/g77-compatible libnetcdf.a (as comes from the vendor on many Linux systems). For pgf77/pgf90/pgcc versions of libnetcdf.a, remove -Msecond_underscore from ARCHFLAGS in Makeinclude.Linux2_x86pg.
  • Alpha OSF1 using Compaq f90 and cc
  • SunOS5 using f77
  • SunOS5f90 using f90
  • cray UNICOS10/F90

From these examples and from a knowledge of the compiler user manual, it should be relatively easy to build Makeinclude files for most UNIX or UNIX-like platforms and compilers. Instructions for building the I/O API library libioapi_v2.1.a and the I/O API tool executable programs are as given below.

NOTE: It is important to ensure that the compile flags used for building the I/O API library are consistent with both the compile flags used for building your program and the compile flags used for building the netCDF, PVM, and other libraries used by your programs. Failure to do so may result in link-failures or (in the case of the Linux Portland Group compilers) run-time program deadlocks (program hangs, doing no work while consuming all available CPU resources).

  1. Download the gzipped tar-file ioapi_21.tar.gz. It contains directories ioapi for the I/O API library source code, ioapi_doc for the HTML documentation, and ioapi_tools for the related tool programs.

  2. cd to the directory under which you wish to build the I/O API. gunzip and untar the ioapi_21.tar.gz (with Gnu tar,
    tar xvfz ioapi_v2.1.tar.gz
    does unzip-untar all in one step).

  3. setenv BIN <machinetype>
    where <machinetype> matches the extension on one of the Makeinclude.* (building your own Makeinclude if yours is not one of the supported systems).
    The usual pattern for generating BIN is
    setenv BIN `uname -s``uname -r | cut -d. -f1`
    although there are exceptions where more work is needed for Cray, SGI and Linux systems, and F90 or DEBUG compiles.

  4. The default directory for both executables and object libraries is in directory ../$BIN relative to the source code directories for the I/O API and tools. Edit the Makefile to put OBJDIR wherever you want it (if you want somewhere other than the default ../$BIN location).
    NOTE 1: Different compilers generate linker-visible object names in different ways (some with multiple options...). It is important that all of the compiles for an entire executable program (including the compiles for netCDF, PVM, and other libraries) use the same scheme; this is controlled by various parts of the ARCHFLAGS variable in the (compiler-system dependent) Makeinclude.$BIN files.
    NOTE 2: By default on most systems, OpenMP parallelism is enabled; see the OMPFLAGS variable in Makeinclude.$BIN. The I/O API does not have parallel sections of its own; however, enabling OpenMP does allow the activation of critical sections allowing the I/O API to be thread-safe for OpenMP-parallel programs (like the MAQSIP-RT air quality model, the WRF or MCPL-enabled MM5 meteorology models, research versions of SMOKE, and others. You may set OMPFLAGS and OMPLIBS to empty-strings in order to disable this thread-protection, if you wish.

  5. In the I/O API library source directory ioapi, type make to build the object library. The current build process will generate $OBJDIR/libioapi_v2.1.a (to distinguish it from earlier Version 1 $OBJDIR/libioapi.a); mv it to libioapi.a or libm3io.a, if desired.

  6. If necessary, get netCDF and build libnetcdf.a; if you're building with Coupling Mode active, do the same for PVM.
  7. Copy or link (ln -s ...) libnetcdf.a (and libpvm3.a if you built it) to your OBJDIR.

  8. In the I/O API tool source directory ioapi_tools, type either make (if you have a F90-compliant Fortran compiler), or make f77 (if you don't) to build the tool-program executables (adjusting the LIB make-variable if you renamed $OBJDIR/libioapi_v2.1.a in the preceeding step). Note that there are a number of these programs that do require F90: dayagg, ginterp, m3cple, m3agmax, m3agmask, m3combo, m3merge, mtxblend, mtxbuild, mtxcalc, mtxcple, presterp, and selmrg2d.

Binary executables and object libraries are no longer available (June 11, 2012 ).

NOTE: On the Sun SunOS and SGI IRIX platforms, there are link incompatibilities between f77 and f90, caused by the fact that these vendors' f77 and f90 by default use different system libraries. Compaq, Cray, and IBM compilers do not seem to do this; however, it is probably advisable not to mix f77 and f90 compiles on a not-well-understood new platform.

Back to Contents


NOTE on File Types

Files with extension ".gz" or ".tgz" are compressed with the GNU gzip utility. Source for gzip is available from a number of FTP sites; executables for a variety of platforms are available from several sources, including ftp://ftp.netscape.com/pub/unsupported/gnu/. On windows platforms, either winzip or the Cygnus windows port of gzip can deal with ".gz" and ".tgz" files.

Back to Contents


DISCLAIMER

This source code and these object libraries are available in unsupported form only, and neither MCNC nor Baron Advanced Meteorological Systems LLC accepts any liability for its usability or correctness for any purpose. See the "Notices" document for acknowledgements, copyright and licensing information.

On the other hand, its author thinks it is good stuffTM and welcomes comments and suggestions. Back to Contents


I/O API Documents

Documentation for the EDSS/Models-3/BAMS I/O API is available only as HTML -- it was designed from the start as a heavily cross-linked hyperdocument, and is not available in linear dead-tree document-forms such as PDF. The Documentation for the EDSS/Models-3 I/O API is copyright (C) 1992-2002 MCNC and Carlie J. Coats Jr, and copyright (C) 2003-2011 Baron Advanced Meteorological Systems. A gzipped tar-file snapshot for I/O API Version 2.1 is available here.;

Back to Contents


UP: I/O API home page

UP: Environmental Modeling Center at MCNC home page

UP: The Environmental Decision Support System Pages

Back to the I/O API User Manual

Send comments to

Carlie J. Coats, Jr.
cjcoats@email.unc.edu

    $Id: AVAIL.html 2011 2012-12-13 16:16:30Z coats@borel $