======================================================================
INSTRUCTIONS FOR INSTALLING PYLITH ON THE CALTECH LINUX CLUSTER GARUDA
 ======================================================================

These installation instructions assume suitable C/C++ and Fortran
compilers have been installed along with MPI. MPI on clusters is often
hardware dependent and should be installed by system
administrators. We also assume that GNU autoconf tools and subversion
have been installed.

We install development versions of the CIG software. Installing using
source tarballs is easy, simply replace downloading from the source
repository and running autoreconf with downloading and unpacking the
tarball.

To make things easier, I create environment variables TOOLS_DIR,
PYLITH_DIR, TOOLS_FORMAT, and PYTHON_VERSION. The TOOLS_FORMAT variable
makes it easier to maintain multiple copies of packages built with
different options (e.g., different compilers and different levels of
optimization). Note that the python version should match the version
of Python you will be using. If you don't know the version of python,
run "python --version".

See the files in the Cluster_CIT-garduda directory for the settings
used to setup the environment variables using the bash shell. The
$TOOLS_FORMAT environment variable is used to differentiate between
packages built with different compilers and/or different optimization
settings. In this case, TOOLS_FORMAT is intel-10.1. Because this is a
cluster the tools are built for production runs using
high-optimization level. You should setup your environment BEFORE
building the packages.

Source files

  Source files are placed under $HOME/src/PACKAGE.

Build directories

  Packages supporting building in separate directories are built in
  $HOME/scratch/build/$TOOLS_FORMAT.

Install directorires

  Packages are installed in $HOME/tools/$TOOLS_FORMAT. PyLith is
  installed in $HOME/pylith-dev/$TOOLS_FORMAT since we may want both
  the development version and stable releases of PyLith installed.


1. Python (http://www.python.org)

  Download and unpack the tarball.
    cd $HOME/src/python-2.6.5
    wget http://www.python.org/ftp/python/2.6.5/Python-2.6.5.tgz
    tar -zxvf Python-2.6.5.tgz

  Run configure in the build dir.
    cd $HOME/scratch/build/$TOOLS_FORMAT/python-2.6
    $HOME/src/python-2.6/Python-2.6.5/configure --prefix=${TOOLS_DIR} --enable-shared CC=icc CXX=icpc CFLAGS="-O2 -fPIC" CXXFLAGS="-O2 -fPIC"

  Run "make" and "make install".
    make
    make install

  Test Python
    python

  You should get the Python prompt and see the correct version and
  time of the build.


2. cppunit

  Download and unpack the tarball.
    cd $HOME/src/cppunit-1.12.1
    wget http://downloads.sourceforge.net/cppunit/cppunit-1.12.1.tar.gz
    tar -zxvf cppunit-1.12.1.tar.gz

  Run configure in the build dir.
    cd $HOME/scratch/build/$TOOLS_FORMAT/cppunit-1.12.1
    $HOME/src/cppunit-1.12.1/cppunit-1.12.1/configure --prefix=${TOOLS_DIR} CC=icc CXX=icpc
  
  Run "make" and "make install"
    make
    make install


3. proj.4

  Download and unpack the tarball and horizontal datum files.
    cd $HOME/src/proj-4.7.0
    wget   http://download.osgeo.org/proj/proj-4.7.0.tar.gz
    wget http://download.osgeo.org/proj/proj-datumgrid-1.5.zip
    tar -zxvf proj-4.7.0.tar.gz
    cd proj-4.7.0/nad
    unzip ../../proj-datumgrid-1.5.zip

  Run configure in the build dir.
    cd $HOME/scratch/build/$TOOLS_FORMAT/proj-4.7.0
    $HOME/src/proj-4.7.0/proj-4.7.0/configure --prefix=${TOOLS_DIR} CC=icc CFLAGS="-O3 -fPIC"

  Unpack the datum files in the nad directory of the build.
    cd nad
    unzip $HOME/src/proj-4.7.0/proj-datumgrid-1.5.zip

  Run "make" and "make install".
    make
    make install

  Make sure the $HOME/tools/$TOOLS_FORMAT/share/proj contrains
  "conus", "alaska", "nad27", and "nad83" among the other files.


4. FIAT (version 0.3.5 is the most recent version known to work)

  Depends on Python.

  Download and unpack the tarball.
    cd $HOME/src/fiat-0.3.5
    wget http://www.fenics.org/pub/software/fiat/FIAT-0.3.5.tar.gz

  Install using python.
    cd FIAT-0.3.5
    python setup.py install --prefix=$TOOLS_DIR

  Test
    python
    import FIAT


5. swig

  Depends on Python.

  Download and unpack the tarball.
    cd $HOME/src/swig-1.3.40
    wget http://downloads.sourceforge.net/project/swig/swig/swig-1.3.40/swig-1.3.40.tar.gz
    tar -zxvf swig-1.3.40.tar.gz

  Run configure in the build directory.
    cd $HOME/scratch/build/$TOOLS_FORMAT/swig-1.3.40
    $HOME/src/swig-1.3.40/swig-1.3.40/configure --prefix=${TOOLS_DIR} --with-python=`which python` --without-python3 CC=icc CXX=icpc CFLAGS="-O2 -fPIC" CXXFLAGS="-O2 -fPIC"

  Run "make" and "make install".
    make
    make install


6. netcdf

  Download and unpack the tarball.
    cd $HOME/src/netcdf-4.1.1
    wget http://www.unidata.ucar.edu/downloads/netcdf/ftp/netcdf-4.1.1.tar.gz
    tar -zxvf netcdf-4.1.1.tar.gz

  Run configure in the build directory.
    cd $HOME/scratch/build/$TOOLS_FORMAT/netcdf-4.1.1
    $HOME/src/netcdf-4.1.1/netcdf-4.1.1/configure --prefix=${TOOLS_DIR} --enable-shared CC=icc CXX=icpc FC=ifort F77=ifort CFLAGS="-O2 -fPIC" CXXFLAGS="-O2 -fPIC" FFLAGS="-O2 -fPIC" FCFLAGS="-O2 -fPIC"

  Run "make" and "make install".
    make
    make install


7. pythia

  Depends on Python.

  Check out pythia from the publicly accessible CIG SVN repository.
    cd $HOME/src
    svn co http://geodynamics.org/svn/cig/cs/pythia/trunk pythia

  Install using python.
    cd pythia
    python setup.py install --prefix=$TOOLS_DIR

  Test
    python
    import pyre


8. numpy

  Depends on Python and math libraries.

  Download and unpack the tarball.
    cd $HOME/src/numpy-1.4.1
    wget http://downloads.sourceforge.net/project/numpy/NumPy/1.4.1/numpy-1.4.1.tar.gz
    
  Edit the site.cfg file to use the Intel MKL library. If you don't
  have the Intel MKL library, then use the system BLAS/LAPACK. These
  settings differ among MKL releases.
    [mkl]
    library_dirs = /opt/intel/mkl/10.0.1.014/lib/em64t/
    lapack_libs = mkl_lapack
    mkl_libs = mkl, guide

  Install using python.
    cd numpy-1.4.1
    python setup.py install --prefix=$TOOLS_DIR

  Test
    python
    import numpy
    import numpy.linalg

  Note: If the build or install fails and you change the site.cfg
  file, then you may need to remove the build directory in
  $HOME/src/numpy-1.4.1/numpy-1.4.1 before rebuilding to insure the
  old build files are not used.


9. spatialdata

  Depends on Python, SWIG, numpy, pythia, proj.4.

  Check out spatialdata from the publicly accessible CIG SVN repository.
    cd $HOME/src
    svn co http://geodynamics.org/svn/cig/cs/spatialdata-0.1/trunk spatialdata-dev
    cd spatialdata-dev
    autoreconf -if

  Run configure in the build dir.
    cd $HOME/scratch/build/$TOOLS_FORMAT/spatialdata-dev
    ${HOME}/src/spatialdata/configure  --config-cache  --enable-swig --enable-testing --enable-pythia  --prefix=${TOOLS_DIR}  CPPFLAGS="-I${TOOLS_DIR}/include -DNDEBUG"  LDFLAGS="-L${TOOLS_DIR}/lib " CC=icc CXX=icpc CFLAGS="-O3 -fPIC" CXXFLAGS="-O3 -fPIC"

  Edit libtool to fix Intel build issue.
    Comment out setting postdep_objects.

  Run "make" and "make install".
    make
    make install

  Test
    make check


10. mercurial

  Depends on Python.

  Download and unpack the mercurial tarball.
    cd $HOME/src/mercurial-1.5.2
    wget http://mercurial.selenic.com/release/mercurial-1.5.2.tar.gz
    tar -zxvf mercurial-1.5.2.tar.gz

  Install using python.
    cd mercurial-1.5.2
    python setup.py install --prefix=$TOOLS_DIR
  

11. PETSc

  Depends on MPI, Python, math libraries.

  Checkout the development version of PETSc.
    cd $HOME/src
    hg clone http://petsc.cs.iit.edu/petsc/petsc-dev
    cd petsc-dev/config
    hg clone http://petsc.cs.iit.edu/petsc/BuildSystem BuildSystem

  Run configure in the *source* directory.
    cd $HOME/src/petsc-dev
    config/configure.py --PETSC_ARCH=${TOOLS_FORMAT}_opt --with-debugging=0 --with-clanguage=c++ --with-mpi-compilers=1 --with-shared=1 --with-dynamic=1 --with-64-bit-points=1 --with-large-file-io=1 --with-blas-lib=[/opt/intel/mkl/10.0.1.014/lib/em64t/libmkl.so,/opt/intel/mkl/10.0.1.014/lib/em64t/libguide.so] --with-lapack-lib=/opt/intel/mkl/10.0.1.014/lib/em64t/libmkl_lapack.so --with-lgrind=0 --with-chaco=1 --download-chaco=1 --with-parmetis=1 --download-parmetis=1 --with-sieve=1 --with-boost=1 --download-boost=1 --with-ml=1 --download-ml=1

  Run "make" and "make test"
    make

  "make test" may fail if the cluster is not setup to run MPI jobs on
  the login node. However, the code test examples should build.


12. nemesis

  Depends on Python, MPI, pythia.

  Check out nemesis from the publicly accessible CIG SVN repository.
    cd $HOME/src
    svn co http://geodynamics.org/svn/cig/cs/nemesis/trunk nemesis-dev
    cd nemesis-dev
    autoreconf -if

  Run configure in the build dir.
    cd $HOME/scratch/build/$TOOLS_FORMAT/nemesis-dev
    $HOME/src/nemesis-dev/configure --prefix=${TOOLS_DIR}

 Run "make" and "make install".
    make
    make install

  Test
    nemesis
    import mpi

  Running nemesis should startup Python. Nemesis is an MPI wrapper around Python.
  
 
13. PyLith (finally!)

    Depends on all of the above.

    Check out PyLith from the publicly accessible CIG SVN repository.
    cd $HOME/src
    svn co http://geodynamics.org/svn/cig/short/3D/PyLith/trunk pylith-dev
    cd pylith-dev
    autoreconf -if

  Run configure in the build dir.
    cd $HOME/scratch/build/$TOOLS_FORMAT/pylith-dev
    ${HOME}/src/pylith-dev/configure  --config-cache  --enable-swig --enable-testing --enable-cubit --prefix=${PYLITH_DIR}  CPPFLAGS="-I${TOOLS_DIR}/include -NDEBUG"  LDFLAGS="-L${TOOLS_DIR}/lib "  CC=mpicc CXX=mpiCC  CFLAGS="-O3 -fPIC" CXXFLAGS="-O3 -fPIC"

 Run "make" and "make install".
    make
    make install

  Test
    make check

