Created on: 10/24/17, Last Updated on: 4/16/18
Category: GIS , Tags: #CMAQ #Ubuntu

Preliminary

Essential Packages

sudo apt-get update
sudo apt-get install g++ vim make csh tcsh

MPICH

Install MPICH:

sudo apt-get install libcr-dev mpich2 mpich2-doc

Test MPICH:

nano hello.c

Copy and paste the test code:

/* C Example */
#include <mpi.h>
#include <stdio.h>

int main (int argc, char* argv[])
{
  int rank, size;

  // starts MPI
  MPI_Init (&argc, &argv);      
  // get current process id
  MPI_Comm_rank (MPI_COMM_WORLD, &rank);        
  // get number of processes
  MPI_Comm_size (MPI_COMM_WORLD, &size);        

  printf( "Hello world from process %d of %d\n", rank, size );
  MPI_Finalize();
  return 0;
}

Run and test:

mpicc -o hello hello.c
mpirun -np 2 ./hello

The correct output would be like this:
Hello world from process 0 of 2
Hello world from process 1 of 2

Reference: https://jetcracker.wordpress.com/2012/03/01/how-to-install-mpi-in-ubuntu/

FAQ

Problem: Conflict with OpenMPI

Solution: Remove openmpi

apt-get remove openmpi-common
apt-get purge openmpi-common

Reference: https://stackoverflow.com/questions/7011519/mpiexec-fails-as-mpi-init-aborts

zlib

Install zlib:

wget http://www.zlib.net/zlib-1.2.11.tar.gz
tar xf zlib-1.2.11.tar.gz
cd zlib-1.2.11/
./configure
make check
make install

Uninstall zlib:

cd ~/tmp/zlib-1.2.11
make uninstall

Reference: https://fanzheng.org/archives/14#6.1

HDF5

Install HDF5:

cd /usr/local/app
wget https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.8/hdf5-1.8.16/src/hdf5-1.8.16.tar.gz
tar xf hdf5-1.8.16.tar.gz
cd hdf5-1.8.16/
./configure --with-zlib=/usr/local --prefix=/usr/local
make
make check
make install

Build, test, and install HDF5 parallel support:

cd /usr/local/app
wget https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.8/hdf5-1.8.16/src/hdf5-1.8.16.tar.gz
tar xf hdf5-1.8.16.tar.gz
cd /usr/local/app/hdf5-1.8.16
CC=mpicc CPPFLAGS=-I${H5DIR}/include LDFLAGS=-L${H5DIR}/lib
./configure --enable-parallel --with-zlib=/usr/local --prefix=/usr/local
make
make check
make install

Uninstall HDF5:

cd /usr/local/app/hdf5-1.8.16
make uninstall

Reference: https://support.hdfgroup.org/HDF5/release/obtainsrc518.html#conf
https://fanzheng.org/archives/14#6.2

netCDF

netCDF-C 4.4.0

Download netcdf-c-4.4.0:

cd /usr/local/app
wget https://github.com/Unidata/netcdf-c/archive/v4.4.0.tar.gz
sudo apt-get install m4

Set variables:

nano ~/.bashrc
  export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
  export H5DIR=/usr/local
  export NCDIR=/usr/local
  export CPPFLAGS=-I${H5DIR}/include
  export LDFLAGS=-L${H5DIR}/lib
source ~/.bashrc

Building netCDF-4 and the Remote Data Client from source:

tar xf v4.4.0.tar.gz
cd /usr/local/app/netcdf-c-4.4.0/
./configure --prefix=${NCDIR}
make check
sudo make install

Build, test, and install netCDF-4 with parallel support:

cd /usr/local/app/netcdf-c-4.4.0
CC=mpicc CPPFLAGS=-I${H5DIR}/include LDFLAGS=-L${H5DIR}/lib
./configure --disable-shared --enable-parallel-tests --prefix=/usr/local
make check
make install

Test after installation:

nc-config

Reference: http://www.unidata.ucar.edu/software/netcdf/docs/getting_and_building_netcdf.html https://fanzheng.org/archives/14#6.3

netCDF-Fortran

Install netCDF-Fortran:

cd /usr/local/app
wget https://github.com/Unidata/netcdf-fortran/archive/v4.4.3.tar.gz
tar xf netcdf-fortran-4.4.3.tar.gz
cd netcdf-fortran-4.4.3
./configure
make check
sudo make install

Test:

nf-config

Reference:

https://fanzheng.org/archives/14#6.4

I/O API

Download I/O API 3.1:

wget https://www.cmascenter.org/ioapi/download/ioapi-3.1.tar.gz
cd /usr/local/app
wget http://cloud.csiss.gmu.edu/CMAQ/ioapi-3.1.tar.gz
mkdir ioapi31
cp ioapi-3.1.tar.gz ioapi31
cd ioapi31
tar xf ioapi-3.1.tar.gz
mkdir Linux2_x86_64

Configure and build:

nano ~/.bashrc

  export BIN=Linux2_x86_64

source ~/.bashrc

mv Makefile Makefile.backup
cd ioapi
cp Makefile.nocpl Makefile

nano Makefile

  BASEDIR = /usr/local/app/ioapi31

nano Makeinclude.Linux2_x86_64

  ARCHFLAGS = \
   -DAUTO_ARRAYS=1 \
   -DF90=1 -DFLDMN=1 \
   -DFSTR_L=int \
   -DIOAPI_NO_STDOUT=1 \
  # -DNEED_ARGS=1

make

Verify installation, check if libioapi.a exists or not:

ll /usr/local/app/ioapi31/Linux2_x86_64

Reference:

http://blog.sciencenet.cn/home.php?mod=space&uid=200199&do=blog&id=1050897

CMAQ Installation

Download CMAQ

CMAQ 5.1 document: https://www.cmascenter.org/download/software/cmaq/tracker/cmaq5-1.cfm

cd /usr/local/app
wget http://cloud.csiss.gmu.edu/CMAQ/CMAQv5.1.Jan2016.tar
wget http://cloud.csiss.gmu.edu/CMAQ/CMAQv5.1_CMAQ_TOOLS.Apr2016.tar
wget http://cloud.csiss.gmu.edu/CMAQ/DATA.CMAQv5.1.Nov2015.tar
wget http://cloud.csiss.gmu.edu/CMAQ/DATA_REF.CMAQv5.1.Nov2015.tar
tar xf CMAQv5.1.Jan2016.tar
tar xf CMAQv5.1_CMAQ_TOOLS.Apr2016.tar
tar xf DATA.CMAQv5.1.Nov2015.tar

Download CMAQ v5.1 by Github:

git clone -b 5.1 https://github.com/CMASCenter/CMAQ.git

Configure CMAQ

Configure the environment path:

nano ~/.bashrc

  M3HOME=/usr/local/app/CMAQv5.1
  M3SCRIPT=/usr/local/app/CMAQv5.1/scripts
  M3MODEL=/usr/local/app/CMAQv5.1/models
  M3DATA=/usr/local/app/CMAQv5.1/data
  M3LIB=/usr/local/app/CMAQv5.1/lib/x86_64/gcc

source ~/.bashrc

Configure config.cmaq, set M3HOME and complier:

cd $M3SCRIPT
nano config.cmaq

  #> model source code repository location
   setenv M3HOME  /usr/local/app/CMAQv5.1
   setenv M3MODEL ${M3HOME}/models
   setenv M3DATA  ${M3HOME}/data

  ...

  #> Set the compiler
  setenv compiler gcc  #> options: intel, pgi, gcc

./config.cmaq

Install External Libraries

cd $M3LIB
ln -s /usr/lib/mpich mpi
mkdir netcdf
cd netcdf
ln -s /usr/local/lib lib
ln -s /usr/local/include include
cd ..
mkdir ioapi
cd ioapi
ln -s /usr/local/app/ioapi31/Linux2_x86_64 lib
ln -s /usr/local/app/ioapi31/Linux2_x86_64 include
ln -s /usr/local/app/ioapi31/ioapi/fixed_src src

Build CMAQ Executables

Build Bldmake Utility:

cd $M3HOME/scripts/build
./bldit.bldmake

Build ICON:

cd $M3HOME/scripts/icon
./bldit.icon |& tee bldit.icon.profile.log

Build BCON:

cd $M3HOME/scripts/bcon
./bldit.bcon |& tee bldit.bcon.profile.log

Build CCTM:

cd $M3HOME/scripts/cctm
./bldit.cctm |& tee bldit.cctm.log

Build MCIP, set M3LIB and uncomment gfortran:

nano $M3HOME/scripts/mcip/src/Makefile

  # Requirements: set M3LIB before running this script
  M3LIB = /usr/local/app/CMAQv5.1/lib/x86_64/gcc

  ...

  #...gfortran
  FC      = gfortran
  NETCDF = $(M3LIB)/netcdf
  IOAPI_ROOT = $(M3LIB)/ioapi
  FFLAGS  = -O3 -I$(NETCDF)/include -I$(IOAPI_ROOT)/include
  #FFLAGS = -g -O0  \
  #          -ffpe-trap='invalid','zero','overflow','underflow'  \
  #          -I$(NETCDF)/include -I$(IOAPI_ROOT)/include
  LIBS    = -L$(IOAPI_ROOT)/lib -lioapi  \
            -L$(NETCDF)/lib -lnetcdf -lgomp -lnetcdff

  #...Intel Fortran
  #FC      = ifort
  #NETCDF = $(M3LIB)/netcdf
  #IOAPI_ROOT = $(M3LIB)/ioapi
  ##FFLAGS  = -g -O0 -check all -C -traceback -FR -I$(NETCDF)/include  \
  ##          -I$(IOAPI_ROOT)/include
  #FFLAGS  = -FR -O3 -traceback -openmp -I$(NETCDF)/include -I$(IOAPI_ROOT)/include
  #LIBS    = -L$(IOAPI_ROOT)/lib -lioapi \
  #          -L$(NETCDF)/lib -lnetcdf
  #         #-limf

make

Reference: http://www.u396.com/cmaq-compile-install.html

FAQ

Fail when performing “make check” on netcdf-c

alt text

Solution: Install hdf5 1.8.16 or earlier, and netcdf 4.4.0

Reference:
https://www.unidata.ucar.edu/support/help/MailArchives/netcdf/msg13567.html

Errors when performing “make check” command during netCDF-fortran installation

alt text

Solution: Unknown. But netcdf-fortran could be successfully installed eventually.

“No command ‘setenv’ found” error when running config.cmaq

Solution: Install C-Shell (csh)
sudo apt-get install csh

Reference: https://ubuntuforums.org/showthread.php?t=1182511

“make[1]: *** [mcip.exe] Error 1” error when build MCIP

Solution: add -lnetcdff in Makefile

nano $M3HOME/scripts/mcip/src/Makefile

  #...gfortran
  FC      = gfortran
  NETCDF = $(M3LIB)/netcdf
  IOAPI_ROOT = $(M3LIB)/ioapi
  FFLAGS  = -O3 -I$(NETCDF)/include -I$(IOAPI_ROOT)/include
  #FFLAGS = -g -O0  \
  #          -ffpe-trap='invalid','zero','overflow','underflow'  \
  #          -I$(NETCDF)/include -I$(IOAPI_ROOT)/include
  LIBS    = -L$(IOAPI_ROOT)/lib -lioapi  \
            -L$(NETCDF)/lib -lnetcdf -lgomp -lnetcdff

Reference: https://stackoverflow.com/questions/40104267/error-undefined-reference-to-netcdf-functions

“collect2: error: ld returned 1 exit status” error when build MCIP