FLEXPART installation guide

This post shares my experience installing the FLEXPART software.

What’s FLEXPART and about this guide

FLEXPART (“FLEXible PARTicle dispersion model”) is a Lagrangian
transport and dispersion model suitable for the simulation of a large
range of atmospheric transport processes.

I am currently learning to use this tool to facilitate researches on atmospheric moisture transports. So this guide is by no means comprehensive, nor guaranteed to work 100% of the time in a different setup. Rather, it is my working notes recording the procedures I took to get it installed on my 2 Linux desktop machines (both running Manjaro Linux). Hopefully it would be of any help to others.

References

System requirements

OS: Linux, it is said to work also on Mac.

Basic build tools (probably already installed in one’s Linux distro):

  • g++
  • gfortran
  • autoconf
  • libtool
  • automake
  • flex
  • bison
  • git
  • python3

Things I installed manually on desktop (Manjaro Linux):

  • cmake
  • gcc-fortran
  • netcdf-fortran
  • netcdf-fortran-openmpi (necessary?)
  • hdf5

These things should be readily available in the Linux distro’s software repository.

Dependency libraries

GRIB library

Note that FLEXPART used GRIB_API from ECMWF for reading GRIB formatted data. Version >= 1.6.1 of GRIB_API is needed for reading GRIB2 format.

However, GRIB_API has been deprecated and ECMWF now uses eccodes for I/O of GRIB formatted data.

Older versions of FLEXPART can use either GRIB_API or eccodes, the source code of the model doesn’t need change, but the makefile does.

Jasper library

"Nowadays, the jasper lib is not required separately, grib_api or eccodes will handle it."

I didn’t install this.

Install eccodes

Source code of eccodes: https://confluence.ecmwf.int/display/ECC/Releases

Installation guide: https://confluence.ecmwf.int/display/ECC/ecCodes+installation

Installation steps:

  1. download tarbar eccodes-x.y.z-Source.tar.gz from above link.
  2. in shell:
> tar -xzf  eccodes-x.y.z-Source.tar.gz
> mkdir build ; cd build

> cmake -DCMAKE_INSTALL_PREFIX=/usr/local/ ../eccodes-x.y.z-Source
...

> make
> ctest
> make install

cmake options:

  • DCMAKE_INSTALL_PREFIX: default to /usr/local/

NOTE that the default /usr/local/ location is causing me a bit trouble, the eccodes python module cannot find it. I had to symlink these files:

  • libeccodes_f90.so and libeccodes.so from /usr/local/lib to /usr/lib/
  • eccodes_config.h eccodes_ecbuild_config.h eccodes.h eccodes.mod eccodes_version.h eccodes_windef.h grib_api.h grib_api.mod from /usr/local/include/ to /usr/include
  1. install eccodes python module:

After make install, install the Python package using

pip install eccodes

Tried conda install -c conda-forge eccodes, didn’t work.

After installation, execute the following to test:

$ python -m eccodes selfcheck
Found: ecCodes v2.21.0.
Your system is ready.

Install GRIB_API

Didn’t do this step. See this excerpt:

Please note that GRIB-API is no longer maintained. The primary GRIB
encoding–decoding package used at the ECMWF is currently ecCodes. Any
new features for the handling of GRIB files will only be developed in
ecCodes. However, for FLEXPART v10.4 grib_api is sufficient. We
keep the grib_api instructions for backward consistency.
 
For future versions, ensure that the path *usr/local/lib* is in the
environment variable $PATH; otherwise, ecCodes may not find it.

Install GRIB_API if needed:

tar xvzf grib_api-1.28.0-Source.tar.gz
mkdir grib_api
cd grib_api-1.28.0-Source
./configure --prefix=/usr/local/grib_api --with-jasper=/usr/local/jasper
make
make check
make install

Install FLEXPART

  1. download source from: https://www.flexpart.eu/wiki/FpRoadmap

  2. unpack the tarbar:

    tar -xvf flexpart_v10.4.tar
    

    the output files are in {flexpart_root}.

  3. edit the {flexpart_root}/src/makefile:

    changes I made:

#ROOT_DIR = /homevip/flexpart/
ROOT_DIR = /usr/local

F90	  = /usr/bin/gfortran
MPIF90    = /usr/bin/mpifort

#INCPATH1  = ${ROOT_DIR}/gcc-5.4.0/include
INCPATH1  = /usr/include
INCPATH2  = /usr/local/include
#LIBPATH1 = ${ROOT_DIR}/gcc-5.4.0/lib
LIBPATH1 = /usr/lib
LIBPATH2 = /usr/local/lib

...

## LIBRARIES
#LIBS = -lgrib_api_f90 -lgrib_api -lm -ljasper -lnetcdff
#LIBS = -lgrib_api_f90 -lgrib_api -lm -ljasper $(NCOPT)
LIBS = -leccodes_f90 -leccodes -lm -ljasper $(NCOPT)
**NOTE** that the last line is telling the compiler to use
`eccodes` instead of `GRIB_API`.
  1. set LD_LIBRARY_PATH (not sure necessary):

    export LD_LIBRARY_PATH="/usr/lib/:$LD_LIBRARY_PATH"
    
  2. make with netcdf output support:

    make ncf=yes
    

    To see if everything goes ok, try:

    ./FLEXPART
    

    The output:

Welcome to FLEXPART Version 10.4 (2019-11-12)
FLEXPART is free software released under the GNU General Public License.
#### TRAJECTORY MODEL ERROR! FILE "pathnames"#### 
#### CANNOT BE OPENED IN THE CURRENT WORKING #### 
#### DIRECTORY.                              ####

Install flex_extract

flex_extract is for:

  1. obtaining meteorological data from ECMWF (ERA40, ERA-I, ERA5, 20C reanalyses or operational), in GRIB format.
  2. pre-process obtained data for usage as FLEXPART input.

References

  • Documentation webpage: https://www.flexpart.eu/flex_extract/index.html
  • Software Installation Plan (SIP): <file:///home/guangzhi/Documents/MeiTingTrunk/climate/_collections/Unknown_unknown_flex_extract software installation plan (SIP).pdf>
  • Software User Tutorial (SUT): <file:///home/guangzhi/Documents/MeiTingTrunk/climate/_collections/Unknown_unknown_flex_extract software user tutorial.pdf>

ECMWF accounts

One needs to register at ECMWF for data retrieval.

Major modes

  • Remote (member-state users only)
  • Gateway (member-state users only)
  • Local, member-state user
  • Local, public user

Will focus on local, public only.

System environment

Python:

  • Python 3
  • numpy
  • genshi
  • eccodes
  • ecmwf-api-client (for everything except ERA5)
  • cdsapi (for ERA5 and member user)

These can be installed in conda, or using pip. Check installation using:

import eccodes
import genshi
import numpy
import ecmwfapi
import cdsapi

Fortran:

  • gfortran
  • eccodes (see above)
  • emoslib (Fortran interpolation lib)
  • fftw3 (required by emoslib)

LD_LIBRARY_PATH is recommended to set to include installed libs.

Install the emoslib (optional) dependency

NOTE that this is only needed for those data sets which don’t contain vertical velocity as etadot.

NOTE emoslib has been deprecated by ECMWF.

Steps:

> tar -xzf libemos-4.0.0-Source.tar.gz
> cd libemos-4.0.0-Source
> mkdir build ; cd build 
> cmake -DCMAKE_INSTALL_PREFIX=/usr/local/emos -DECCODES_PATH=/usr/local/eccodes -DCMAKE_Fortran_FLAGS="-fallow-argument-mismatch" ../libemos-4.5.9-Source

...

> make
> make test
> make install

Trouble shooting 1

Trouble shooting 2

  • more undefined references to `grib_get_error_message':
/usr/bin/ld: ../lib/libemosR64.a(describe_input_field.c.o):/home/guangzhi/Downloads/emos_source/libemos-4.5.9-Source/interpolation/describe_input_field.c:527: more undefined references to `grib_get_error_message' follow
collect2: error: ld returned 1 exit status
make[2]: *** [sandbox/CMakeFiles/emos_tool.dir/build.make:106: bin/emos_tool] Error 1
make[1]: *** [CMakeFiles/Makefile2:1474: sandbox/CMakeFiles/emos_tool.dir/all] Error 2
make: *** [Makefile:182: all] Error 2

See this: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239438.

The proposed solution is to set environment variable before cmake (NOT make):

export LDFLAGS="-leccodes"

Install flex_extract

  1. Get the repository:
git clone --single-branch --branch master https://www.flexpart.eu/gitmob/flex_extract
  1. Setup a Python environment, see above.

  2. Edit flex_extract/Source/Fortran/makefile_local_gfortran:

    Change:

    • ECCODES_INCLUDE_DIR to /usr/local/include, because eccodes is installed in /usr/local

    • EMOSLIB to -L/usr/local/emos/lib -lemosR64, because Emoslib is installed there.

  3. run setup_local.sh to install

    Installation is successful if the below is shown:

(eccodes):flex_extract$ ./setup_local.sh 
WARNING: installdir has not been specified
flex_extract will be installed in here by compiling the Fortran source in /home/guangzhi/Downloads/flex_extract/Source/Fortran
Install flex_extract_v7.1.2 software at local in directory /home/guangzhi/Downloads/flex_extract

Using makefile: makefile_local_gfortran
/home/guangzhi/.local/anaconda3/envs/eccodes/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
self.stdin = io.open(p2cwrite, 'wb', bufsize)
/home/guangzhi/.local/anaconda3/envs/eccodes/lib/python3.9/subprocess.py:941: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
self.stdout = io.open(c2pread, 'rb', bufsize)
/home/guangzhi/.local/anaconda3/envs/eccodes/lib/python3.9/subprocess.py:946: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
self.stderr = io.open(errread, 'rb', bufsize)
gfortran  -O3  -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian  -c	./rwgrib2.f90
gfortran  -O3  -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian  -c	./phgrreal.f90
gfortran  -O3  -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian  -c	./grphreal.f90
gfortran  -O3  -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian  -c	./ftrafo.f90
gfortran  -O3  -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian  -c	./calc_etadot.f90
gfortran  -O3  -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian  -c	./posnam.f90
gfortran rwgrib2.o calc_etadot.o ftrafo.o grphreal.o posnam.o phgrreal.o -o calc_etadot_fast.out  -O3  -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -fopenmp
ln -sf calc_etadot_fast.out calc_etadot

lrwxrwxrwx 1 guangzhi guangzhi 20 Mar 31 11:51 ./calc_etadot -> calc_etadot_fast.out

NOTE that this is different from installation guides in the documentation paper (Pisso et al. 2019). This version is likely more up-to-date because the Pisso et al. 2019 instructions use a install.py script which is still Python2.

The CONVERT2 executable is also not compiled using the new method. The makefile doesn’t specify such an EXE.

  1. test flex_extract
> cd Testing/Installation/Calc_etadot
# execute the Fortran progam without arguments
> ../../../Source/Fortran/calc_etadot
Installation is successful if the below is shown:

    STATISTICS:          98842.4598  98709.7359   5120.5385
    STOP SUCCESSFULLY FINISHED calc_etadot: CONGRATULATIONS

23 Comments

  1. Hi GUANGZHI, your post has been really helpful as i have been having problems with FLEXPART installation for quite awhile. Can I ask for clarification on Trouble shooting 2 session for emoslib? I am not sure what was done to overcome the error “grib_get_error_message”
    Thank you so much!

    • Hi JIA HAO,

      In case you have not solved the issue, consider trying the following and see if it works.

      1. clear the build directory.
      2. set the LDFLAGS variable like so: export LDFLAGS="-leccodes" (if you are using bash shell).
      2b. (alternative to step 2, I’m not sure which is better but worth a try if step 2 doesn’t work): LDFLAGS="-L${ECCODES}/lib -leccodes", where ${ECCODES} is the eccodes installation location.
      3. (not sure this is necessary but for good measure just set it): export LD_LIBRARY_PATH="${ECCODES}/lib:$LD_LIBRARY_PATH". Again, ${ECCODES} is the eccodes installation location.
      4. run that long cmake command again.
      5. Hopefully it works.

  2. (let me try that again) Hi Guangzhi, I’m running into the same issue but unfortunately setting these variables is not fixing the problem 🙁 would you happen to know of any other solutions? Also, you mentioned that the emos lib is optional, in the case where you don’t install it, which directory can you give the emos lib variable in makefile_local_gfortran?

  3. Hi, I’ve followed your working notes and I have the same error with “Trouble Shooting 2″ on emoslib. However, when I do the command as export LDFLAGS=”-leccodes” before cmake, it makes same error. Any recommendation for this error?

    • Hi Woohui,
      I’m not sure. I haven’t used FLEXPART for a good while.
      Maybe try running `make clean` to clear things up, before doing the `export LDFLAGS=”-leccodes”` part?

      • Thank you for the suggestion, but still, I have the same error.
        The other suggestions (to Jin Hao) cannot solve the problem. 🙁

        • hello, have you solved the problem yet? I also have the same problem and still seeking the solver

        • Hi,

          Have you found a solution?

          I repeated the emoslib installation today and got the same error of “grib_get_error_message”, and again the export LDFLAGS="-leccodes" trick worked for me.
          I’m not sure how to help. What version of libemos were you using?

          • I use libemos 4.5.9, and I still not found the solution yet. I currently installed flexpart without emoslib. will it be okay I wonder? and what version do you use?

            • I used 4.5.9 as well.
              From descriptions in https://www.flexpart.eu/wiki/FpCompileLibs, I think it is Ok to not install emoslib. Does your FLEXPART run without emoslib?

              • yeah, I installed it without emoslib, do you have enough experiences on running flexpart? I’m really new with this programme, now I have problem on running the programme. when i run the command ./FLEXPART it said that the available file cannot be opened. do you have any idea whats going on? and how do you make available file btw? are we supposed to change the name on available file manually?

                I’m sorry for my bad english

                • I only did a couple of test runs of FLEXPART, without even analyzing the outputs. It is indeed a bit convoluted, I suggest giving the paper (https://gmd.copernicus.org/articles/12/4955/2019) a good read and following the instructions therein (particularly the Appendix sections). I don’t think we do manual file renaming. The data preparation is done using their “flex_extract” tool. Then you run the ./FLEXPART command.

  4. hello thank you for the tutorial, it’s really helpful. But I kinda stuck when installing flexpart. when I run command make ncf=yes

    Cannot open module file ‘grib_api.mod’ for reading at (1): No such file or directory
    compilation terminated.
    make: *** [makefile:236: class_gribfile_mod.o] Error 1

    that was the output. Can uou help me

    • Hi,
      Did you successfully install GRIB_API or eccodes? In my tutorial I installed eccodes. In this case, I also changed the path_to_flexpart_folder/src/makefile file to let it use eccodes: change

      LIBS = -lgrib_api_f90 -lgrib_api -lm -ljasper $(NCOPT)
      

      to

      LIBS = -leccodes_f90 -leccodes -lm -ljasper $(NCOPT)
      

      Please doulbe-check that you followed these.
      Hope that helps

    • ​Perhaps I have solved the problem. Copy this mode from other folder (usr/local/grib_api/include/) to /usr/include.

Leave a Reply to hoyu Cancel reply