What’s FLEXPART and about this guide
FLEXPART (“FLEXible PARTicle dispersion model”) is a Lagrangian
transport and dispersion model suitable for the simulation of a large
range of atmospheric transport processes.
I am currently learning to use this tool to facilitate researches on atmospheric moisture transports. So this guide is by no means comprehensive, nor guaranteed to work 100% of the time in a different setup. Rather, it is my working notes recording the procedures I took to get it installed on my 2 Linux desktop machines (both running Manjaro Linux). Hopefully it would be of any help to others.
References
- Official site install guide: https://www.flexpart.eu/wiki/FpInstall
- Another guide: https://www.jianshu.com/p/6bc7cee6c9bf
- See also Appendix in Pisso et al. 2019 (https://doi.org/10.5194/gmd-12-4955-2019).
System requirements
OS: Linux, it is said to work also on Mac.
Basic build tools (probably already installed in one’s Linux distro):
- g++
- gfortran
- autoconf
- libtool
- automake
- flex
- bison
- git
- python3
Things I installed manually on desktop (Manjaro Linux):
- cmake
- gcc-fortran
- netcdf-fortran
- netcdf-fortran-openmpi (necessary?)
- hdf5
These things should be readily available in the Linux distro’s software repository.
Dependency libraries
GRIB library
Note that FLEXPART used GRIB_API from ECMWF for reading GRIB
formatted data.
Version >= 1.6.1 of GRIB_API is needed for reading GRIB2 format.
However, GRIB_API has been deprecated and ECMWF now uses eccodes
for I/O of GRIB formatted data.
Older versions of FLEXPART can use either GRIB_API or eccodes, the
source code of the model doesn’t need change, but the makefile does.
Jasper library
"Nowadays, the jasper lib is not required separately, grib_api or eccodes will handle it."
I didn’t install this.
Install eccodes
Source code of eccodes:
https://confluence.ecmwf.int/display/ECC/Releases
Installation guide: https://confluence.ecmwf.int/display/ECC/ecCodes+installation
Installation steps:
- download tarbar
eccodes-x.y.z-Source.tar.gzfrom above link. - in shell:
> tar -xzf eccodes-x.y.z-Source.tar.gz > mkdir build ; cd build > cmake -DCMAKE_INSTALL_PREFIX=/usr/local/ ../eccodes-x.y.z-Source ... > make > ctest > make install
cmake options:
DCMAKE_INSTALL_PREFIX: default to/usr/local/
NOTE that the default /usr/local/ location is causing me a bit
trouble, the eccodes python module cannot find it. I had to
symlink these files:
libeccodes_f90.soandlibeccodes.sofrom/usr/local/libto/usr/lib/eccodes_config.heccodes_ecbuild_config.heccodes.heccodes.modeccodes_version.heccodes_windef.hgrib_api.hgrib_api.modfrom/usr/local/include/to/usr/include
- install
eccodespython module:
After make install, install the Python package using
pip install eccodes
Tried conda install -c conda-forge eccodes, didn’t work.
After installation, execute the following to test:
$ python -m eccodes selfcheck
Found: ecCodes v2.21.0.
Your system is ready.
Install GRIB_API
Didn’t do this step. See this excerpt:
Please note that GRIB-API is no longer maintained. The primary GRIB
encoding–decoding package used at the ECMWF is currently ecCodes. Any
new features for the handling of GRIB files will only be developed in
ecCodes. However, for FLEXPART v10.4 grib_api is sufficient. We
keep the grib_api instructions for backward consistency.
For future versions, ensure that the path *usr/local/lib* is in the
environment variable $PATH; otherwise, ecCodes may not find it.
Install GRIB_API if needed:
tar xvzf grib_api-1.28.0-Source.tar.gz mkdir grib_api cd grib_api-1.28.0-Source ./configure --prefix=/usr/local/grib_api --with-jasper=/usr/local/jasper make make check make install
Install FLEXPART
-
download source from: https://www.flexpart.eu/wiki/FpRoadmap
-
unpack the tarbar:
tar -xvf flexpart_v10.4.tarthe output files are in
{flexpart_root}. -
edit the
{flexpart_root}/src/makefile:changes I made:
#ROOT_DIR = /homevip/flexpart/
ROOT_DIR = /usr/local
F90 = /usr/bin/gfortran
MPIF90 = /usr/bin/mpifort
#INCPATH1 = ${ROOT_DIR}/gcc-5.4.0/include
INCPATH1 = /usr/include
INCPATH2 = /usr/local/include
#LIBPATH1 = ${ROOT_DIR}/gcc-5.4.0/lib
LIBPATH1 = /usr/lib
LIBPATH2 = /usr/local/lib
...
## LIBRARIES
#LIBS = -lgrib_api_f90 -lgrib_api -lm -ljasper -lnetcdff
#LIBS = -lgrib_api_f90 -lgrib_api -lm -ljasper $(NCOPT)
LIBS = -leccodes_f90 -leccodes -lm -ljasper $(NCOPT)
**NOTE** that the last line is telling the compiler to use
`eccodes` instead of `GRIB_API`.
-
set
LD_LIBRARY_PATH(not sure necessary):export LD_LIBRARY_PATH="/usr/lib/:$LD_LIBRARY_PATH" -
make with netcdf output support:
make ncf=yesTo see if everything goes ok, try:
./FLEXPARTThe output:
Welcome to FLEXPART Version 10.4 (2019-11-12) FLEXPART is free software released under the GNU General Public License. #### TRAJECTORY MODEL ERROR! FILE "pathnames"#### #### CANNOT BE OPENED IN THE CURRENT WORKING #### #### DIRECTORY. ####
Install flex_extract
flex_extract is for:
- obtaining meteorological data from ECMWF (ERA40, ERA-I, ERA5, 20C reanalyses or operational), in GRIB format.
- pre-process obtained data for usage as FLEXPART input.
References
- Documentation webpage: https://www.flexpart.eu/flex_extract/index.html
- Software Installation Plan (SIP): <file:///home/guangzhi/Documents/MeiTingTrunk/climate/_collections/Unknown_unknown_flex_extract software installation plan (SIP).pdf>
- Software User Tutorial (SUT): <file:///home/guangzhi/Documents/MeiTingTrunk/climate/_collections/Unknown_unknown_flex_extract software user tutorial.pdf>
ECMWF accounts
One needs to register at ECMWF for data retrieval.
Major modes
- Remote (member-state users only)
- Gateway (member-state users only)
- Local, member-state user
- Local, public user
Will focus on local, public only.
System environment
Python:
- Python 3
- numpy
- genshi
- eccodes
- ecmwf-api-client (for everything except ERA5)
- cdsapi (for ERA5 and member user)
These can be installed in conda, or using pip. Check installation using:
import eccodes
import genshi
import numpy
import ecmwfapi
import cdsapi
Fortran:
- gfortran
- eccodes (see above)
- emoslib (Fortran interpolation lib)
- fftw3 (required by emoslib)
LD_LIBRARY_PATH is recommended to set to include installed libs.
Install the emoslib (optional) dependency
NOTE that this is only needed for those data sets which don’t contain vertical velocity as etadot.
NOTE emoslib has been deprecated by ECMWF.
- Download: https://confluence.ecmwf.int/display/EMOS/Emoslib
- Guide: https://confluence.ecmwf.int/display/EMOS/Installation+Guide
Steps:
> tar -xzf libemos-4.0.0-Source.tar.gz > cd libemos-4.0.0-Source > mkdir build ; cd build > cmake -DCMAKE_INSTALL_PREFIX=/usr/local/emos -DECCODES_PATH=/usr/local/eccodes -DCMAKE_Fortran_FLAGS="-fallow-argument-mismatch" ../libemos-4.5.9-Source ... > make > make test > make install
Trouble shooting 1
-
Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(8)/INTEGER(4))This is due to a feature in gcc version 10. See this https://github.com/Unidata/netcdf-fortran/issues/212#issuecomment-623046448
Solution: add this Fortran flag:
-DCMAKE_Fortran_FLAGS="-fallow-argument-mismatch".
Trouble shooting 2
more undefined references to `grib_get_error_message':
/usr/bin/ld: ../lib/libemosR64.a(describe_input_field.c.o):/home/guangzhi/Downloads/emos_source/libemos-4.5.9-Source/interpolation/describe_input_field.c:527: more undefined references to `grib_get_error_message' follow collect2: error: ld returned 1 exit status make[2]: *** [sandbox/CMakeFiles/emos_tool.dir/build.make:106: bin/emos_tool] Error 1 make[1]: *** [CMakeFiles/Makefile2:1474: sandbox/CMakeFiles/emos_tool.dir/all] Error 2 make: *** [Makefile:182: all] Error 2
See this: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239438.
The proposed solution is to set environment variable
before cmake (NOT make):
export LDFLAGS="-leccodes"
Install flex_extract
- Get the repository:
git clone --single-branch --branch master https://www.flexpart.eu/gitmob/flex_extract
-
Setup a Python environment, see above.
-
Edit
flex_extract/Source/Fortran/makefile_local_gfortran:Change:
-
ECCODES_INCLUDE_DIRto/usr/local/include, becauseeccodesis installed in/usr/local -
EMOSLIBto-L/usr/local/emos/lib -lemosR64, becauseEmoslibis installed there.
-
-
run
setup_local.shto installInstallation is successful if the below is shown:
(eccodes):flex_extract$ ./setup_local.sh WARNING: installdir has not been specified flex_extract will be installed in here by compiling the Fortran source in /home/guangzhi/Downloads/flex_extract/Source/Fortran Install flex_extract_v7.1.2 software at local in directory /home/guangzhi/Downloads/flex_extract Using makefile: makefile_local_gfortran /home/guangzhi/.local/anaconda3/envs/eccodes/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used self.stdin = io.open(p2cwrite, 'wb', bufsize) /home/guangzhi/.local/anaconda3/envs/eccodes/lib/python3.9/subprocess.py:941: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used self.stdout = io.open(c2pread, 'rb', bufsize) /home/guangzhi/.local/anaconda3/envs/eccodes/lib/python3.9/subprocess.py:946: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used self.stderr = io.open(errread, 'rb', bufsize) gfortran -O3 -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian -c ./rwgrib2.f90 gfortran -O3 -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian -c ./phgrreal.f90 gfortran -O3 -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian -c ./grphreal.f90 gfortran -O3 -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian -c ./ftrafo.f90 gfortran -O3 -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian -c ./calc_etadot.f90 gfortran -O3 -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian -c ./posnam.f90 gfortran rwgrib2.o calc_etadot.o ftrafo.o grphreal.o posnam.o phgrreal.o -o calc_etadot_fast.out -O3 -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -L/usr/local/emos/lib -lemosR64 -fopenmp ln -sf calc_etadot_fast.out calc_etadot lrwxrwxrwx 1 guangzhi guangzhi 20 Mar 31 11:51 ./calc_etadot -> calc_etadot_fast.out
NOTE that this is different from installation guides in the
documentation paper (Pisso et al. 2019). This version is likely
more up-to-date because the Pisso et al. 2019 instructions use a
install.py script which is still Python2.
The CONVERT2 executable is also not compiled using the new
method. The makefile doesn’t specify such an EXE.
- test flex_extract
> cd Testing/Installation/Calc_etadot # execute the Fortran progam without arguments > ../../../Source/Fortran/calc_etadot
Installation is successful if the below is shown:
STATISTICS: 98842.4598 98709.7359 5120.5385
STOP SUCCESSFULLY FINISHED calc_etadot: CONGRATULATIONS
Hi GUANGZHI, your post has been really helpful as i have been having problems with FLEXPART installation for quite awhile. Can I ask for clarification on Trouble shooting 2 session for emoslib? I am not sure what was done to overcome the error “grib_get_error_message”
Thank you so much!
Hi JIA HAO,
In case you have not solved the issue, consider trying the following and see if it works.
1. clear the
builddirectory.2. set the
LDFLAGSvariable like so:export LDFLAGS="-leccodes"(if you are using bash shell).2b. (alternative to step 2, I’m not sure which is better but worth a try if step 2 doesn’t work):
LDFLAGS="-L${ECCODES}/lib -leccodes", where${ECCODES}is the eccodes installation location.3. (not sure this is necessary but for good measure just set it):
export LD_LIBRARY_PATH="${ECCODES}/lib:$LD_LIBRARY_PATH". Again,${ECCODES}is the eccodes installation location.4. run that long
cmakecommand again.5. Hopefully it works.
Hi
(let me try that again) Hi Guangzhi, I’m running into the same issue but unfortunately setting these variables is not fixing the problem 🙁 would you happen to know of any other solutions? Also, you mentioned that the emos lib is optional, in the case where you don’t install it, which directory can you give the emos lib variable in makefile_local_gfortran?
Hi Jada, Sorry for the delayed reply.
I’m afraid I don’t really have an answer to your question. Have you figured it out?
Hi, I’ve followed your working notes and I have the same error with “Trouble Shooting 2″ on emoslib. However, when I do the command as export LDFLAGS=”-leccodes” before cmake, it makes same error. Any recommendation for this error?
Hi Woohui,
I’m not sure. I haven’t used FLEXPART for a good while.
Maybe try running `make clean` to clear things up, before doing the `export LDFLAGS=”-leccodes”` part?
Thank you for the suggestion, but still, I have the same error.
The other suggestions (to Jin Hao) cannot solve the problem. 🙁
hello, have you solved the problem yet? I also have the same problem and still seeking the solver
Hi,
Have you found a solution?
I repeated the emoslib installation today and got the same error of “grib_get_error_message”, and again the
export LDFLAGS="-leccodes"trick worked for me.I’m not sure how to help. What version of libemos were you using?
I use libemos 4.5.9, and I still not found the solution yet. I currently installed flexpart without emoslib. will it be okay I wonder? and what version do you use?
I used 4.5.9 as well.
From descriptions in https://www.flexpart.eu/wiki/FpCompileLibs, I think it is Ok to not install emoslib. Does your FLEXPART run without emoslib?
yeah, I installed it without emoslib, do you have enough experiences on running flexpart? I’m really new with this programme, now I have problem on running the programme. when i run the command ./FLEXPART it said that the available file cannot be opened. do you have any idea whats going on? and how do you make available file btw? are we supposed to change the name on available file manually?
I’m sorry for my bad english
I only did a couple of test runs of FLEXPART, without even analyzing the outputs. It is indeed a bit convoluted, I suggest giving the paper (https://gmd.copernicus.org/articles/12/4955/2019) a good read and following the instructions therein (particularly the Appendix sections). I don’t think we do manual file renaming. The data preparation is done using their “flex_extract” tool. Then you run the ./FLEXPART command.
hello thank you for the tutorial, it’s really helpful. But I kinda stuck when installing flexpart. when I run command make ncf=yes
Cannot open module file ‘grib_api.mod’ for reading at (1): No such file or directory
compilation terminated.
make: *** [makefile:236: class_gribfile_mod.o] Error 1
that was the output. Can uou help me
Hi,
Did you successfully install GRIB_API or eccodes? In my tutorial I installed eccodes. In this case, I also changed the
path_to_flexpart_folder/src/makefilefile to let it use eccodes: changeto
Please doulbe-check that you followed these.
Hope that helps
I succesfully installed grib_api and eccodes, i change the lib to leccodes but I need to remove -ljasper this time, because -ljasper not found it saic, is it okay?
I’m really new with this program. I actually still not figured out yet how to use it. Excuse me for asking, do you run it with metview? or metview just to visual the result?
It is said in the official doc that “Nowadays, the jasper lib is not required separately, grib_api or eccodes will handle it.”, so I think it is fine to delete “-ljasper” if it is not found on your system.
I’m afraid I don’t have any good clue what’s causing your problem. I suggest joining the mailing list of FLEXPART (https://lists.univie.ac.at/mailman/listinfo/flexpart) and seeking some advice from there.
I didn’t use metview. In fact, my experience with FLEXPART is very limited, the initial plan didn’t work out well so I didn’t really go deeper with it.
thank you, but I cannot open the link, it need permission it says.
Did you mean you can’t open this link (https://lists.univie.ac.at/mailman/listinfo/flexpart)? That page has a title at the very top saying “FLEXPART — Mailing list for FLEXPART users”. How about this link https://www.flexpart.eu/wiki/RegisteredUser ? You can find that same link somewhere in the lower part of that page.
I’ll try it, thank you very much. It really big help of you ^.^
Perhaps I have solved the problem. Copy this mode from other folder (usr/local/grib_api/include/) to /usr/include.