In a previous post I introduced CDAT as a Python package for the manipulation of netCDF data, and made some comparisons with netcdf4. In my own opinion, implementations like CDAT, Iris and Xarray stick to the netCDF philosophy better than netcdf4, in that they permit meta-data preserving operations. Compared with Xarray, CDAT‘s array manipulation API is closer to native numpy.
In this post, and a number of posts to follow, I’d like to share some experiences using CDAT. I’ll start from installation, some basic file reading and saving, then move onto array manipulation, meta-data maintenance, etc., before covering some more advanced topics. So stay tuned if you are interested in learning this power tool.
CDAT is supported in Linux or MacOS. If you can setup a WSL ("Windows Subsystem for Linux"), it can be used in Windows 10 as well (although technically it is still running inside Linux). Of cause you can create a Linux virtual machine inside Windows to achieve the similar effects but at a greater cost of system resources. The CDAT team provides some very detailed instructions on how to setup a WSL for CDAT installation.
In this post I will only show the installation in Linux. The process in MacOS is the same. Aside from the setting-up of WSL or a virtual machine, the subsequent installation in Windows is also the same.
Install CDAT in Linux
Installation of CDAT has become way easier than before with the help of conda. When I first learned CDAT about 8 years ago, I had to compile all the dependencies myself, then configure and compile CDAT from source code.
This wiki page gives full instructions on the installation of Anaconda or Miniconda (a lighter-weight version of the former), and the installation of CDAT via conda. I’ll only show the steps to install the "lite" version of CDAT, which contains only the core data array manipulation modules, leaving out the virtualization-related modules. For plot creation we will be using the matplotlib+Cartopy combination.
Step 1: Install Anaconda
Go to the download page of Anaconda, download the installer for Linux (labeled as "64-Bit (x86) Installer" in the screenshot below):
Once finished, you will get a file with a name like
Anaconda3-2020.02-Linux-x86_64.sh. Navigate to the folder storing
this file in a terminal, then run:
This will launch the command-line installer.
Just follow the instructions given within. Specifically,
when asked about an installation location (see Figure 2 below),
choose the default (which is the
anaconda3 folder in your HOME
installation finished being printed out to the
terminal, it will prompt you to decide
Do you wish the installer to initialize Anaconda3 by running conda init? [yes|no]
Remember to close the terminal window and open a new one to finalize the anaconda installation.
Step 2: Create a new conda environment
A Python environment is an isolated virtual space in the system. Inside the environment one can install Python packages that are isolated from those in other environments, so that different versions of the same packages, or even different versions of Python, can co-exist in the same system. This way, one can create different environments, with different packages, for different workflows or tasks.
It is advised to always build up your Python working environment from an empty one, so that the chance of package conflicts can be minimized, and if anything goes wrong, it is easier to destroy the compromised environment and start afresh.
To create a new environment (or env for short) for CDAT installation:
conda create -n cdat python=3
-n cdat part specifies the name of the environment, you can
choose a different name if you like. In the remaining part of the
command you can specify what packages to install upon env creation. In this
case we only specify the Python version to be
3. You can also be more
specific about the version by using, for instance,
When finished, activate the environment:
conda activate cdat
[As a side note, in Windows, you will be using
instead to activate an env. Since I’m assuming you are doing
this in WSL or a Linux virtual machine, the activation command would be
After that the shell prompt will be prepended with the env name, like:
Another side note: if you use a specific environment frequently, it
may be worth adding the activation command to the
.bashrc file, so
that the env is activated in every new terminal session.
Step 3: Install packages
Make sure you are already in the newly created
environment. Then we’ll install the 2 core module of CDAT – cdms2 and
cdutil, plus a few extra numerical (scipy and pandas) and
plotting packages (matplotlib and cartopy) that one is almost
certainly gonna use sooner or later. You can add your own choices
to the list, or install them separately when in need.
The installation command is:
conda install -c conda-forge scipy pandas matplotlib cartopy cdms2 cdutil
Then validate the installation:
python -c "import cdms2, MV2, cdtime"
If nothing prints out, installation is done.