WRF

A. Description

The Weather Research and Forecasting (WRF) model is a numerical weather prediction (NWP) and atmospheric simulation system designed for both research and operational applications.  WRF is supported as a common tool for the university/research and operational communities to promote closer ties between them and to address the needs of both.

The development of WRF has been a multi-agency effort to build a next-generation mesoscale forecast model and data assimilation system to advance the understanding and prediction of mesoscale weather and accelerate the transfer of research advances into operations.

The WRF effort has been a collaborative one among the National Center for Atmospheric Research's (NCAR) Mesoscale and Microscale Meteorology (MMM) Division, the National Oceanic and Atmospheric Administration's (NOAA) National Centers for Environmental Prediction (NCEP) and Earth System Research Laboratory (ESRL), the Department of Defense's Air Force Weather Agency (AFWA) and Naval Research Laboratory (NRL), the Center for Analysis and Prediction of Storms (CAPS) at the University of Oklahoma, and the Federal Aviation Administration (FAA), with the participation of university scientists.

WRF source code, examples, and documentation can be found at http://wrf-model.org

B. How to download WRF

From the main WRF page, select "User Resources" and follow the link below "WRF Releases".  From the "Download" link at the top of the page, you will be able to download the most recent versions of WRF (the main weather forecasting model code) and WPS (the WRF pre-processing system), as well as other related software and previous versions.

If you are a new WRF user, and have never downloaded WRF before, you will be asked to answer a few questions in order to register as a WRF user and create an account for future downloads.  If you have already registered as a WRF user, you will be asked to enter your account information.

The installation files are in standard gzipped tar file format; place the tarball in your home or projects directory on Blue Waters, and type the following (WRF version 3.7 is used here as an example; other versions of WRF, WPS, etc... can be downloaded and extracted in a similar manner):

cd ~

tar -xzf WRF3.7.TAR.GZ

This will create a sub-directory called "WRFV3", from which you can configure and install WRF.

C. How to build WRF

On Blue Waters, WRF has been built with the Cray, PGI, and Intel Programming Environments. It should be possible to build with the GNU Programming Environment, but this has not been tested.

The following examples assume you are using the standard CShell; if you prefer to use a different shell, then some syntax modifications will be necessary.

C.1 Building WRF with the Cray environment

- Configure programming environment.  The "PrgEnv-cray" Progamming Environment module is loaded by default on Blue Waters; if you have changed to a different Programming Environment, e.g., PGI, use the "module swap" command to return to the Cray Programming Environment, for example:

tcsh

module swap PrgEnv-pgi PrgEnv-cray

Then load the NetCDF module and set a few environment variables (the prefix "cray-" on the "netcdf" module name has nothing to do with which compiler is being used):

module load cray-netcdf

setenv NETCDF ${NETCDF_DIR}

setenv WRFIO_NCD_LARGE_FILE_SUPPORT 1

- Configure and build WRF.  There are a few things that need to be manually changed before building WRF on Blue Waters.  When you run the configure script, you will be presented with several choices based on the compiler and type of parallelization desired - choose one of the Cray XE and Cray compiler options.

source ./configure --prefix=<...path_to_directory_to_build_WRF_in...>

After running the configure script, a file called "configure.wrf" will be generated.  It may be necessary to edit this file, and manually change the names of the Fortran and C compilers to "ftn" and "cc", respectively.  We have found that leaving the serial and regular C compilers as "gcc" seems to work on Blue Waters, as in the following snippet from "configure.wrf":

...

SFC=ftn

SCC=gcc

CCOMP=gcc

DM_FC=ftn

DM_CC=cc -DMPI2_SUPPORT -DMPI2_THREAD_SUPPORT

...

Many, if not all, WRF distributions have Makefiles with a "-w" flag on some of the rules for C compiling; this flag causes unwanted behavior with the Cray compilers.  In version 3.7 of WRF, for example, the following lines of the "makefile" in directory "WRFV3/external/io_int" need to be modified in order to remove this flag:

CHANGE

io_int_idx.o: io_int_idx.c io_int_idx.h io_int_idx_tags.h

       $(CC) -o $@ -c -w $*.c

TO

io_int_idx.o: io_int_idx.c io_int_idx.h io_int_idx_tags.h

       $(CC) -o $@ -c $*.c

Different versions of WRF may have other instances of this "-w" flag in various makefiles that need to be removed in a similar fashion.  Now we are ready to begin the build process - it can take several hours to complete, and there may be long periods with no output generated, even though the build is continuing.

cd ~/WRV3

source ./clean

source ./compile -j 8 wrf

- Executable file "wrf.exe" will be generated in the "WRFV3/run" directory.

C.2 Building WRF with the PGI environment

- Configure programming environment (the prefix "cray-" on the "netcdf" module name has nothing to do with which compiler is being used):

module swap PrgEnv-cray PrgEnv-pgi

module load cray-netcdf

setenv NETCDF ${NETCDF_DIR}

setenv WRFIO_NCD_LARGE_FILE_SUPPORT 1

- Configure and build WRF.  There are a few things that need to be manually changed before building WRF on Blue Waters.  When you run the configure script, you will be presented with several choices based on the compiler and type of parallelization desired - choose one of the Cray XE and PGI compiler options.

source ./configure --prefix=<...path_to_directory_to_build_WRF_in...>

After running the configure script, a file called "configure.wrf" will be generated.  It may be necessary to edit this file, and manually change the names of the Fortran and C compilers to "ftn" and "cc" (don't use "pgcc"), respectively, as in the following snippet from "configure.wrf":

...

SFC=ftn

SCC=gcc

CCOMP=cc

DM_FC=ftn

DM_CC=cc -I$(MPICH_DIR) -DMPI2_SUPPORT

...

Now we are ready to begin the build process - it can take several hours to complete, and there may be long periods with no output generated, even though the build is continuing.

cd ~/WRFV3

source ./clean

source ./compile -j 8 wrf

- Executable file "wrf.exe" will be generated in the "WRFV3/run" directory

C.3 Building WRF with the Intel environment

- Configure programming environment (the prefix "cray-" on the "netcdf" module name has nothing to do with which compiler is being used):

module swap PrgEnv-cray PrgEnv-intel

module load cray-netcdf

setenv NETCDF ${NETCDF_DIR}

setenv WRFIO_NCD_LARGE_FILE_SUPPORT 1

- Configure and build WRF.  There are a few things that need to be manually changed before building WRF on Blue Waters.  When you run the configure script, you will be presented with several choices based on the compiler and type of parallelization desired - choose one of the Cray XE and Intel compiler options.

source ./configure --prefix=<...path_to_directory_to_build_WRF_in...>

After running the configure script, a file called "configure.wrf" will be generated.  It may be necessary to edit this file, and manually change the names of the Fortran and C compilers to "ftn" and "cc" (don't use "icc"), respectively, as in the following snippet from "configure.wrf":

...

SFC=ftn

SCC=cc

CCOMP=cc

DM_FC=ftn

DM_CC=cc -DMPI2_SUPPORT -DMPI2_THREAD_SUPPORT

...

Now we are ready to begin the build process - it can take several hours to complete, and there may be long periods with no output generated, even though the build is continuing.

cd ~/WRFV3

source ./clean

source ./compile -j 8 wrf

- Executable file "wrf.exe" will be generated in the "WRFV3/run" directory