Skip to content
Merged
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,9 @@ for a complete example.

Published wheels are built without MPI support. For fire-atmosphere coupling
with MesoNH, or to tune the build for your CPU, build from source instead
(see [Build from source](#build-from-source)).
(see [Build from source](#build-from-source), and
[`docs/mnh/coupledMNHInstall.md`](docs/mnh/coupledMNHInstall.md) for the
coupled setup).

## Quick Start with Docker

Expand Down
235 changes: 117 additions & 118 deletions docs/mnh/coupledMNHInstall.md
Original file line number Diff line number Diff line change
@@ -1,164 +1,163 @@
# How to Install Coupled ForeFire with Meso NH; tested on OSX Ventura with Apple Silicon
# Running ForeFire coupled with Meso-NH

This guide details the steps to install coupled ForeFire with Meso NH on Apple Silicon Macs running OSX Ventura.
[Meso-NH](https://mesonh.cnrs.fr) is the French atmospheric research model
(CNRS / Météo-France). Coupled runs load ForeFire as a shared library: at
every step Meso-NH sends wind to the fire and receives heat and vapour fluxes
back. This guide sets the coupling up from scratch and shows how to check
that a change to ForeFire does not alter coupled results.

## Requirements
Tested on Ubuntu 24.04 (WSL) with GCC/gfortran 13.3, OpenMPI 4.1.6 and
Meso-NH 6.0.1. macOS notes, from an older Meso-NH 5.7 setup, are at the end.

Before proceeding, ensure the following packages are installed via Homebrew:
## 1. Prerequisites

- GCC (should be pre-installed)
- GFortran
- OpenMPI

Acquire MesoNH from the official depot at [MesoNH Depot](http://mesonh.aero.obs-mip.fr). If encountering Git issues, download the tarball from [MNH-V5-7-0.tar.gz](http://mesonh.aero.obs-mip.fr/mesonh/dir_open/dir_MESONH/MNH-V5-7-0.tar.gz).

**Important Note:** Utilize Bash instead of Ksh to avoid potential script compatibility issues.

## Configuration Steps

1. **Export Environment Variables**

Begin by setting the necessary environment variables in your terminal:

```bash
export ARCH=LXgfortran
export VER_MPI=MPIAUTO
export OPTLEVEL=O2
export MNH_FOREFIRE=1.0
```

2. **Run Configuration Script**

Navigate to the `src` directory within the MesoNH directory and execute the configuration script:

```bash
./configure
```

This should generate a configuration similar to:
```bash
sudo apt install build-essential gfortran openmpi-bin libopenmpi-dev \
cmake git git-lfs libnetcdf-dev libnetcdf-c++4-dev netcdf-bin
```

```
$SRC_MESONH/conf/profile_mesonh-LXgfortran-R8I4-MNH-V5-7-0-FF-MPIAUTO-O2
```
Use Bash, not Ksh, for everything below. Meso-NH's profile reads
`SRC_MESONH` before defining it, so do not run with `set -u`.

3. **Load the Configuration Profile**
## 2. Get and build Meso-NH

Load the generated configuration profile by running:
The 6.0.1 release is on [Zenodo](https://zenodo.org/records/21395048). The
[CNRS GitLab](https://src.koda.cnrs.fr/mesonh/mesonh-code) also works when it
is reachable (it needs Git LFS). Official instructions:
[download](https://mesonh.readthedocs.io/en/mnh-6-0/getting_started/download.html),
[compile](https://mesonh.readthedocs.io/en/mnh-6-0/getting_started/compile.html).

```bash
. $SRC_MESONH/conf/profile_mesonh-LXgfortran-R8I4-MNH-V5-7-0-FF-MPIAUTO-O2
```
```bash
mkdir -p ~/mesonh
tar xzf mnh601.tar.gz -C ~/mesonh
cd ~/mesonh/mesonh-code-PACK-MNH-V6-0-1/src
export ARCH=LXgfortran VER_MPI=MPIAUTO OPTLEVEL=O2
./configure
. ../conf/profile_mesonh-LXgfortran-R8I4-MNH-V6-0-1-MPIAUTO-O2
make -j 12
make installmaster
```

The build takes about 15 minutes on 12 cores. Load the profile in every new
shell before running a case: it sets `SRC_MESONH` and `XYZ`, which the test
scripts use.

## Installation and Configuration
## 3. Build ForeFire and link it

After setting up your environment variables, proceed with the compilation:
MPI is on by default in a plain CMake build. Check that the configure output
says MPI was found: without it the coupled run does not work. A warning that
the Git LFS check failed for files under `tests/runff` does not matter here;
the `mnh_*` files must be listed as OK.

```bash
make -j 4
git clone https://github.com/forefireAPI/forefire.git
cd forefire
git lfs pull
cmake -S . -B build
cmake --build build -j 4
```

Then installation :
This produces `lib/libforefireL.so`. Meso-NH opens
`$SRC_MESONH/exe/libForeFire.so` at run time, so link it there:

```bash
make installmaster
ln -sf "$PWD/lib/libforefireL.so" "$SRC_MESONH/exe/libForeFire.so"
```

This command initiates the installation process, and upon completion, you should have a working setup of Meso NH ready for further configuration and compilation of additional components, such as ForeFire.

### Compiling ForeFire and Linking with Meso NH

ForeFire must be compiled separately before being linked. Follow the cmake instruction for forefire, to link, follow the steps below after compiling ForeFire:

1. Ensure ForeFire is compiled with a shared library, on osX it should be located at `firefront/lib/libforefireL.dylib`.
2. Navigate to the Meso NH `exe/` directory and create a symbolic link to the ForeFire library:
## 4. Run a coupled case

```bash
ln -s PATHTOCOMPILEDFF/lib/libforefireL.dylib libForeFire.so
```
Two cases ship with ForeFire:

Replace `PATHTOCOMPILEDFF` with the actual path to your compiled ForeFire library. This step integrates ForeFire with Meso NH, enabling them to work together seamlessly.
| Case | What it is |
| --- | --- |
| `tests/mnh_real_nested` | Pedrógão Grande 2017, two nested domains, 4 MPI processes, 1200 s of simulation (about 7 min) |
| `tests/mnh_ideal` | Idealised relief, single domain, a few seconds; needs `ncgen` (from `netcdf-bin`) |

In the `exe/` directory, you should now have manies entries but most importantly :

- `MESONH-LXgfortran-R8I4-MNH-V5-7-0-FF-MPIAUTO-O2-> $SRC_MESONH/src/dir_obj-LXgfortran-R8I4-MNH-V5-7-0-FF-MPIAUTO-O2/MASTER/MESONH`

And one more, in the same directory for ForeFire: `libForeFire.so -> PATHTOCOMPILEDFF/lib/libforefireL.dylib`

### Running a Simulation

To run a simulation with ForeFire:

1. Navigate to the case directory, for example, the one for case 016_FOREFIRE.
2. Set the `MPIRUN` environment variable to configure the number of CPUs to use. For instance, to use 2 CPUs, execute:

```bash
export MPIRUN="mpirun -np 2"
```
3. Run the simulation as other KTESTS

## Fixups for OSX

During the installation and configuration of Meso NH and its components on OSX, you may encounter specific issues that require workarounds. Below are some solutions to common problems:
```bash
cd tests/mnh_real_nested
bash clean.bash
bash run.bash
```

### Fixing libxml2 Configuration Issue
`run.bash` only checks that `ForeFire/Outputs/ForeFire.0.nc` exists and is
larger than 15 kB. It does not compare against `ForeFire.0.nc.ref`, and the
grid size depends on the Meso-NH version (680×680 on 6.0.1, 666×666 reported
on 6.1). A meaningful check is the comparison in section 5.

When configuring NetCDF 4.9.2, if you encounter an issue related to libxml2, follow these steps to resolve it:
`mnh_real_nested/ForeFire/Init.ff` starts
`listenHTTP[host=127.0.0.1:8080]`, so port 8080 must be free.

1. Install autotools if you haven't already:
## 5. Check that a change does not alter coupled results

```bash
brew install autoconf automake libtool
```
Run the same case with ForeFire built from `dev` and from the branch under
review, and compare the outputs byte for byte.

2. Navigate to the NetCDF C library source directory:
1. Build both versions with the same options, each in its own worktree
(`git worktree add ../ff-pr <branch>`).
2. Make a clean copy of the case for each run.
3. Run `dev`, branch, `dev`, branch, switching the `libForeFire.so` link
before each run.
4. Compare with `cmp` the files in `ForeFire/Outputs`, `MODEL1`, `MODEL2` and
the `RUN12.*.FIRE.*.nc` files.

```bash
cd $SRC_MESONH/src/LIB/netcdf-c-4.9.2/
```
Two runs of the same build already differ in a few places, so the second
`dev` run is what separates noise from a real change:

3. Open `configure.ac` and locate the following line around line 1222:
- `RUN12.*.FIRE.*.nc`: 7 to 9 bytes, the timestamp in the `history`
attribute and the HDF5 header checksum.
- `MODEL*/output.*.air_density`: meaningless values (NaN, ±10¹⁰) that change
on every run. This comes from Meso-NH, not from ForeFire.

```plaintext
if test "x$ISOSX" = xyes && test "x$have_libxml2" = xno; then
```
Everything else must be identical. A file that differs between `dev` and the
branch, but not between the two `dev` runs, is a real difference.

4. Modify the line by removing the second `test` keyword, so it reads:
## macOS notes (Meso-NH 5.7)

```plaintext
if test "x$ISOSX" = xyes && "x$have_libxml2" = xno; then
```
These come from an earlier setup on Apple Silicon with macOS Ventura and
Meso-NH 5.7. They may no longer apply to 6.x.

5. Run `autoreconf` to regenerate the `configure` script from the modified `configure.ac`. This change should allow the configuration process to proceed without the libxml2 error.
Install `gcc`, `gfortran` and `open-mpi` with Homebrew. On 5.7, also
`export MNH_FOREFIRE=1.0` before `./configure`. The library is
`lib/libforefireL.dylib`; link it as `libForeFire.so` in `$SRC_MESONH/exe`
all the same.

### Fixing NetCDF C++ Configuration Hanging Issue
### libxml2 error when configuring NetCDF 4.9.2

If the configuration of NetCDF C++ hangs due to a version file issue, perform the following steps:
```bash
brew install autoconf automake libtool
cd $SRC_MESONH/src/LIB/netcdf-c-4.9.2/
```

1. In the `$SRC_MESONH/src/LIB/netcdf-cxx4-4.3.1` directory, you will find a `VERSION` file that incorrectly contains a version number (e.g., "-n 4.4.3"). This file needs to be empty.
In `configure.ac`, around line 1222, change

2. You will need to use two terminals for this fix. In one terminal, navigate to the `$SRC_MESONH/src/LIB/netcdf-cxx4-4.3.1` directory. In the other terminal, run the Meso NH make process.
```plaintext
if test "x$ISOSX" = xyes && test "x$have_libxml2" = xno; then
```

3. While the make process is running, frequently clear the `VERSION` file in the first terminal by running:
to

```bash
cat VERSION; echo "" > VERSION
```
```plaintext
if test "x$ISOSX" = xyes && "x$have_libxml2" = xno; then
```

This needs to be done quickly between the creation and usage of the `VERSION` file during the make process. Pay attention to the compilation stages: first HDF5 (which takes a few minutes), then a quick NetCDF 9 configuration, followed by the NetCDF C++ configuration where you need to apply the fix.
then run `autoreconf`.

### Fixing MPI Run Infinite Loop Issue
### NetCDF C++ configuration hangs

On OSX, you might encounter an infinite loop issue with `mpirun` due to case sensitivity in paths. To resolve this:
`$SRC_MESONH/src/LIB/netcdf-cxx4-4.3.1/VERSION` gets a bad version string
(e.g. `-n 4.4.3`) and must be empty. While `make` runs in one terminal, empty
it from another as soon as it appears, after HDF5 and the NetCDF C
configuration:

1. Navigate to the Meso NH bin directory:
```bash
cat VERSION; echo "" > VERSION
```

```bash
cd $SRC_MESONH/bin
```
### `mpirun` loops forever

2. Rename `Mpirun` to avoid conflicts with the case-insensitive filesystem of OSX:
The case-insensitive filesystem confuses `Mpirun` with `mpirun`:

```bash
mv Mpirun NoMpirun
```
```bash
cd $SRC_MESONH/bin
mv Mpirun NoMpirun
```
6 changes: 3 additions & 3 deletions scripts/check_all_lfs.bash
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,16 @@ BASE_URL="https://github.com/forefireAPI/forefire/raw/refs/heads/master"
# File list: md5:path
FILES_MD5_LIST="
a4e76c91af4b72d2770759fce508b640:tests/mnh_ideal/ForeFire/ForeFire.0.nc.ref
afcd2476e600d4d7580bca442b961191:tests/mnh_ideal/RELIEF3D.des
6ab0615f365868ede4cb74a9234ed229:tests/mnh_ideal/RELIEF3D.des
53f1165f78ad3a697e8d12eee8af2b44:tests/mnh_ideal/RELIEF3D.lfi
4e98a1093da9167e545df07a6693582c:tests/mnh_ideal/RELIEF3DX.lfi
92d4a0008e01e2cca58468bd8aa0da37:tests/mnh_real_nested/ForeFire/ForeFire.0.nc.ref
98d167f873fc70a238c2d33c1da9a114:tests/mnh_real_nested/ForeFire/data.nc
a397d61ad1dd4f73a69e002d6e452541:tests/mnh_real_nested/M2.20170617.00.15.des
5d9c55102c51621f0178a101d5238953:tests/mnh_real_nested/M2.20170617.00.15.des
4df6c56bc7012d41da26d26add903095:tests/mnh_real_nested/M2.20170617.00.15.nc
da97d0c3dd63f6f6d2d941524c526e75:tests/mnh_real_nested/PGD_D3200mA.nested.nc
b64f537ae25fad1d002085cc48a6e5d4:tests/mnh_real_nested/PGD_D800mA.nested.nc
dbe0a29d7f94fc8cbe55470fecf27cc4:tests/mnh_real_nested/RUN12.1.PRUN1.004.des
a62bcbef1eb263735f21d0262e2a0597:tests/mnh_real_nested/RUN12.1.PRUN1.004.des
cd6cf1c61b5433b425728b0c10a6870f:tests/mnh_real_nested/RUN12.1.PRUN1.004.nc
fe3a89db4253de28e9c42cea22ec91cc:tests/python/360wind.png.ref
badddd3b68c3bf0c011700bf0d3b9880:tests/runff/ForeFire.0.nc.ref
Expand Down
8 changes: 5 additions & 3 deletions tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use-cases of ForeFire.
| Directory | What it covers | Needs |
| --- | --- | --- |
| `unit` | C++ tests of one model at a time, without running a simulation, covering every registered propagation and flux model | Nothing beyond the build; see `unit/README.md` |
| `mnh_ideal` | ForeFire / **Meso-NH** coupling on an idealised atmospheric case | `SRC_MESONH` set; Meso-NH compiled with the ForeFire library in its `exe` directory |
| `mnh_ideal` | ForeFire / **Meso-NH** coupling on an idealised atmospheric case | Meso-NH profile loaded; Meso-NH compiled with the ForeFire library in its `exe` directory |
| `mnh_real_nested` | ForeFire / **Meso-NH** coupling on a real nested case | Same as above |
| `python` | The Python bindings, through two example simulations | `PYTHONEXE` set to a Python interpreter that can `import pyforefire` |
| `runANN` | The built-in feed-forward network evaluator, on a network fitted to Rothermel | `bin/ANN_test`, built by default (`-DFOREFIRE_BUILD_TOOLS=ON`) |
Expand All @@ -30,10 +30,12 @@ ctest --test-dir build --output-on-failure
* The test fixtures are stored in **Git LFS**. `git lfs pull` if the `.nc`
files look like short text stubs.

* *(optional)* **Meso-NH**, for the `mnh_*` tests only:
* *(optional)* **Meso-NH**, for the `mnh_*` tests only. Setup and the `dev` × PR
check: [`docs/mnh/coupledMNHInstall.md`](../docs/mnh/coupledMNHInstall.md).

```bash
export SRC_MESONH=/path/to/your/mesonh
# sets SRC_MESONH and XYZ, which the mnh_* scripts use
. /path/to/mesonh/conf/profile_mesonh-LXgfortran-R8I4-MNH-V6-0-1-MPIAUTO-O2
```

* *(optional)* **A Python interpreter with the bindings available**, for the
Expand Down
2 changes: 1 addition & 1 deletion tests/mnh_ideal/EXSEG1.nam
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
&NAM_CONFIO LCDF4=T, LLFIOUT=T, LLFIREAD=F /
&NAM_CONFIO /
&NAM_CONFZ
! NZ_VERB = 5 , NZ_PROC = 0 , NB_PROCIO_R = 8 , NB_PROCIO_W = 8
/
Expand Down
4 changes: 2 additions & 2 deletions tests/mnh_ideal/RELIEF3D.des
Git LFS file not shown
1 change: 0 additions & 1 deletion tests/mnh_ideal/run.bash
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
. $HOME/runMNH
export MPIRUN="mpiexec -np 4"
ncgen -o ForeFire/lpsimple.nc ForeFire/lpsimple.cdl
mkdir ForeFire/Outputs
Expand Down
4 changes: 2 additions & 2 deletions tests/mnh_real_nested/M2.20170617.00.15.des
Git LFS file not shown
4 changes: 2 additions & 2 deletions tests/mnh_real_nested/RUN12.1.PRUN1.004.des
Git LFS file not shown
Loading