Proteus

PROTEUS is a multi-node GPU-native moving mesh hydrodynamics code.

It combines the mesh generation of [Ray et. al 2018] with a moving mesh hydro solver similar to "AREPO" [Springel 2010] and is optimized for unified memory architectures like NVIDIA's GH200 chips targeting exascale astrophysical applications.

PROTEUS supports 2D/3D static/moving mesh generation on CPU and GPU (both unified and discrete memory) and is parallelized using an OpenMP + MPI hybrid (GPU-aware).

Additionally to the pure hydro solver there is support for static potential gravity, radiative cooling, a stellar feedback subgrid model as well as a chaotic cold accretion AGN model with thermal and kinetic mode feedback.

Note

The current version is still under active development, so expect rapid changes. Proper documentation will follow eventually.

This project started during my master's thesis, supervised by Dylan Nelson, at the Institute of Theoretical Astrophysics, Heidelberg University.

---

Contents

• Examples
  • Convergence
  • Performance
  • Multi-node scaling
• Getting started
• Roadmap

---

Examples

A $50^2$ Kelvin-Helmholtz instability on static and moving meshes.

2D Riemann problem as in [Kurganov and Tadmor, 2002]

Volume rendering of AGN-feedback in an idealized Perseus-like cool-core cluster. Shown is the velocity magnitude with cool gas ($T < 5\times10^4 K$) overlaid in blue.

Convergence

Acoustic wave test showing second order convergence:

Sod shock tube test showing error relative to AREPO:

Gresho vortex test:

Performance

Per step time for a $454^3$ acoustic wave test on two GH200 chips:

Breakdown of where the per step time is spent:

Roofline plot for the major Kernels. Fast and slow-tier are part of the two-tier mesh generation.

Multi-node scaling

Weak scaling on JUPITER booster for an acoustic wave test with up to $3600^3$ and $1000$ GH200 GPUs retains 75% efficiency.

---

Getting started

Dependencies: HDF5, CUDA (for GPU mode)

1. After cloning the repo select your system in Makefile.systype (or add your own to the Makefile)
2. Configure compilation flags in a Config.sh and build with

make CONFIG=/path/to/Config.sh BUILD_DIR=/path/to/build/ EXEC=/path/to/ProteusGPU

3. Use a create.py script for IC generation and specify simulation parameters in a param.txt
4. Run the simulation with

./ProteusGPU [./ics/param.txt] [restart_flag]

or using MPI with mpirun. If the restart_flag is set, the simulation continues from the last snapshot in the output_folder.

---

Roadmap

• v0.1 - kNN and Voronoi mesh construction (2D and 3D, CPU)
• v0.2 - first order FV hydro (static mesh, 2D and 3D, CPU)
• v0.3 - second order FV hydro (static mesh, 2D and 3D, CPU)
• v0.4 - moving mesh hydro (2D, CPU)
• v0.5 - moving mesh hydro (3D, CPU)
• v0.6 - GPU initial port (moving mesh hydro)
• v0.7 - single GPU optimization
• v0.8 - multi-GPU (distributed memory, MPI)
• v0.9 - multi-GPU optimization
• v1.0 - support for inhomogenous particle distributions

additionally we plan to add more physics modules