A Smoothed Particle Hydrodynamics Mini-App for Exascale (original) (raw)
Related papers
Towards a Mini-App for Smoothed Particle Hydrodynamics at Exascale
2018
The smoothed particle hydrodynamics (SPH) technique is a purely Lagrangian method, used in numerical simulations of fluids in astrophysics and computational fluid dynamics, among many other fields. SPH simulations with detailed physics represent computationallydemanding calculations. The parallelization of SPH codes is not trivial due to the absence of a structured grid. Additionally, the performance of the SPH codes can be, in general, adversely impacted by several factors, such as multiple time-stepping, long-range interactions, and/or boundary conditions. This work presents insights into the current performance and functionalities of three SPH codes: SPHYNX, ChaNGa, and SPHflow. These codes are the starting point of an interdisciplinary co-design project, SPH-EXA, for the development of an Exascale-ready SPH miniapp. To gain such insights, a rotating square patch test was implemented as a common test simulation for the three SPH codes and analyzed on two modern HPC systems. Furthermore, to stress the differences with the codes stemming from the astrophysics community (SPHYNX and ChaNGa), an additional test case, the Evrard collapse, has also been carried out. This work extrapolates the common basic SPH features in the three codes for the purpose of consolidating them into a pure-SPH, Exascaleready, optimized, mini-app. Moreover, the outcome of this serves as direct feedback to the parent codes, to improve their performance and overall scalability.
2019
Core-collapse supernovae (CCSNe) are integral to the formation and distribution of heavy elements across the universe. However, CCSNe are highly complex and inherently non-linear phenomena. Large-scale simulations of these cosmic events can provide us a glimpse of their hydrodynamic and nucleosynthetic processes which are difficult to observe. To enable these massive numerical simulations on high-performance computing (HPC) centers, this study uses FleCSPH, a parallel and distributed code, based on the smooth-particle hydrodynamics (SPH) formulation. In the recent years, the HPC architecture has evolved and the next generation of exascale computers are expected to feature heterogenous architecture. Therefore, it is important to maintain code portability across platforms. This work demonstrates code portability of FleCSPH through the incorporation of Kokkos C++ library and containers using Charliecloud.
Optimization strategies for CPU and GPU implementations of a smoothed particle hydrodynamics method
Computer Physics Communications, 2013
Much of the current focus in high performance computing (HPC) for computational fluid dynamics (CFD) deals with grid based methods. However, parallel implementations for new meshfree particle methods such as Smoothed Particle Hydrodynamics (SPH) are less studied. In this work, we present optimizations for both central processing units (CPU) and graphics processing units (GPU) focused on a Lagrangian Smoothed Particle Hydrodynamics (SPH) method. In particular, the obtained performance and a comparison between the most efficient implementations for CPU and GPU are shown using the DualSPHysics code.
Eulerian incompressible smoothed particle hydrodynamics on multiple GPUs
Computer Physics Communications, 2021
Recent advances in the development of Eulerian incompressible smoothed particle hydrodynamics (EISPH), such as highorder convergence and natural coupling with Lagrangian formulations, demonstrate its potential as a meshless alternative to traditional computational fluid dynamics (CFD) methods. This work aims to address one of the major outstanding limitations of EISPH, its relatively high computational cost, by providing an implementation that can be deployed on multiple graphics processing units (GPUs). To this end, a pre-existing multi-GPU version of the open-source Lagrangian weakly-compressible code DualSPHysics is converted to an EISPH formulation and integrated with an open-source multi-GPU multigrid solver (AmgX) to treat the pressure Poisson equation implicitly. The integration of AmgX within DualSPHysics presents a significant challenge, since AmgX is designed for distributed systems and therefore conflicts with the single-node shared memory design of the multi-GPU DualSPHysics code. The present implementation is validated against well-known test cases, showing excellent agreement with benchmark solutions and demonstrating second-order convergence. A detailed profiling and performance testing is also presented to investigate memory consumption and scaling characteristics. The results show approximately 87 %-95 % strong scaling efficiency and 92 %-94 % weak scaling efficiency in both 2D and 3D on up to four GPUs. Large spikes in memory consumption during the initialisation of the linear solver library are found to impede full utilisation of the device memory. Nevertheless, the present implementation is shown to permit problem sizes on the order of 69.3 million (2D) and 20.8 million (3D) particles on four GPUs (128 GB total device memory), which is beyond what has previously been reported for incompressible SPH on GPUs, where the PPE is treated implicitly, and demonstrates its potential as an alternative to traditional CFD methods.
High Performance Computing in …, 2000
Smoothed Particle Hydrodynamics (SPH) is a particle method to simulate compressible uids. First we present a brief introduction to SPH, where we focus on an approach to treat the physical viscosity. Then we describe in detail the basic principles of our parallel implementation of the SPH method. The e ciency of the code on Cray T3E and IBM SP2 is discussed. In the last part we present a short introduction to accretion disks in interacting binary stars and report on some new results in that eld achieved with our code.
Particle Mesh Hydrodynamics for Astrophysics Simulations
2007
We present a particle method for the simulation of three dimensional compressible hydrodynamics based on a hybrid Particle-Mesh discretization of the governing equations. The method is rooted on the regularization of particle locations as in remeshed Smoothed Particle Hydrodynamics (rSPH). The rSPH method was recently introduced to remedy problems associated with the distortion of computational elements in SPH, by periodically re-initializing the particle positions and by using high order interpolation kernels.
New multi-GPU implementation for smoothed particle hydrodynamics on heterogeneous clusters
Computer Physics Communications, 2013
A massively parallel SPH scheme using heterogeneous clusters of Central Processing Units (CPUs) and Graphics Processing Units (GPUs) has been developed. The new implementation originates from the single-GPU DualSPHysics code previously demonstrated to be powerful, stable and accurate. A combination of different parallel programming languages is combined to exploit not only one device (CPU or GPU) but also the combination of different machines. Communication among devices uses an improved Message Passing Interface (MPI) implementation which addresses some of the well-known drawbacks of MPI such as including a dynamic load balancing and overlapping data communications and computation tasks. The efficiency and scalability (strong and weak scaling) obtained with the new DualSPHysics code are analysed for different numbers of particles and different number of GPUs. Last, an application with more than 10^9 particles is presented to show the capability of the code to handle simulations tha...
Monthly Notices of the Royal Astronomical Society, 1996
A combined N{body/hydrodynamical code is presented. Hydrodynamical properties are determined using smoothed particle hydrodynamics (Sph). The gravitational interaction of gas and collisionless particles is treated in a direct summation approach which bene ts from the high speed of the special purpose hardware Grape (GRAvity PipE). Besides gravitational forces, Grape also returns the list of neighbours and can, therefore, be used to speed up the hydrodynamical part, too. After the interaction list has been passed, density, pressure forces, propagation and interpolation of particles etc. are calculated on the front end, a 50 MHz SUN SPARC 10. In order to combine Sph and Grape, possible limitations due to the hardware design of Grape are carefully analyzed and modi cations compared to current Sph codes are discussed. The resulting code, Grapesph is similarly exible as Treesph. 50-55% of the CPU time is spent to calculate the densities and the pressure forces on the front end, 15-20% to calculate gravity, and about 10% in miscellaneous subroutines. Another 20% are required by communication, mainly to read out the neighbour list via the VME interface. The main shortcoming is the in exible, hardwired force law (a Plummer law), which makes it di cult to include periodic boundary conditions. Also because of the limited dynamic range, Grapesph seems to be less suitable to perform large scale structure simulations, where the resolution should be high everywhere in the simulation volume. The resulting code seems, however, especially well suited to investigate the formation of individual objects in a large scale structure environment, like e.g., galaxies or clusters. Such simulations require a very high spatial resolution, but only within a relatively small subvolume. By means of a multiple time step scheme, time step constraints due to local stability criteria can almost be avoided. The total performance is at least half as good as Treesph on a CRAY and for most applications it seems to be even better. The CPU time per time step is only slightly dependent on the clustering state. Grapesph, therefore, provides a very attractive alternative to the use of supercomputers in cosmology.
Computer Physics Communications, 2009
We describe the algorithms implemented in the first version of GRADSPH, a parallel, tree-based, smoothed particle hydrodynamics code for simulating self-gravitating astrophysical systems written in FORTRAN 90. The paper presents details on the implementation of the Smoothed Particle Hydro (SPH) description, where a gridless approach is used to model compressible gas dynamics. This is done in the conventional SPH way by means of 'particles' which sample fluid properties, exploiting interpolating kernels. The equations of self-gravitating hydrodynamics in the SPH framework are derived self-consistently from a Lagrangian and account for variable smoothing lengths ('GRAD-h') terms in both the hydrodynamic and gravitational acceleration equations. A Barnes-Hut tree is used for treating selfgravity and updating the neighbour list of the particles. In addition, the code updates particle properties on their own individual timesteps and uses a basic parallelisation strategy to speed up calculations on a parallel computer system with distributed memory architecture. Extensive tests of the code in one and three dimensions are presented. Finally, we describe the program organisation of the publicly available 3D version of the code, as well as details concerning the structure of the input and output files and the execution of the program.