A new parallel PM3 code for very large-scale cosmological simulations

By T. MacFarland, Hugh M. P. Couchman, Frazer R. Pearce, Jakob Pichlmeier. 1998

Abstract We have developed a parallel Particle–Particle, Particle–Mesh ( P 3 M ) simulation code for the Cray T3E parallel supercomputer that is well suited to studying the time evolution of systems of particles interacting via gravity and gas forces in cosmological contexts. The parallel code is based upon the public-domain serial Adaptive P 3 M -SPH (http://coho.astro.uwo.ca/pub/hydra/hydra.html) code of Couchman et al. (1995)[ApJ, 452, 797]. The algorithm resolves gravitational forces into a long-range component computed by discretizing the mass distribution and solving Poisson's equation on a grid using an FFT convolution method, and a short-range component computed by direct force summation for sufficiently close particle pairs. The code consists primarily of a particle–particle computation parallelized by domain decomposition over blocks of neighbour-cells, a more regular mesh calculation distributed in planes along one dimension, and several transformations between the two distributions. The load balancing of the P 3 M code is static, since this greatly aids the ongoing implementation of parallel adaptive refinements of the particle and mesh systems. Great care was taken throughout to make optimal use of the available memory, so that a version of the current implementation has been used to simulate systems of up to 109 particles with a 10243 mesh for the long-range force computation. These are the largest Cosmological N-body simulations of which we are aware. We discuss these memory optimizations as well as those motivated by computational performance. Performance results are very encouraging, and, even without refinements, the code has been used effectively for simulations in which the particle distribution becomes highly clustered as well as for other non-uniform systems of astrophysical interest.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Stable clustering, the halo model and non-linear cosmological power spectra (2002, methodology): “In HYDRA, the transition from pure Newtonian to constant force is achieved using a ‘spline-kernel softening’; with this method, the interparticle forces become precisely Newtonian after 2.34 times the softening length.”
Galaxy Clusters in Hubble Volume Simulations: Cosmological Constraints from Sky Survey Populations (2001, methodology): “Processor time for these computations was minimized by employing a parallel algorithm well matched to the machine architecture (MacFarland et al. 1998) and by simulating large volumes that entail a minimum of message passing overhead.”
Shapes and Alignments of Galaxy Cluster Halos (2004, methodology): “Simulations The HV simulations are a pair of gigaparticle N-body simulations created using a parallel version of the HydraN-body code (Macfarland et al. 1998).”
The subhalo populations of ΛCDM dark haloes (2004, methodology): “First, we ran the simulation from high redshift until z = 2.2 with the parallel SHMEM version of HYDRA (Couchman, Thomas & Pearce 1995; Macfarland et al. 1998).”
Chemical enrichment of the intracluster and intergalactic medium in a hierarchical galaxy formation model (2003, methodology): “The simulation was performed using a parallel P3M code (Macfarland et al. 1998) and followed 5123 particles with a particle mass of 7 × 1010 h−1 M in a comoving box of size 479 h−1 Mpc on a side.”