An Open-Source, Pseudo-Spectral Convection Code for O(105) Cores

Abstract:

Spectral algorithms are a popular choice for modeling systems of turbulent, incompressible flow, due in part to their inherent numerical accuracy and also, as in the case of the sphere, geometrical considerations. These advantages must be weighed against the high cost of communication, however, as any time step taken by a spectral method will typically require multiple, global reorganizations (i.e. transposes) of the distributed flow fields and thermal variables. As more processors are employed in the solution of a particular problem, the total computation time decreases, but the number of inter-processor messages initiated increases. It is this property of spectral algorithms that ultimately limits their parallel scalability because, for any given problem size, there exists a sufficiently large process count such that the message initiation time overwhelms any gains in computation time. I will discuss the parallelization of a community-sourced spectral code that has been designed to mitigate this problem by minimizing the number of messages initiated within a single time step. The resulting algorithm possesses efficient strong scalability for problems both small (512³ grid points, 16,000 cores) and large (2048³ grid points, 130,000 cores). This code, named Rayleigh, has been designed with the study of planetary and stellar dynamos in mind, and can efficiently simulate anelastic MHD convection within both spherical and Cartesian geometries. Rayleigh is being developed through the Computational Infrastructure for Geodynamics (UC Davis), and will be made publicly available in winter of 2015.