Current Bottlenecks in the Scalability of OpenFOAM on Massively Parallel Clusters

The scaling behavior of different OpenFOAM versions is analyzed on two benchmark
problems. Results show that the applications scale reasonably well up to a thousand tasks.
An in-depth profiling identifies the calls to the MPI_Allreduce function in the linear algebra
core libraries as the main communication bottleneck. A sub-optimal performance on-core is
due to the sparse matrices storage format that does not employ any cache-blocking
mechanism at present. Possible strategies to overcome these limitations are proposed and
analyzed, and preliminary results on prototype implementations are presented.