Designing and Implementing a Single-Phase Row-Column-Parallel Sparse Matrix Vector Multiplication Algorithm based on 2D Matrix Partitioning

One-dimensional (1D) partitioning of sparse matrices results in lower quality partitioning than two-dimensional (2D)
partitionings in the context of parallel sparse matrix vector multiply (SpMxV) operations. However, 2D partitioning
schemes incur two communication phases. We propose a novel sparse matrix partitioning scheme which achieves a
single communication phase as in 1D schemes and partitions the nonzeros with a flexibility close to that in 2D
schemes. In the proposed partitioning scheme, a nonzero is assigned to either the receiver or the sender processor
associated with the related input- or output-vector entries. We observe that a fine-grain partitioning should satisfy this
constraint for most of the nonzeros. Based on this observation, we propose a simple yet effective heuristic to obtain
such a partition in two steps. In the first step, we obtain partitions on the rows, columns, and nonzeros of the given
matrix using some known methods. In the second step, we refine the nonzero partition such that the aforementioned
constraint is met while keeping the row and column partitions obtained in the first step intact. We demonstrate that
the proposed partitioning scheme improves the performance of parallel SpMxV operations both in theory and practice
with respect to 1D and 2D partitionings.