Skip to content

Commit

Permalink
ompi/comm: improve comm_split_type scalability
Browse files Browse the repository at this point in the history
This commit introduces a new algorithm for MPI_Comm_split_type. The
old algorithm performed an allgather on the communicator to decide
which processes were part of the new communicators. This does not
scale well in either time or memory.

The new algorithm performs a couple of all reductions to determine the
global parameters of the MPI_Comm_split_type call. If any rank gives
an inconsistent split_type (as defined by the standard) an error is
returned without proceeding further. The algorithm then creates a
communicator with all the ranks that match the split_type (no
communication required) in the same order as the original
communicator. It then does an allgather on the new communicator (which
should be much smaller) to determine 1) if the new communicator is in
the correct order, and 2) if any ranks in the new communicator
supplied MPI_UNDEFINED as the split_type. If either of these
conditions are detected the new communicator is split using
ompi_comm_split and the intermediate communicator is freed.

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
(cherry picked from commit 4c49c42)
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
  • Loading branch information
hjelmn committed Oct 12, 2016
1 parent d52a2d0 commit 91337bf
Showing 1 changed file with 183 additions and 158 deletions.
Loading

0 comments on commit 91337bf

Please sign in to comment.