OpenMPI and OpenSHMEM installation with UCX

UCX installation

Requirements: Autoconf 2.63 and above.

1. Get latest version of the UCX code

$ git clone https://github.com/openucx/ucx.git ucx
$ cd ucx

2. Run autogen:

$ ./autogen.sh

3. This step is only required for OpenPOWER platforms - Power 8 On Ubuntu platform the config.guess file is a bit outdated and does not have support for power. In order to resolve the issue you have to download an updated config.guess. From the root of the project:

$ wget https://github.com/shamisp/ucx/raw/topic/power8-config/config.guess

4. Configure:

$ mkdir build
$ cd build
$ ../configure --prefix=/your_install_path

Note: For best performance configuration, use ../contrib/configure-release.
This will strip all debugging and profiling code.

5. Build and install:

$ make
$ make install

7. Running unit tests (using google test).
   This only work if gtest was installed and detected on your platform, and --enable-gtest was passed to configure:

$ make -C test/gtest test

OpenMPI and OpenSHMEM installation

1. Get latest-and-gratest OpenMPI version:

$ git clone https://github.com/open-mpi/ompi.git

2. Autogen:

$ cd ompi
$ ./autogen.pl

3. Configure with UCX:

$ mkdir build
$ cd build
../configure --prefix=/your_install_path/ --with-ucx=/path_to_ucx_installation

4. Build:

$ make
$ make install

Running Open MPI with UCX

Example of the command line (with optional flag to select IB device mlx5_0 port 1):

$ mpirun -np 2 -mca pml ucx -x UCX_NET_DEVICES=mlx5_0:1 ./app

Open MPI runtime optimizations for UCX

• By default OpenMPI enables build-in transports (BTLs), which may result in additional software overheads in the OpenMPI progress function. In order to workaround this issue you may try to disable certain BTLs.

$ mpirun -np 2 -mca pml ucx --mca btl ^vader,tcp,openib -x UCX_NET_DEVICES=mlx5_0:1 -x UCX_TLS=rc,sm ./app

• OpenMPI version https://github.com/open-mpi/ompi/commit/066370202dcad8e302f2baf8921e9efd0f1f7dfc leverages more efficient timer mechanism and there fore reduces software overheads in OpenMPI progress

MPI and OpenSHMEM release versions tested with UCX master.

1. UCX current tarball: https://github.com/openucx/ucx/archive/master.zip

2. The table of MPI and OpenSHMEM distributions that are tested with the HEAD of UCX master

MPI/OpenSHMEM project
OpenMPI/OSHMEM	3.0.0 and above
OpenMPI	2.1.0 and above
MPICH	Latest