Skip to main content

Cornelis Technical Documentation

6.5.8. Running MPI Benchmarks with Open MPI

To run MPI benchmarks using Open MPI, ensure that the ofi MTL and the OPX Provider are specified. The following is an example of running the IMB Uniband benchmark across two nodes using one MPI rank per node:

mpirun -np 2 --map-by ppr:1:node -host hostA:1,hostB:1 \
  -mca mtl ofi -x FI_PROVIDER=opx \
  -mca btl self,vader \
  taskset -c <HFI_local_core> IMB-MPI1 Uniband

  • --map-by ppr:1:node ensures one MPI rank is launched per node.

  • FI_PROVIDER=opx selects the OPX Provider.

  • taskset -c <HFI_local_core> pins the benchmark process to a core local to the SuperNIC NUMA node for optimal latency.

    Note

    Other Open MPI specific methods exist to pin to cores such as --map-by pe-list=0,1,2,3:ordered to pin to core 0,1,2,3 sequentially. This is similar to I_MPI_PIN_PROCESSOR_LIST=0,1,2,3.