Skip to main content

Cornelis Technical Documentation

6.5.5. MPI Message Rate

Message rate is measured using the same Uniband and Biband benchmarks, but with as many ranks per node as physical cores available. The message rate refers to the total number of small (typically 8-byte) MPI messages exchanged per second between two nodes.

6.5.5.1. Example: 256 Ranks Across Two Nodes (128 per Node)

mpirun -np 256 -ppn 128 -host hostA,hostB\
  -genv I_MPI_PIN_PROCESSOR_LIST=0-127 \
  -genv FI_PROVIDER=opx \
  -genv I_MPI_FABRICS=shm:ofi \
  IMB-MPI1 Uniband Biband -npmin 256

Sample output:

#---------------------------------------------------
# Benchmarking Uniband
# #processes = 64
#---------------------------------------------------
#bytes #repetitions Mbytes/sec Msg/sec
...

  • Msg/sec (last column) reflects the message rate for 8-byte transfers.

  • To measure bidirectional message rate, use the Biband benchmark.

Note

Sequential process pinning is required to measure the best MPI message rate. For MPIs other than Intel MPI, the hydra process manager provides methods for this, including options such as -bind-to user:0,1,2,... Refer to mpiexec -bind-to -help for details.