4.2.2. Bandwidth and Message Rate
When evaluating networking products, one of the first specifications to consider is bandwidth—the rate at which data can be transmitted or received, excluding link overhead such as error-checking information and clocking bits. This value is typically reported as the maximum bandwidth and is expressed in gigabits per second (Gbps).
For CN5000 Omni-Path, the maximum bandwidth is 400 Gbps. Because CN5000 uses a full-duplex link, there is no need to distinguish between transmit and receive bandwidth. A full-duplex connection can send and receive data simultaneously at the same rate, providing 400 Gbps of bandwidth in each direction.
It is important to note that 400 Gbps represents the maximum theoretical bandwidth of the link. The effective bandwidth achieved in practice depends on factors such as message size. Larger messages generally make more efficient use of the available bandwidth because a smaller percentage of the transmission is consumed by protocol overhead. The maximum transmission unit (MTU) defines the largest message that can be transmitted in a single packet, and the MTU for CN5000 is 10,240 bytes.
Data transfers larger than the MTU must be divided into multiple packets before transmission. At the maximum MTU, link efficiency approaches its highest level because protocol overhead represents only a small fraction of each packet. As packet sizes decrease, overhead consumes a larger percentage of the transmission, reducing overall bandwidth efficiency. Although smaller messages may reduce bandwidth utilization, they can increase the number of messages that can be processed per second, depending on the workload and communication pattern.