Switching latency is one of the most important—and most commonly discussed—forms of network latency. It is defined as the time required for a Switch to receive a message, determine its destination, and begin transmitting it through the appropriate output port. This delay is caused by several operations, including reading the packet header, performing routing and forwarding decisions, and moving the packet through the switch's internal data path.

CN5000 Switches provide extremely low switching latency—less than 100 nanoseconds (ns)—which helps minimize the impact of this delay on overall application performance. While switching latency cannot be eliminated entirely, it can become a significant contributor to end-to-end communication time, particularly as messages traverse multiple switches.

To simplify discussions of switching latency and topology design, network paths are often described in terms of hops. A hop occurs each time a message passes through a switch. Examples include:

Because each hop introduces additional switching latency, hop count provides a convenient way to compare different network paths and topologies. In general, paths with fewer hops have lower latency, while paths with more hops incur additional forwarding delays as the message traverses each switch along the route.

Propagation delay is another significant contributor to network latency. It is the time required for a signal to travel through a cable from one device to another and depends primarily on the transmission medium.

For Omni-Path links:

Because copper cables are typically only a few meters long and are most commonly used for intra-rack connections, their propagation delay is often small enough to be ignored in practical latency calculations. Longer optical cables; however, can introduce measurable latency. For example, a 50 meter optical cable adds approximately 274 ns of propagation delay, which is comparable to the switching latency introduced by traversing multiple Switches.

Minimizing latency is critical for many high-performance computing (HPC) and distributed applications. Many scientific and engineering workloads divide a problem into a series of time steps, iterations, or sweeps across a computational domain. At the end of each phase, participating processes must synchronize before the next phase can begin. During this synchronization period, processes that complete their work early remain idle while waiting for slower communications to finish. As a result, network latency directly reduces overall application efficiency.

In addition to synchronization events, most applications must exchange data throughout execution. These data exchanges often occur between dependent processes that cannot continue their work until the required data has been received. As network latency increases, the time spent waiting for these communications also increases, reducing effective utilization of compute resources.

Even a small number of high-latency communication paths can affect overall application performance. An application may require all processes to reach a synchronization point before advancing to the next stage of computation. In this scenario, a single process communicating over a path with unusually high latency can delay every other process, causing idle time across the entire system. For this reason, low-latency switches, efficient topologies, and appropriately sized cable lengths are important factors in achieving optimal fabric performance.

Tail Latency is the maximum latency experienced by messages in a fabric. The longer the tail latency, both in absolute terms and relative to the average fabric latency, the more likely and more severe this type of performance bottleneck becomes.

Host-end endpoint processing latency is the time spent within the host and network adapter before data is transmitted onto the network or after it is received from the network. Unlike propagation delay or switching latency, which occur within the network fabric, endpoint processing latency occurs at the communicating endpoints themselves. This latency can include several activities, such as:

For small messages, endpoint processing latency can be a significant component of overall communication time. Because the transmission time for a small packet on a 400 Gbps link is extremely short, the total latency is often dominated by the time required to prepare, process, and complete the communication operation. In these cases, application performance is determined less by available bandwidth and more by how efficiently the host platform, software stack, and SuperNIC can process individual messages.

As a result, reducing endpoint processing latency is critical for latency-sensitive applications and workloads that exchange large numbers of small messages, where communication efficiency depends on both low network latency and high message-processing rates.

Bus latency is the time required for data, descriptors, and completion information to travel between the host system and the network adapter over the host I/O bus. In modern servers, this typically refers to PCI Express (PCIe) latency, since PCIe is the standard interconnect used to connect high-performance network adapters to the CPU and memory subsystem. Several factors contribute to bus latency, including:

Although bus latency is often much smaller than application execution time, it can be a significant component of end-to-end communication latency, particularly for small messages. In these scenarios, the time required to move data and metadata between system memory and the network adapter may be comparable to, or even greater than, the time spent transmitting the data across the network itself. For optimal performance, high-performance computing (HPC) and low-latency applications typically seek to minimize bus latency by placing adapters close to the CPUs that use them, aligning memory allocation with the appropriate NUMA domain, and reducing unnecessary PCIe switch traversals. These practices help reduce communication overhead and improve overall message latency and throughput.

Network topologies with minimal hop counts.

Appropriate cable types and short cable lengths where practical.

Balanced path lengths across the fabric.

Strong NUMA locality between applications, memory, and network adapters.

Efficient host and operating-system configurations.

Minimal PCIe switch traversal and other host-side bottlenecks.