Skip to main content

Cornelis Technical Documentation

2.4.3. Routing

Routing in Omni-Path fabrics relies on Local Identifiers (LIDs), which the Fabric Manager assigns to every device in the fabric. These LIDs differ from Global Unique Identifiers (GUIDs), which manufacturers assign to each device. The system sets LID length to either 16 or 24 bits based on the header length specified in the opafm.xml configuration file, corresponding to 9-byte or 16-byte headers, respectively.

Linear Local Identifiers (LLIDs) use a single 16-bit or 24-bit integer to represent a node’s address. Each switch sets up forwarding tables to read an incoming message’s LLID, locate its entry in the table, determine the corresponding outgoing port, and send the message through that port. Hierarchical Local Identifiers always use 24 bits, with 22 bits allocated to three separate integers representing a node’s address. These integers follow a hierarchical structure, defining increasingly granular levels of addressing in the following format: POD.SWITCH.TERMINAL

The first 8-bit integer identifies the pod, a large region of the fabric. In a fat tree topology, it represents the location of a tier-2 or tier-3 switch rather than the specific node’s location. The second 7-bit integer specifies the switch directly connected to the desired node, which, in a fat tree, is a tier-1 switch. The final 7-bit integer designates the port of the desired node on the tier-1 switch. These distinct addressing fields enable the construction of significantly larger fabrics compared to LLIDs. Omni-Path 100 endpoints do not support HLIDs or 16-byte headers.