4.3.6. Dual-Plane Fat-Tree Fabrics
CN5000 uses a dual-plane fabric model for high-performance fat-tree deployments. In this model, each CN5000 dual-port SuperNIC connects to two independent fabric planes, commonly referred to as Plane 0 and Plane 1. Each adapter port operates at up to 400 Gbps, and the two ports together provide up to 800 Gbps of aggregate adapter bandwidth when both planes are used. In a conventional single-plane fat-tree, each endpoint connects to one fabric tree. The fat-tree provides path diversity through multiple uplinks and core switches, and routing distributes traffic across the available paths. A dual-plane CN5000 fat-tree should be understood as two parallel fat-tree fabrics, not as one fat-tree with arbitrary cross-connections between planes. Each plane has its own edge, aggregation, and core switch connectivity, and routing is performed independently within that plane.
SuperNIC are designed around two independent 400 Gbps processing paths. Each processing path has its own send and receive resources, flow-control behavior, and fabric port. This architecture allows traffic to progress independently on the two planes. This design allows CN5000 to use dual-plane fabrics for both aggregate bandwidth scaling and traffic separation. A workload that can use both adapter ports can send traffic across both planes, achieving higher aggregate throughput than a single 400 Gbps port.
4.3.6.1. Advantages
A dual-plane fabric offers two separate pathways within the network, enhancing performance and resilience. In a CN5000 setup, each adapter connects one 400 Gbps port to each fabric plane, reaching a total of 800 Gbps when both are active. The key benefit of this architecture is increased bandwidth, as traffic is spread across two distinct fabric planes rather than a single route. This setup enables systems and applications to utilize both planes to achieve higher overall throughput.
This approach enhances resiliency and fault isolation by separating the planes both physically and logically. When an issue occurs in one plane, it can often be isolated without impacting the other, helping to maintain connectivity and simplify recovery. However, if one plane becomes unavailable, available bandwidth may be reduced. Additionally, this design allows for more predictable scaling, as each plane can be independently designed, routed, and evaluated. This approach makes it easier to assess capacity, manage oversubscription, and plan for fabric expansion.
Finally, dual-plane fabrics simplify operations and troubleshooting. Clear separation between planes makes it easier to document cabling, verify routing, and isolate performance issues before analyzing the combined behavior of the full fabric.
4.3.6.2. Limitations
A CN5000dual-plane fabric provides higher aggregate bandwidth and improved fault isolation, but it also adds complexity and does not automatically improve every workload. The full 800 Gbps aggregate adapter bandwidth is available only when the software stack and workload can effectively use both 400 Gbps ports. If traffic uses only one plane, performance is limited to that plane. Dual-plane designs also require additional infrastructure and careful planning to keep the planes separate, balanced, and operationally manageable.
Key limitations include:
800 Gbps aggregate bandwidth requires the use of both adapter ports.
Two fabric planes require more switches, cables, power, rack space, and management.
Plane separation must be maintained for predictable routing.
Fault isolation is improved, but system-level single points of failure can still remain.
Operations and troubleshooting are more complex than in a single-plane fabric.