3.6.3. Verify Cables
A loose, damaged, or incorrect cable can mimic symptoms of serious hardware failures. Cables are often the simplest yet most common source of hardware problems. Checking cables first can save significant time and money. It's also a non-invasive troubleshooting step that doesn't risk damaging equipment.
For additional information on verifying cables, refer to CN5000 Maintenance and Troubleshooting Guide, Hardware Troubleshooting.
3.6.3.1. Generate Cable Map Topology Files
Two sample topology files, detailed_topology.xlsx and minimal_topology.xlsx, are provided in the release.
Both sample topology files,
detailed_topology.xlsxandminimal_topology.xlsx, are spreadsheets with 3 tabs:Tab 1 Fabric is for you to customize with EXTERNAL links.
Tab 2 swd06 contains the internal links for a CN5000 Omni-Path Switch.
Tab 3 swd24 contains the internal links for a CN5000 Omni-Path Director Class Switch.
Note
You should not modify tab 2 and 3.
README.topologyandREADME.xlat_topologydescribe best practices for editing the sample topology files.
3.6.3.1.1. Defining Type in the Topology Spreadsheet
All host nodes should be defined as Type = FI in column F of the spreadsheet. All CN5000 Switch should be defined as Type = SW in column L (destination from host to Switch) and column F (source for Switch to core that is also a CN5000 Switch). The following example shows links between host and CN5000 Switch.
R19 opahost1 1 FI R19 opaedge1 13 SW opahost1_opae1p13 1m Cable CU
All links between a CN5000 Switch to core that is also a CN5000 Switch should be defined as Type = SW, as shown in the following example:
row1 rack01 opaedge1 1 SW row1 rack04 opaedgecore1 2 SW opae1p1_opac1p2 5M Cable Fiber
All Director Class Switches should be defined as Type = CL in column L (destination from CN5000 Switch to Director switch). Column J (Name-2) should have the destination leaf and column K should have the port number on that leaf. The following example shows a link between a CN5000 Switch to core that is a Director switch.
R19 opaedge1 5 SW R72 opadirector1 01 L105B 11 CL opae1p5opad1L105Bp11 30m Fiber
All Director Class Switches should be defined as shown in the following example:
Core Name:opadirector1 Core Group:row1 Core Rack:rack72 Core Size:1152 Core Full:0
Set Core Full to 0 if the Director switch is not fully populated with all the leafs and spines. If it is fully populated, set Core Full to 1.
3.6.3.1.2. Creating the Topology File
To create a topology file, perform the following steps:
Copy and save either
detailed_topology.xlsxorminimal_topology.xlsxlocated in/usr/share/opa/samplesfrom the Fabric Manager node to your local PC for editing.Edit tab 1 in the spreadsheet to reflect your specific installation details. Save tab 1 as
<topologyfile>.csvand copy this CSV file back to the Fabric Manager node.Note
The cable label field can be up to 57 characters.
Generate the topology file in XML format using the following command and your customized CSV file as the source:
# opaxlattopology <topologyfile>.csv <topologyfile>.xml
If there are Director Class Switches defined in the CSV file, then opaxlattopology includes all the ISL (internal chassis links between leafs and spines) in the XML file.
3.6.3.2. Verify Cable Map Topology
This section describes how to use the fabric topology.xml file created in Defining Type in the Topology Spreadsheet to verify that fabric topology (cabling) is consistent with the cable map.
The command opareport -o verify* -T <topologyfilename>.xml compares the live fabric interconnect against the topology file created based on the cable map. These commands test links, switches, and SM topology. If successful, the output reports a total of 0 Incorrect Links found, 0 Missing, 0 Unexpected, 0 Misconnected, 0 Duplicate, and 0 Different.
# opareport -o verifyfis -T <topologyfilename>.xml # opareport -o verifyextlinks -T <topologyfilename>.xml # opareport -o verifyall -T <topologyfilename>.xml
In most cases, links reported with errors are either due to incorrect cabling to the wrong port or the topology.csv file has incorrect source and port destinations.
After all topology issues have been resolved, copy the topologyfile.xml from the local working directory to cat /etc/opa/topology.0\:0.xml.