3. CN5000 Fabric Installation

Updated Install the OPX Software to move copying software to the remaining servers under RHEL and SLES-specific instructions.

Updated Configuring PXE Boot Over IPoIB to include instructions for RHEL 9 and 10.

Added note to Overview and Architecture pertaining to using shimx64.efi instead of grubx64.efi for secure PXE Boot.

Added Arabic translations to the Safety Statements.

Updated label information and image in “SuperNIC Manufacturing Information” under “Unpacking the SuperNIC”. Includes Version I and Version II board information.

Updated “Starting the Fabric Manager” to include Version II SuperNIC board prerequisite.

Updated “Setting Up PXE Boot” to add a note explaining that RHEL 9.8 ships a smaller install initrd and a different driver dependency layout than previous RHEL versions.

Updated “Prepare the Image for the Client” and “Configure the Boot Loader” to include RHEL 9.8 specific steps.

Updated “Installing CUDA” to include instructions for RHEL, SLES, and Ubuntu.

Added new section, “UEFI Secure Boot Support”.

Added an overview paragraph to “Fabric Verification”.

Added “Resolving CUDA Dependency Errors for OPX Software Installation”.

Updated “Compiling Libfabric on a System With or Without CN5000 Omni-Path Hardware”, Step 5 to add a note for configuring GPU options.

Added “Rebuilding the Libfabric Source RPM with CUDA or ROCm Support”.

Added "Running Kubernetes on a CN5000 Fabric”.

Updated “Liquid-Cooled Equipment Requirements” with table and figures specifying liquid-cooled values and limits, respectively.

Added NOTE pertaining to hfi1 to “NVIDIA GPU Requirements”.

Updated “Compiling Libfabric on a System With or Without CN5000 Omni-Path Hardware” to include systems with Omni-Path hardware.

Updated “Install the OPX Software”, "Create the Repository File" and "Import the GPG Key" steps.

Added Signal Words (Caution, Warning) to Appendix A “Safety Statements”.

Updated "Install the OPX Software", Step 6 for resolving Ubuntu dependency conflicts.

Added "Liquid-Cooled Equipment Requirements".

Added caution to "Cable Guidelines" that DAC/ACC cables are not rated for installation in air plenums.

Updated "Install the DCS" to better present liquid-cooled installations.

Updated "Liquid-Cooled Installations".

Added "IOMMU: Passthrough Mode".

Updated "Install the OPX Software" to include the following:

Added Ubuntu instructions for "Configuring IPoIB Driver".

Added "Setting Router Advertisement for IPv6".

Updated "IPoIB Bonding" with a note to not bond IPoIB interfaces to the same port that is running the Subnet Manager.

Updated graphics and text in “Liquid-Cooled Installations” to indicate hot and cold lines.

Added new section, “Using RCCL”.

Added new section, “Compiling Libfabric on a System Without CN5000 Omni-Path Hardware”.

Updated “Install the OPX Software” to include new Ubuntu instructions.

Updated “Configure IPoIB Network Interface” to include Ubuntu instructions.

Added new section, “Power Precautions” for the CN5000 Director Class Switch.

Updated “Cabling the CN5000 Switch” to include an important note to ensure that the attached cable supports the speed of the configured port.

Updated “Starting the Fabric Manager” to include a note ensuring the Fabric Manager is run from a node connected to the latest generation switch.

Added new section, “Configuring Boot Over Fabric”, including “Setting Up PXE Boot”.

Updated “AMD GPU Requirements” to include information for installing kernel-devel-matched and dkms.

Updated “NVIDIA GPU Requirements” to reorganize information and remove -y from “Installing CUDA on RHEL/Rocky Linux” step commands

Added Cable Management kit installation to “Cabling the DCS”.

Added “Liquid-Cooled Installations”.

Updated “CN5000 Switch and DCS Start-up Procedures” to add note for liquid-cooled fan sensor error message on power up that can be ignored.

Added nvidia-driver-cuda-libs to “Installing CUDA on RHEL/Rocky Linux”.

Initial release.