5.3. Single Root I/O Virtualization
Single Root I/O Virtualization (SR-IOV) is a technology that allows a single physical network adapter to appear as multiple virtual adapters able to be passed to and used by virtual machines. Think of it like having one highway that can create multiple dedicated lanes for different vehicles, each with direct access to their destination without traffic jams.
The CN5000 Omni-Path implements SR-IOV with the following key features:
Each PF supports up to seven Virtual Functions (VFs), for a total of up to 14 VFs across both data lanes.
Each VF maps to a System Image (SI) rather than a separate virtual SuperNIC.
Note
Enabling VFs does not create additional fabric endpoints—the CN5000 appears as a single SuperNIC to the fabric regardless of how many VFs are active.
Up to seven Virtual Functions (VFs) per Physical Function (PF): This means one physical CN5000 SuperNIC can support up to seven separate virtual connections.
Direct hardware access: Each Virtual Machine (VM) gets its own dedicated hardware pathway.
Isolation and security: Each VF operates independently, ensuring one VM cannot interfere with another.
Important Notes
SR-IOV auto-enable is off by default. VFs must be explicitly enabled using the procedure in Configure and Verify SR-IOV.
Driver Requirements: Ensure the hfi1 driver is properly installed and loaded.
Persistence: SR-IOV settings are not persistent across reboots. Refer to Persist the SR-IOV Configuration (Recommended).
Procedure | Reference |
|---|---|
Enable platform settings | |
Enable IOMMU | |
Enable large prefetchable BAR support | |
Configure and verify VFs | |
Persist the SR-IOV configuration | |
Configure SR-IOV per operating system instructions |