Table Of Contents
InfiniBand Subnet Management Tasks
Using the InfiniBand Menu
Viewing and Managing Subnet Manager Properties
Viewing Subnet Manager Properties
Adding a Subnet Manager
Removing a Subnet Manager
Configuring Subnet Manager Properties
Configuring Subnet Manager Priority
Configuring the Sweep Interval
Configuring Response Timeout
Configuring the Master Poll Interval
Configuring the Number of Master Poll Retries
Configuring the Maximum Supported Number of Active Standby Subnet Managers
Configuring LID Mask Control
Configuring Switch Lifetime
Configuring Switch Link HoQ Life
Configuring Maximum Hop Count
Configuring MAD Retries
Configuring Node Timeout
Configuring Wait Report Response
Configuring Subnet Administrator MAD Queue Depth
Viewing and Managing Database Synchronization
Viewing Database Synchronization
Configuring Database Synchronization
Enabling Subnet Manager Database Synchronization
Configuring the Maximum Number of Backup Subnet Managers to Synchronize
Configuring a Session Timeout
Configuring the Poll Interval
Configuring the Cold Synchronization Timeout Value
Configuring the Cold Synchronization Limit Value
Configuring the Cold Synchronization Limit Period
Configuring the New Session Delay
Configuring the Resynchronization Interval
Viewing the Database Synchronization State
Viewing and Managing Nodes and Ports
Viewing Node Information
Viewing Port Information
Routing Around Nodes and Ports
Routing Around Nodes
Routing Around Ports
Viewing and Managing Partitions
Viewing Partitions
Creating a Partition
Removing a Partition
Enabling or Disabling IPoIB for a Partition
Viewing Partition Details
Adding Full Members to a Partition
Adding Available Members to a Partition
Adding Unavailable Members to a Partition
Adding Limited Members to a Partition
Adding Available Limited Members
Adding Unavailable Members
Viewing and Managing Multicast Groups
Viewing Multicast Groups
Viewing Multicast Group Details
Viewing Multicast Group Members
Configuring Multicast Groups
Configuring IPoIB Broadcast Multicast Groups
Viewing InfiniBand Services
Viewing and Managing InfiniBand Routes
Viewing InfiniBand Routes
Routing Around Components in an InfiniBand Network
Viewing Route-Around Information
Adding Routes to the Route-Around Table
Removing Routes from the Route-Around Table
Viewing Other Subnet Managers Information
Viewing Event Subscriptions
Viewing Forwarding Tables
Viewing Multicast Forwarding Information
Viewing Linear Forwarding Information
InfiniBand Subnet Management Tasks
These topics describe the InfiniBand menu subnet management tasks for Element Manager:
•
Using the InfiniBand Menu
•Viewing and Managing Subnet Manager Properties
•Viewing and Managing Database Synchronization
•Viewing and Managing Nodes and Ports
•Viewing and Managing Partitions
•Viewing and Managing Multicast Groups
•Viewing InfiniBand Services
•Viewing and Managing InfiniBand Routes
•Viewing Other Subnet Managers Information
•Viewing Event Subscriptions
•Viewing Forwarding Tables
Note See "InfiniBand Concepts" to familiarize yourself with the InfiniBand technology. For hardware-specific information, consult the relevant hardware documentation.
Using the InfiniBand Menu
The InfiniBand menu has two choices for performing InfiniBand subnet management tasks:
•Subnet Management
•Subnet Management (tabular format)
These topics describe how to use the Subnet Management menu option. Most of the tasks can also be performed by selecting the Subnet Management (tabular format) menu option, which presents information and configurable options in tables, but is a less user friendly way to perform your InfiniBand subnet management tasks.
Viewing and Managing Subnet Manager Properties
These topics describe procedures for performing the following tasks:
•Viewing Subnet Manager Properties
•Adding a Subnet Manager
•Removing a Subnet Manager
•Configuring Subnet Manager Properties
Viewing Subnet Manager Properties
To view Subnet Manager properties, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Manager window opens.
Step 2 Select a subnet.
A table of Subnet Manager properties appears under the General tab. Table 8-1 describes the fields.
Table 8-1 Subnet Management Window, General Tab Fields
Field
|
Description
|
GUID
|
Displays the GUID of the port on which the Subnet Manager runs.
|
Status
|
Status of the Subnet Manager. The status can be master, standby, inactive, or discovery.
|
Activity Count
|
Activity counter that increments each time that the Subnet Manager issues a subnet management packet (SMP) or that performs other management activities.
|
SM Key
|
64-bit subnet management key assigned to a Subnet Manager.
|
Priority
|
Priority of the Subnet Manager relative to other Subnet Managers in the InfiniBand network. Priority is assigned to the higher number.
|
Sweep Interval
|
Specifies how frequently a Subnet Manager queries the InfiniBand fabric for network changes.
|
Response Timeout
|
Maximum amount of time that the Subnet Manager waits for a response after it sends a packet to a port. If the Subnet Manager does not receive a response in time, the Subnet Manager identifies the port as unresponsive.
|
Master Poll Interval
|
Interval at which a slave Subnet Manager polls a master to see if it still runs.
|
Master Poll Retries
|
Number of unanswered polls that cause a slave to identify a master as dead.
|
Max Active SMs
|
Maximum number of standby Subnet Managers that a master supports. A value of 0 indicates unlimited Subnet Managers.
|
LID Mask Control
|
Number of path bits present in the base LID to each channel adapter port. Increasing the LMC value increases the number of LIDs assigned to each port to increase the number of potential paths to reach each port.
|
Switch Life Time
|
The lifetime of a packet inside a server switch. This value defaults to 20.
|
Switch Link HoQ Life
|
The lifetime of a packet at the head of queue of a switch port. This value defaults to 20.
|
CA Link HoQ Life
|
The lifetime of a packet at the head of queue of the host port. This value defaults to 20.
Note This value is set automatically to the same value as Switch Link HoQ Life.
|
Maximum Hop Count
|
Specifies the number of hops. Range is from 0 to 64. Default is 64. A value of 0 causes the Subnet Manager to calculate and use the lowest possible value that will still ensure connectivity between all endpoints.
|
MAD Retries
|
Number of times that a Subnet Manager resends a MAD after not receiving a response. The default value is 5.
|
Node Timeout
|
Minimum amount of time in seconds that a HCA is unresponsive before the Subnet Manager removes it from the InfiniBand fabric. The default value is 10 seconds.
|
Wait Report Response
|
Whether or not a Subnet Manager waits to receive ReportResponse MADs in response to the Report MAD that it forwards. If this Boolean value is set to false, the Subnet Manager only sends the Report MADs once; if set to true, the Subnet Manager will continue to send the Report MADs until either the Report Response MAD is received or the maximum number of Report MADs have been sent. The default value is false.
|
SA MAD Queue Depth
|
Size of a Subnet Administrator internal queue for receiving MADs. The default value is 256.
|
See the "Configuring Subnet Manager Properties" procedure for details on how to configure these properties.
Adding a Subnet Manager
To add a Subnet Manager to your server switch, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 In the navigation pane, click Subnet Managers.
The Subnet Managers display appears in the right pane of the window.
Step 3 Click Add.
The Add Subnet Manager window opens.
Step 4 In the Subnet Prefix field, enter a subnet prefix.
Step 5 In the Priority field, enter a subnet priority level.
Step 6 (Optional) In the smKey field, enter a subnet management key.
Step 7 (Optional) In the LID Mask Control field, enter a value to increase the number of LIDs assigned to each port to increase the number of potential paths to reach each port.
Step 8 Click Add.
The new Subnet Manager appears in the Summary table in the Subnet Managers display.
Removing a Subnet Manager
To remove a Subnet Manager from your server switch, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 In the navigation pane, click Subnet Managers.
The Subnet Managers display appears in the right pane of the window.
Step 3 In the Summary table in the Subnet Managers display, click the Subnet Manager that you want to remove.
Step 4 Click Remove.
The entry disappears from the display and the server switch configuration.
Configuring Subnet Manager Properties
The Subnet Managers navigation menu provides tuning for a number of system-wide attributes. These topics explain each attribute and describe how to configure it:
•Configuring Subnet Manager Priority
•Configuring the Sweep Interval
•Configuring Response Timeout
•Configuring the Master Poll Interval
•Configuring the Number of Master Poll Retries
•Configuring the Maximum Supported Number of Active Standby Subnet Managers
•Configuring LID Mask Control
•Configuring Switch Lifetime
•Configuring Switch Link HoQ Life
•Configuring Maximum Hop Count
•Configuring MAD Retries
•Configuring Node Timeout
•Configuring Wait Report Response
•Configuring Subnet Administrator MAD Queue Depth
Configuring Subnet Manager Priority
Every Subnet Manager in the InfiniBand network carries a priority value, and at any given time the Subnet Manager with the highest integer value priority becomes the master Subnet Manager. To configure the Subnet Manager priority on your server switch, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the Priority field, select the value, and replace it with the value you want to apply.
The integer value 15 has the highest priority.
Step 4 Click Apply.
Configuring the Sweep Interval
The sweep interval specifies how frequently the Subnet Manager queries the InfiniBand fabric for network changes. To configure the sweep interval on your server switch, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the Sweep Interval field, select the value, and replace it with the value you want to apply.
This interval represents the number of seconds between sweeps.
Step 4 Click Apply.
Configuring Response Timeout
The response timeout of a Subnet Manager specifies the maximum amount of time that the Subnet Manager waits for a response after it sends a packet to a port. If the Subnet Manager does not receive a response in the response-time interval, the Subnet Manager identifies the port as unresponsive. To configure the response timeout, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the Response Timeout field, select the value, and replace it with the value you want to apply.
The Subnet Manager measures the response timeout in milliseconds.
Step 4 Click Apply.
Configuring the Master Poll Interval
The master poll interval determines the interval at which the slave Subnet Manager polls the master to see if the master still runs. To configure the master poll interval, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the Master Poll Interval field, select the value, and replace it with the value you want to apply.
The value represents the interval, in seconds.
Step 4 Click Apply.
Configuring the Number of Master Poll Retries
Master poll retries specifies the number of unanswered polls that cause a slave to identify a master as dead. To specify this value, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the Master Poll Retries field, select the value, and replace it with the value you want to apply.
Step 4 Click Apply.
Configuring the Maximum Supported Number of Active Standby Subnet Managers
Note To configure an unlimited number of active standby (slave) Subnet Managers, enter a value of 0. However, the limit set here is not enforced in this release.
To configure the maximum number of active standby Subnet Managers that the master Subnet Manager supports, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the Max active SMs field, select the value, and replace it with the value you want to apply.
Step 4 Click Apply.
Configuring LID Mask Control
Local ID mask control assigns the number of path bits present in the base LID to each channel adapter port. Increasing the LMC value increases the number of LIDs assigned to each port to increase the number of potential paths to reach each port. To configure LID mask control, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the LID Mask Control field, select the value, and replace it with the value you want to apply.
Step 4 Click Apply.
Configuring Switch Lifetime
Switch lifetime is one parameter that governs the transmitter packet discard policy of switches in the subnet. It determines the lifetime of packets in a switch from the point of ingress to egress. If this parameter is set to 20 or greater, then switch lifetimes are infinite (default). See InfiniBand Architecture Release 1.2, Volume 1 for more information.
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the Switch Life Time field, select the value, and replace it with the value you want to apply.
Step 4 Click Apply.
Configuring Switch Link HoQ Life
Switch link head of queue life determines how long an InfiniBand packet lives at the head of a switch port VL queue before it is discarded. If this parameter is set to 20 or greater, then HoQ lifetimes are infinite (default). See InfiniBand Architecture Release 1.2, Volume 1 for more information.
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the Switch Link HoQ Life field, select the value, and replace it with the value you want to apply.
Step 4 Click Apply.
Configuring Maximum Hop Count
We recommend that InfiniBand switch elements be connected so that all paths between any pair of switch elements are the same distance (same number of hops), if possible.
The range of values is from 0 to 64. Default is 64. A value of 0 causes the Subnet Manager to calculate and use the lowest possible value that will still ensure connectivity between all endpoints.
Note Selecting any nondefault value restricts the length of paths used by the Subnet Manager. The Subnet Manager might therefore select paths that are optimal for distance, but not for other factors, such as link capacity.
To configure the maximum number of hops for an InfiniBand Subnet Manager, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the Maximum Hop Count field, select the value, and replace it with the value you want to apply.
Step 4 Click Apply.
Configuring MAD Retries
MAD retries specifies the number of times that a Subnet Manager resends a management datagram after not receiving a response. The default value is 5.
To configure MAD retries, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the MAD Retries field, select the value, and replace it with the value that you want to apply.
Step 4 Click Apply.
Configuring Node Timeout
Node Timeout is the minimum amount of time in seconds that a HCA is unresponsive before the Subnet Manager removes it from the InfiniBand fabric. The default value is 10 seconds.
To configure the node timeout, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the Node Timeout field, select the value, and replace it with the value that you want to apply.
Step 4 Click Apply.
Configuring Wait Report Response
Wait Report Response configures whether or not a Subnet Manager waits to receive ReportResponse MADs in response to the Report MAD that it forwards. If you set this Boolean value to false, the Subnet Manager only sends the Report MADs once; if you set it to true, the Subnet Manager will continue to send the Report MADs until either the Report Response MAD is received or the maximum number of Report MADs have been sent. The default value is false.
To configure the wait report response, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the Wait Report Response field, check the Enable box.
Step 4 Click Apply.
Configuring Subnet Administrator MAD Queue Depth
This procedure configures the size of a Subnet Administrator internal queue for receiving MADs. The default value is 256.
To configure the Subnet Administrator MAD queue depth, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Click the Subnet Manager that you want to configure.
A table of subnet manager properties appears under the General tab.
Step 3 In the SA MAD Queue Depth field, select the value, and replace it with the value that you want to apply.
Step 4 Click Apply.
Viewing and Managing Database Synchronization
Element Manager provides multiple screens that you can use to view and configure database synchronization. This section describes the following tasks:
•Viewing Database Synchronization.
•Configuring Database Synchronization.
Viewing Database Synchronization
To view database synchronization details, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Manager window opens.
Step 2 Select a subnet
Step 3 Click the Database Sync tab.
Details appear in the table below the tab. Table 8-2 describes the fields.
Note Database synchronization is enabled by default.
Table 8-2 Subnet Management Window, Database Sync Tab Fields
Field
|
Description
|
SM Database Synchronization
|
Check box to enable or disable synchronization of the database with a standby subnet manager.
|
Max Backup SMs
|
The maximum number of backup subnet managers that will synchronize with the master subnet manager.
|
Session Timeout
|
The interval, in seconds, during which a synchronization session status management datagram packet must arrive at the master subnet manager to maintain synchronization.
|
Poll Interval
|
Interval at which the master subnet manager polls an active slave subnet manager to verify synchronization.
|
Cold Sync Timeout
|
Maximum amount of time in which subnet managers can perform a cold synchronization. During the cold-sync, the master subnet manager copies all out-of-sync tables to the standby subnet manager.
|
Cold Sync Limit
|
Maximum number of cold synchronizations that can take place during the cold-sync period.
|
Cold Sync Limit Period
|
Length in seconds of the interval during which cold-syncs can occur.
|
New Session Delay
|
Amount of time in seconds that the master subnet manager waits before it attempts to initiate a synchronization session with a new subnet manager.
|
Resync Interval
|
Specifies the interval at which the master subnet manager sends a re-synchronization request to all active synchronization sessions.
|
State
|
Specifies whether or not the subnet manager is synchronized with the backup.
|
Configuring Database Synchronization
The database synchronization feature propagates information from the database of the master Subnet Manager to the standby Subnet Managers. These topics describe how to configure this feature:
•Enabling Subnet Manager Database Synchronization
•Configuring the Maximum Number of Backup Subnet Managers to Synchronize
•Configuring a Session Timeout
•Configuring the Poll Interval
•Configuring the Cold Synchronization Timeout Value
•Configuring the Cold Synchronization Limit Value
•Configuring the Cold Synchronization Limit Period
•Configuring the New Session Delay
•Configuring the Resynchronization Interval
•Viewing the Database Synchronization State
Enabling Subnet Manager Database Synchronization
If you are configuring more than one InfiniBand chassis in your fabric, it is likely that you will want to enable database synchronization of the Subnet Managers.
Note This feature is enabled by default.
To enable Subnet Manager database synchronization to update standby Subnet Managers with information from the master Subnet Manager, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Click the Subnet Manager that you want to configure.
Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 3 Click the Database Sync tab in the right pane.
Step 4 In the SM Database Synchronization field, check the Enable check box.
Step 5 Click Apply.
Configuring the Maximum Number of Backup Subnet Managers to Synchronize
To configure the maximum number of backup Subnet Managers that will synchronize with the master Subnet Manager, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Click the Subnet Manager that you want to configure.
Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 3 In the right pane, click the Database Sync tab.
Step 4 In the Max Backup SMs field, enter an integer value.
Step 5 Click Apply.
Configuring a Session Timeout
To configure the session timeout interval, in seconds, during which a synchronization session status MAD packet must arrive at the master Subnet Manager to maintain synchronization, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Click the Subnet Manager that you want to configure.
Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 3 In the right pane of the display, click the Database Sync tab.
Step 4 In the Session Timeout field, enter an integer value.
This value determines the timeout duration, in seconds.
Step 5 Click Apply.
Configuring the Poll Interval
To configure the interval, in seconds, at which the master Subnet Manager polls an active slave Subnet Manager to verify synchronization, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
Step 2 The Subnet Management window opens.
Step 3 Click the Subnet Manager that you want to configure.
Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 4 In the right pane of the display, click the Database Sync tab.
Step 5 Enter an integer value in the Poll Interval field.
This value sets the poll interval, in seconds.
Step 6 Click Apply.
Configuring the Cold Synchronization Timeout Value
To configure the amount of time, in seconds, that a cold synchronization tries to initiate before it times out, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Click the Subnet Manager that you want to configure.
Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 3 In the right pane of the display, click the Database Sync tab.
Step 4 In the Cold Sync Timeout field, enter an integer value.
This value sets the timeout interval, in seconds.
Step 5 Click Apply.
Configuring the Cold Synchronization Limit Value
To configure the maximum number of cold synchronizations to perform during a given cold synchronization period, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Click the Subnet Manager that you want to configure.
Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 3 In the right pane of the display, click the Database Sync tab.
Step 4 In the Cold Sync Limit field, enter an integer value.
This value sets the maximum number of synchronizations that can occur during the synchronization period. (See "Configuring the Cold Synchronization Limit Period" section.)
Step 5 Click Apply.
Configuring the Cold Synchronization Limit Period
To specify the length of the interval during which cold synchronizations may occur, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Click the Subnet Manager that you want to configure.
Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 3 In the right pane of the display, click the Database Sync tab.
Step 4 In the Cold Sync Limit Period field, enter an integer value.
This value sets the length of the interval during which cold synchronizations may occur.
Step 5 Click Apply.
Configuring the New Session Delay
To configure the amount of time that the master Subnet Manager waits before it attempts to initiate a synchronization session with a new Subnet Manager, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Click the Subnet Manager that you want to configure.
Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 3 In the right pane of the display, click the Database Sync tab.
Step 4 In the New Session Delay field, enter an integer value.
This value determines the amount of time, in seconds, that the master Subnet Manager waits before it attempts to initiate a synchronization session with a new Subnet Manager.
Step 5 Click Apply.
Configuring the Resynchronization Interval
To specify the interval at which the master Subnet Manager sends a resynchronization request to all active synchronization sessions, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Click the Subnet Manager that you want to configure.
Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 3 In the right pane of the display, click the Database Sync tab.
Step 4 In the Resync Interval field, enter an integer value.
This value specifies the interval, in seconds, at which the master Subnet Manager sends a resynchronization request to all active synchronization sessions.
Step 5 Click Apply.
Viewing the Database Synchronization State
To view the database synchronization state and verify that the master Subnet Manager and slave Subnet Manager(s) are synchronized, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Select the Subnet Manager with the state you wan to view.
Step 3 In the right pane of the display, click the Database Sync tab.
Step 4 Look at the State field.
Viewing and Managing Nodes and Ports
This section provides procedures for performing the following tasks:
•Viewing Node Information
•Viewing Port Information
•Routing Around Nodes and Ports
Viewing Node Information
To view Subnet Manager node information, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Expand the Subnet Manager that you want to view.
Step 3 Select Nodes.
The Nodes in Subnet tab displays the Node GUID, Type, Description, Number of Ports, System Image GUID, and the Vendor ID information. See Table 8-3 for details.
Step 4 Click Show Advanced to display the additional information about each of the nodes in the subnet. This information includes Base Version, Class Version, Port GUID, Partition Cap, Device ID, Revision, and Local Port Number. Table 8-3 describes these fields.
Table 8-3 Nodes in Subnet Tab Fields
Field
|
Description
|
Node GUID
|
GUID of the node.
|
Type
|
Type of node being managed. The value appears as channel adapter, switch, router, or error. An error entry indicates an unknown type.
|
Description
|
Text string describing the device.
|
Number of Ports
|
Number of physical ports on the node.
|
System Image GUID
|
GUID of an associated supervisory node. No supervisory node exists if the output displays 00:00:00:00:00:00:00:00.
|
Vendor ID
|
Device vendor ID. The value appears the same for all ports on the node.
|
Base Version
|
Supported base management datagram (MAD) version. Indicates that this channel adapter, switch, or router supports versions up to and including this version. See section 13.4.2, "Management Datagram Format," in InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
Class Version
|
Supported MAD class format version. Indicates that this channel adapter, switch, or router supports versions up to, and including, this version.
|
Device ID
|
Manufacturer-assigned device identification.
|
Revision
|
Manufacturer-assigned device revision.
|
Partition Cap
|
Capacity of entries in the partition table for channel adapter, router, and the switch management port. The value appears the same for all ports on the node. This defaults to at least 1 for all nodes including switches. You cannot configure this value.
|
Local Port Number
|
The link port number from which this subnet management packet (SMP) arrived. The value appears the same for all ports on the node.
|
Viewing Port Information
To view information about specific ports, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Expand the Subnet Manager with ports you want to view.
Step 3 Select Nodes.
Step 4 Expand Nodes.
Step 5 Expand the computer icon for the node with ports you want to view
Step 6 Select a specific port to see the information described in Table 8-4. Figure 8-1 shows a sample display.
Figure 8-1 Individual Port Information
Table 8-4 Ports Field Descriptions
Field
|
Description
|
Port
|
Local port number for this port.
|
LID
|
16-bit base LID of this port.
|
Port State
|
State of the port, as follows:
•noStateChange
•sleep
•polling
•disabled
•portConfigurationTraining
•linkup
•linkErrorRecovery
•reserved
•active
•down
|
Active link width
|
Used in conjunction with Active Link Speed to determine the link rate between two nodes. The value appears as 1x, 4x, or 12x.
|
Management Key
|
64-bit management key for this port. See section 14.2.4, "Management Key" and 3.5.3, "Keys," in InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
GID Prefix
|
64-bit Global identifier prefix for this port. The subnet manager assigns this prefix based upon the port routes and the rules for local identifiers. See section 4.1.3, "Local Identifiers," in InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
Master SM LID
|
16-bit base LID of the master subnet manager managing this port.
|
Cap Mask
|
The capability mask identifies the functions that the host supports. 32-bit bitmask that specifies the supported capabilities of the port. A bit value of 1 (one) indicates a supported capability. The bits are 0, 11-15, 18, 21-31 (Reserved and always 0.), 1 IsSM, 2 IsNoticeSupported, 3 IsTrapSupported, 4 IsResetSupported, 5 IsAutomaticMigrationSupported, 6 IsSLMappingSupported, 7 IsMKeyNVRAM (supports M_Key in NVRAM), 8 IsPKeyNVRAM (supports P_Key in NVRAM), 9 Is LED Info Supported, 10 IsSMdisabled, 16 IsConnectionManagementSupported, 17 IsSNMPTunnelingSupported, 19 IsDeviceManagementSupported, 20 IsVendorClassSupported.Values are expressed in hexadecimal.
|
Diagnostic Code
|
16-bit diagnostic code. See section 14.2.5.6.1 "Interpretation of Diagcode," in InfiniBand Architecture®, Vol. 1, Release 1.1, for more information. This field does not currently apply to your server switch.
|
MKey Lease Period
|
Initial value of the lease-period timer, in seconds. The lease period is the length of time that the M_Key protection bits are to remain non-zero after a SubnSet (PortInfo) fails an M_Key check. After the lease period expires, clearing the M_Key protection bits allows any subnet manager to read (and then set) the M_Key. Set this field to 0 to indicate that the lease period never expires. See InfiniBand Architecture®, Vol. 1, Release 1.1, section 14.2.4, "Management Key."
|
Enabled Link Width
|
Enabled link width (bandwidth). The value can be one of the following:
•no state change
•1x
•4x
•1x, 4x
•8x
•1x, 8x
•4x, 8x
•1x, 4x, 8x
•12x
•1x, 12x
•4x, 12x
•1x, 4x, 12x
•8x, 12x
•1x, 8x, 12x
•4x, 8x, 12x
•1x, 4x, 8x, 12x
•reserved
•linkwidthsupported value
|
Supported Link Width
|
Supported link width. The value appears as one of the following:
•1x,
•1x, 4x
•1x, 4x, 8x
•1x, 4x, 12x,
•1x, 4x, 8x, 12x
•reserved
|
Supported Link Speed
|
Supported link speed. The value appears as one of the following:
•sdr
•sdr, ddr
|
Physical State
|
Indicates the physical state of the port, whether or not electricity flows between nodes and that they can perform a handshake. The value appears as noStateChange, sleeping, polling, disabled, portConfigurationTraining, linkup, or linkErrorRecovery. The state, upon power-up, defaults to polling.
|
Link Down Def State
|
Default LinkDown state to return to. The value appears as noStateChange, sleeping, or polling. See section 5.5.2, "Status Outputs (MAD GET)," InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
MKey Prot Bits
|
Management key protection bits for the port. The bits are 0, 1, 2, and 3. See section 14.2.4.1, "Levels of Protection," InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
LID Mask
|
Local-identifier mask control (LMC) for multi-path support. A LMC resides on each channel adapter and router port on the subnet. It provides multiple virtual ports within a single physical port. The value of the LMC specifies the number of path bits in the LID. A value of 0 (zero) indicates one LID can apply to this port. See sections 3.5.10, "Addressing," and 4.1.3, "Local Identifiers," InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
Active Link Speed
|
Speed of an active link. The value appears as one of the following:
•sdr
•ddr
|
Enabled Link Speed
|
Maximum speed that the link can handle. The value appears as one of the following:
•sdr
•ddr
•sdr, ddr
|
Neighbor MTU
|
Active maximum transmission unit enabled on this port for transmit. Check the MTU cap value at both ends of every link and use the lesser speed. The value appears as 256, 512, 1024, 2048, or 4096.
|
MasterSmSL
|
Administrative service level required for this port to send a non-SMP message to the subnet manager.
|
Virtual Lanes Cap
|
Maximum range of data virtual lanes supported by this port. The value appears as vl0, vl0-Vl1, vl0-Vl3, vl0-Vl7, or vl0-Vl14. See also oper-VL. Each port can support up to 15 virtual lanes (VLs 0 - 15). The VL-cap field displays the range of those lanes (for example, lanes 0 - 7) that the port currently supports.
|
Virtual Lane High Limit
|
Maximum high-priority limit on the number of bytes allowed for transmitting high-priority packets when both ends of a link operate with multiple data virtual-lanes. Used with the virtual-lane arbitration table. The maximum high-limit matches the VLArbHighCap on the other side of the link and then negotiating downward.
|
VLArbHighCap
|
Highest arbitration value allowed by the arbiter in determining the next packet in a set of packets to send across the link. Used with the virtual-lane arbitration table and specified as a VL/Weight pair. See section 14.2.5.9, "VL Arbitration Table," InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
VLArb Low Cap
|
Lowest arbitration value allowed by the arbiter in determining the next packet in a set of packets to send across the link. Used with the virtual-lane arbitration table and specified as a VL/Weight pair. See section 14.2.5.9, "VL Arbitration Table," InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
MTU Cap
|
Used in conjunction with Neighbor MTU to determine the maximum transmission size supported on this port. The lesser of MTU cap and Neighbor MTU determines the actual MTU used. The value appears as 256, 512, 1024, 2048, or 4096.
|
VL Stall Count
|
Number of sequentially dropped packets at which the port enters a VLStalled state. The virtual lane exits the VLStalled state (8 * HLL) units after entering it. See section 18.2.5.4, "Transmitter Queuing," InfiniBand Architecture®, Vol. 1, Release 1.1, for a description of HLL.
|
HOQ Life
|
Maximum duration allowed to packets at the head of a virtual-lane queue. Used with VL Stall Count to determine the outgoing packets to discard.
|
Oper VL
|
Administrative limit for the number of virtual lanes allowed to the link. Do not set this above the Virtual Lanes Cap value. The value appears as vl0, vl0-Vl1, vl0-Vl3, vl0-Vl7, or vl0-Vl14.
|
In Partition Enforcement
|
Boolean value that indicates whether or not to support optional partition enforcement for the packets that were received by this port. No default value applies.
|
Out Partition Enforcement
|
Boolean value that indicates whether or not to support optional partition enforcement for the packets transmitted by this port. No default value applies.
|
In Filter RawPacket Enforcement
|
Boolean value that indicates whether or not to support optional raw packet enforcement for the raw packets that were received by this port. No default value applies.
|
Out Filter RawPacket Enforcement
|
Boolean value that indicates whether or not to support optional raw packet enforcement for the raw packets transmitted by this port. No default value applies.
|
MKeyViolation
|
Number of subnet management packets (SMPs) that have been received on this port with invalid M_Keys since initial power up or the last reset. See section 14.2.4, "Management Key," InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
PKeyViolation
|
Number of subnet management packets that have been received on this port with invalid P_Keys since initial power up or the last reset. See section 9.2.7, "Partition Key (P_KEY)," InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
QKeyViolation
|
Number of subnet management packets that have been received on this port with invalid Q_Keys since initial power up or the last reset. See section 10.2.4, "Q Keys," InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
GUID Cap
|
Number of GUID entries allowed for this port in the port table. Any entries that exceed this value are ignored on write and read back as zero. See section 14.2.5.5, "GUIDCap," InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
Subnet Timeout
|
Maximum propagation delay allowed for this port to reach any other port in the subnet. This value also affects the maximum rate at with traps can be sent from this port. Switch configuration affects delay. Requestors can use this parameter to determine the interval to wait for a response to a request. Duration matches (4.096 ms * 2^SubnetTimeout).
|
Response Time
|
Maximum time allowed between the port reception of a subnet management packet and the transmission of the associated response. See section 13.4.6.2, "Timers and Timeouts," InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
Local Phys Error
|
Threshold at which ICRC, VCRC, FCCRC, and all physical errors result in an entry into the BAD PACKET or BAD PACKET DISCARD states of the local packet receiver. See section 7.12.2, "Error Recovery Procedures," InfiniBand Architecture®, Vol. 1, Release 1.1, for more information.
|
Local Overrun Error
|
Threshold at which the count of buffer overruns, across consecutive flow-control update periods, results in an overrun error. A possible cause of such errors is when an earlier packet has physical errors and the buffers are not immediately reclaimed.
|
Routing Around Nodes and Ports
These topics describe how to route around nodes and ports using the Nodes displays:
•Routing Around Nodes
•Routing Around Ports
For a complete discussion of routing around components, including routing around chassis, see the "Routing Around Components in an InfiniBand Network" section.
Routing Around Nodes
To route around a node or to re-include a node that had previously been excluded, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Expand the Subnet Manager that manages the node you want to work on.
Step 3 Expand Nodes.
A list of nodes managed by the subnet manager appears in the left pane.
Step 4 Select the node you want to exclude from routing calculations or include in routing calculations.
Step 5 Right-click on the highlighted node.
Step 6 Select Start Routing Around or Stop Routing Around.
Routing Around Ports
To route around a port, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens. Each Subnet Manager appears in the navigation pane with a Subnet Manager icon (
).
Step 2 Expand the Subnet Manager that you want to view.
Step 3 Expand Nodes.
A list of nodes managed by the subnet manager appears in the left pane.
Step 4 Expand the node containing the port you want to work on.
Step 5 Select the port you want to exclude from routing calculations or to include in routing calculations.
Step 6 Right-click on the highlighted port.
Step 7 Select Start Routing Around or Stop Routing Around.
Viewing and Managing Partitions
This section provides procedures for performing the following tasks:
•Viewing Partitions
•Creating a Partition
•Removing a Partition
•Enabling or Disabling IPoIB for a Partition
•Viewing Partition Details
•Adding Full Members to a Partition
•Adding Limited Members to a Partition
Viewing Partitions
To view the partitions on your InfiniBand network, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with partitions that you want to view.
The navigation menu expands.
Step 3 Click the Partitions (
) branch.
The partitions summary appears in the right pane. Table 8-5 describes the fields in this pane.
Table 8-5 Partitions Summary Field Descriptions
Field
|
Description
|
Partition Key
|
Partition key (numeric identifier) of the partition.
|
Full Member Count
|
Number of full partition members.
|
Limited Member Count
|
Number of limited partition members.
|
IPoIB
|
Specifies whether IPoIB is enabled for this partition.
|
Creating a Partition
To create an InfiniBand partition, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager under which you want to create a partition.
Step 3 Select the Partitions (
) branch.
Step 4 Click Add.
The Add Partition window opens.
Step 5 In the PKey field, enter a partition key for the new partition.
Step 6 Check the IPoIB check box to enable or uncheck to disable IPoIB for the partition.
Step 7 Click OK.
Removing a Partition
To delete a partition, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with partition that you want to delete.
Step 3 Expand the Partitions (
) branch.
Step 4 Click the partition in the Summary display that you want to remove, and then click Remove.
Enabling or Disabling IPoIB for a Partition
Disabling IPoIB removes all current multicast group members and prevents further multicast joins. To enable or disable IPoIB on a partition, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with the partitions for which you want to enable or disable IPoIB.
The navigation menu expands.
Step 3 Click the Partitions (
) branch.
The partitions summary appears in the right pane.
Step 4 Click on the summary line of the partition for which you want to enable or disable IPoIB.
Step 5 Click Edit.
The Add Partition window opens.
Step 6 Check the IPoIB check box to check (enable) or uncheck (disable) IPoIB for the partition.
Step 7 Click OK.
Viewing Partition Details
To view partition details, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with partitions that you want to view.
Step 3 Expand the Partitions (
) branch to display all partitions.
Step 4 Click the partition key of the partition with details that you want to view.
The members (full and limited) of the partition appear in the display.
Note To view the GUIDs of the Server Switch management ports in the display, click Show Switch Mgmt Ports. Click Hide Switch Mgmt Ports to remove these GUIDs from the display.
Adding Full Members to a Partition
Full members of a partition can communicate to other full members and to limited members.
These topics describe how to add full members to a partition:
•Adding Available Members to a Partition
•Adding Unavailable Members to a Partition
Adding Available Members to a Partition
To add available members to a partition, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with the partition to which you want to add a member.
Step 3 Expand the Partitions (
) branch to display all partitions in the navigation menu.
Step 4 Select the partition key of the partition to which you want to add members.
The members (full and limited) of the partition appear in the display.
Step 5 In the Available Members field, click the port that you want to add to the partition, and then click the right arrow next to the Full Members field.
Adding Unavailable Members to a Partition
To add unavailable members (members that do not appear in the Available Members pool) to a partition, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with the partitions to which you want to add a member.
Step 3 Expand the Partitions (
) branch to display all partitions in the navigation menu.
Step 4 Click the partition key of the partition to which you want to add members.
The members (full and limited) of the partition appear in the display.
Step 5 Click Add Other.
The Add Other Partition Member window opens.
Step 6 In the Node GUID field, enter the GUID of the host that includes the port(s) that you want to add to the partition.
Step 7 In the Port field, specify the port(s) that you want to add to the partition.
Step 8 Click the Full radio button, and then click Add.
Adding Limited Members to a Partition
Limited members of a partition can communicate with full members of the partition but not with other limited members.
These topics describe how to add limited members to a partition:
•Adding Available Limited Members
•Adding Unavailable Members
Adding Available Limited Members
To add available limited members to a partition, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with the partition to which you want to add a member.
The navigation menu expands.
Step 3 Expand the Partitions (
) branch to display all partitions in the navigation menu.
Step 4 Click the partition key of the partition to which you want to add members.
The members (full and limited) of the partition appear in the display.
Step 5 In the Available Members field, click the port that you want to add to the partition, and then click the right arrow next to the Limited Members field.
Adding Unavailable Members
To add an unavailable member (member does not appear in the Available Members pool) to a partition, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with the partition to which you want to add a member.
Step 3 Expand the Partitions (
) branch to display all partitions in the navigation menu.
Step 4 Select the partition key of the partition to which you want to add members.
The members (full and limited) of the partition appear in the display.
Step 5 Click Add Other.
The Add Other Partition Member window opens.
Step 6 In the Node GUID field, enter the GUID of the node that includes the port(s) that you want to add to the partition.
Step 7 In the Port field, specify the port(s) that you want to add to the partition.
Step 8 Click the Limited radio button, and then click Add.
Viewing and Managing Multicast Groups
This section provides procedures for performing the following tasks:
•Viewing Multicast Groups
•Viewing Multicast Group Details
•Viewing Multicast Group Members
•Configuring Multicast Groups
•Configuring IPoIB Broadcast Multicast Groups
Viewing Multicast Groups
To view the multicast groups on your InfiniBand network, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with partitions that you want to view.
The navigation menu expands.
Step 3 Select the Multicast Groups (
) branch.
The multicast groups summary appears in the right pane. Table 8-6 describes the fields in this pane.
Table 8-6 Multicast Group Summary Field Descriptions
Field
|
Description
|
MGID
|
Numeric multicast group identifier of each multicast group on the InfiniBand fabric.
|
QKey
|
16-bit Q-Key of this multicast group.
|
MTU
|
Maximum transmission unit of the multicast group.
|
PKey
|
Partition key of the multicast group.
|
Viewing Multicast Group Details
To view multicast group details, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with multicast groups that you want to view.
The navigation menu expands.
Step 3 Expand the Multicast Groups (
) branch to display all groups in the navigation menu.
Step 4 Click the MGID of the multicast group with details that you want to view, and then click the General tab.
Multicast group details appear in the display. Table 8-7 describes the fields in this display.
Table 8-7 Multicast Group General Details Field Descriptions
Field
|
Description
|
QKey
|
16-bit Q-Key of this multicast group.
|
MLID
|
16-bit LID of this multicast group
|
MTU
|
Maximum transmission unit of the multicast group.
|
TClass
|
Traffic class for the multicast group.
|
PKey
|
16-bit Partition Key for this multicast group.
|
Rate
|
Traffic rate of this multicast group.
|
Packet Life Time
|
Maximum estimated time for a packet to traverse a path within the multicast group.
|
SL
|
Service level of this multicast group.
|
Flow Label
|
Flow label used for this multicast group.
|
Hop Limit
|
Identifies the maximum number of hops a packet can take before being discarded.
|
Scope
|
Scope of this multicast group.
|
User Configured
|
Displays true if a user configured the entry; otherwise displays false.
|
Viewing Multicast Group Members
To view multicast group members, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with multicast groups that you want to view.
The navigation menu expands.
Step 3 Expand the Multicast Groups (
) branch to display all groups in the navigation menu.
Step 4 Click the MGID of the multicast group with details that you want to view.
Multicast group members appear in a table at the bottom of the display. Table 8-8 describes the fields in this display.
Table 8-8 Multicast Group Members Field Descriptions
Field
|
Description
|
Port GID
|
Global identifier of a port that belongs to the multicast group.
|
Join State
|
Displays whether the port is a full member or limited member of the group.
|
Proxy Join Status
|
This field displays false except for trusted requests. For details, see InfiniBand Architecture®, Vol. 1, Release 1.1.
|
Configuring Multicast Groups
To configure multicast groups, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Manager window opens.
Step 2 Expand a subnet.
Step 3 Select Multicast Groups.
Step 4 Click Add.
Step 5 From the drop-down list, select MGID.
Step 6 In the Multicast Group ID field, enter an MGID.
Step 7 (Optional) In the QKey field, enter a queue key.
Step 8 In the MTU field, select a value to configure the maximum transmission unit of the group.
Step 9 In the PKey field, enter a partition key.
Step 10 In the Rate field, select a rate.
Step 11 In the Service Level field, enter an integer value (between 0 and 15).
Step 12 Click Add.
Note The TClass, Packet Lifetime, Flow Label, and Hop Limit attributes are not supported in this release.
Configuring IPoIB Broadcast Multicast Groups
To configure IPoIB broadcast multicast groups, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Manager window opens.
Step 2 Expand a subnet.
Step 3 Select Multicast Groups.
Step 4 Click Add.
Step 5 From the drop-down list, choose IPoIB.
Step 6 (Optional) In the QKey field, enter a queue key.
Step 7 From the drop-down list, select an MTU value.
Step 8 Enter a partition key in the PKey field.
Step 9 From the Rate field, select a data rate.
Step 10 In the Service Level field, enter an integer value (between 0 and 15).
Step 11 In the Scope field, choose a scope value.
Step 12 Click Add.
Note The TClass, Packet Lifetime, Flow Label, and Hop Limit attributes are not included in this release.
Viewing InfiniBand Services
To view the InfiniBand services that run on your server switch, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with services that you want to view.
Step 3 Click the Services (
) branch.
Details of InfiniBand services appear in the right pane. Table 8-9 describes the fields in the Summary section of the pane.
Table 8-9 Services Summary Field Descriptions Pane
Field
|
Description
|
Name
|
ASCII identifier of the service.
|
Service Id
|
Numeric identifier that nodes use to call the service.
|
Service GID
|
64-bit ID of the service.
|
PKey
|
16-bit multicast GID address.
|
Table 8-10 describes the fields in the Services Details section of the display.
Table 8-10 Services Details Field Descriptions
Field
|
Description
|
Service Name
|
ASCII identifier of the service.
|
Service Id
|
Numeric identifier that nodes use to call the service.
|
Service GID
|
GID of the node that provides the service.
|
Service PKey
|
16-bit P-Key.
|
Lease
|
Lease period remaining (in seconds) for this service. A value of 4294967295 means that the lease is indefinite.
|
Key
|
128-bit service key.
|
Data (8 bit)
|
Header of the data type 8.
|
Data (16 bit)
|
Header of the data type 16.
|
Data (32 bit)
|
Header of the data type 32.
|
Data (64 bit)
|
Header of the data type 64.
|
Viewing and Managing InfiniBand Routes
This section provides procedures for performing the following tasks:
•Viewing InfiniBand Routes
•Routing Around Components in an InfiniBand Network
•Removing Routes from the Route-Around Table
Viewing InfiniBand Routes
To view the route between a pair of LIDs in the InfiniBand fabric, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with services that you want to view.
Step 3 Select the Routes (
) branch.
Step 4 Select the Route Filter tab
InfiniBand routes fields appear in the right pane.
Step 5 In the Source LID field, enter the source LID of the route.
Step 6 In the Destination LID field, enter the destination lid of the route.
Step 7 Click Show Route.
Step 8 Click the Switch Route tab.
Table 8-11 describes the fields under the Switch Route tab.
Table 8-11 Switch Route Field Descriptions
Field
|
Description
|
Node GUID
|
Global unique ID of the node.
|
In Port
|
Port of ingress.
|
Out Port
|
Port of egress.
|
Step 9 Click the Switch Element Route tab.
Table 8-12 describes the fields under the Switch Element Route tab.
Table 8-12 Switch Element Route Field Descriptions
Field
|
Description
|
Chassis GUID
|
Global unique ID of the node.
|
In Port
|
Port of ingress.
|
Out Port
|
Port of egress.
|
Routing Around Components in an InfiniBand Network
To route around a chassis, nodes, or ports that are accumulating errors or to route around a component that you want to remove, follow the steps outlines in the subsections that follow.
Uses of this feature include the following:
•Isolating ports that have accumulated errors to avoid a potential job failure. The route-around feature enables you to stop traffic from passing over a link while a job is still running, without disrupting the job.
•Isolating a specific component, such as an InfiniBand switch card, allowing that component to be removed without the potential for job failure. You might do this, for example, before component upgrade or other replacement.
Caution The route-around feature has the potential to exclude any chassis, node, or port from routing calculations to the extent that it is possible to disable entirely a connection between a pair of endpoints. Use care to avoid segmenting the InfiniBand fabric when using this feature.
Note You can also route around nodes or ports (but not chassis) from the Nodes table as described in the "Routing Around Nodes and Ports" section.
These topics describe how to route around components in an InfiniBand network:
•Viewing Route-Around Information
•Adding Routes to the Route-Around Table
•Removing Routes from the Route-Around Table
Viewing Route-Around Information
To view active route-around operations, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with services that you want to view.
Step 3 Select the Routes (
) branch.
Step 4 Select the Route Around tab.
Excluded routes appear in the right pane. Table 8-13 describes the fields in the pane.
Table 8-13 Route Around Tab Field Descriptions Pane
Field
|
Description
|
Type
|
Type of component excluded. Possible values are port, node, and chassis.
|
GUID
|
GUID of the excluded node, chassis, or node of the excluded port.
|
Port Number
|
Excluded port number.
|
Adding Routes to the Route-Around Table
To add a component to the route-around table, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with the routing information you want to change.
Step 3 Select the Routes (
) branch.
Step 4 Select the Route Around tab.
Step 5 Click Add.
A Route Around dialog box appears.
Step 6 In the Add Route Around dialogue box, define the route you want to exclude from routing calculations:
a. In the Type drop-down menu, select the Port, Node, or Chassis to be excluded.
b. In the GUID field, enter the GUID of the node or chassis.
c. In the Port Number field, specify the port number if you selected Port from the Type drop-down menu.
Step 7 Click Add.
Removing Routes from the Route-Around Table
To remove a route-around from the table, follow these steps.
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with the routing information you want to change.
Step 3 Select the Routes (
) branch.
Step 4 Select the Route-Around tab.
Step 5 Select the route-around that you want to remove from the table.
Step 6 Click Remove.
Viewing Other Subnet Managers Information
To view information on other Subnet Managers in the network, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with neighbor subnets managers that you want to view.
The navigation menu expands.
Step 3 Expand Subnet Managers Info.
The Port GUID, Priority, and Subnet Manager state information appears in the right pane.
Table 8-14 describes the fields in the Details pane.
Table 8-14 Subnet Managers Information Details Pane
Field
|
Description
|
Port GUID
|
Displays the port GUID of the networking device on which the Subnet Manager runs.
|
SM Key
|
64-bit subnet management key assigned to the Subnet Manager. The Subnet Manager key serves as the prefix of all GIDs and brands nodes as members of this subnet.
|
Activity Count
|
Activity counter that increments each time the Subnet Manager issues a subnet management packet (SMP) or that performs other management activities.
|
Priority
|
Priority of the Subnet Manager relative to other Subnet Managers in the network. The number 15 has the highest priority.
|
SM State
|
State of the Subnet Manager.
|
Note This menu provides information on subnet managers that are not local to the chassis to which an Element Manager is connected.
Viewing Event Subscriptions
To view the Subnet Management event subscriptions information, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
The Subnet Management window opens.
Step 2 Expand the Subnet Manager with event subscriptions that you want to view.
The navigation menu expands.
Step 3 Choose Event Subscriptions.
The LID, Node GUID, and Port Number information appears in the right pane.
Table 8-15 describes the fields under Subnet Management Event Subscriptions Details.
Table 8-15 Subnet Management Event Subscriptions Details Pane
Field
|
Description
|
LID
|
Local ID of the subscriber.
|
Node GUID
|
Global unique ID of the subscriber node.
|
Port Number
|
Port number of the subscriber.
|
Source QPN
|
24-bit source queue pair number of the subscriber.
|
GID
|
Global ID.
|
LID Range Start
|
Lowest legal Local ID number.
|
LID Range End
|
Highest legal Local ID number.
|
Is Generic
|
If `true," forward all generic traps. If "false," forward all vendor-specific traps.
|
Type
|
Type of trap for which you subscribed.
|
Trap Number Device ID
|
If generic, this is the trap number for which you subscribed. If not generic, this is the device ID for which you subscribed. 0xFFFF means forward all trap numbers/device IDs.
|
Response Time Value
|
Response Time Value of the subscriber.
|
Producer Type Vendor ID
|
If not generic, this is the 24-bit IEEE OUI assigned to the vendor.
|
Viewing Forwarding Tables
This section provides procedures for performing the following tasks:
•Viewing Multicast Forwarding Information
•Viewing Linear Forwarding Information
Viewing Multicast Forwarding Information
To view the multicast forwarding configuration, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
Step 2 Click the MulticastForwardings tab.
Table 8-16 describes the information that appears.
Table 8-16 Multicast Forwarding Entries
Field
|
Description
|
Node Guid
|
GUID of the switch node in the subnet with the FDB that you want to access.
|
MLID
|
Local ID of the multicast group.
|
Port Mask 0
|
Shows to which ports a multicast packet for the given LID will be transmitted.
|
Port Mask 1
|
Port mask.
|
Viewing Linear Forwarding Information
To view the linear forwarding configuration, follow these steps:
Step 1 From the InfiniBand menu, choose Subnet Management.
Step 2 Click the LinearForwardings tab.
Table 8-17 describes the displayed fields.
Table 8-17
Field
|
Description
|
Node Guid
|
GUID of the switch node in the subnet with the FDB that you want to access.
|
LID
|
Local ID.
|
Port Number
|
Port number of the port through which the given LID will be forwarded.
|
Linear Forwarding Entries
Feedback
