-
Cisco MDS 9000 Family NX-OS System Management Configuration Guide
-
Index
-
New and Changed Information
-
Preface
-
Overview
-
Using the CFS Infrastructure
-
Configuring System Message Logging
-
Configuring Call Home
-
Scheduling Maintenance Jobs
-
Monitoring System Processes and Log
-
Configuring the Embedded Event Manager
-
Configuring SNMP
-
Configuring RMON
-
Configuring Domain Parameters
-
Configuring Fabric Configuration Servers
-
Monitoring Network Traffic Using SPAN
-
Feedback
Table Of Contents
Information About Fibre Channel Domains
Autoreconfiguring Merged Fabrics
Specifying Static or Preferred Domain IDs
CFS Distribution of Allowed Domain ID Lists
Contiguous Domain ID Assignments
Persistent FC ID Configuration
About Unique Area FC IDs for HBAs
Persistent FC ID Selective Purging
Configuring Fibre Channel Domains
Enabling Domain Manager Fast Restart
Enabling or Disabling fcdomains
Specifying Static or Preferred Domain IDs
Configuring Allowed Domain ID Lists
Enabling Allowed Domain ID Distribution
Enabling Contiguous Domain ID Assignments
Enabling the Persistent FC ID Feature
Configuring Unique Area FC IDs for an HBA
Verifying FC Domain Configuration
Displaying CFS Distribution Status
Displaying fcdomain Information
Feature History for Domain Parameters
Configuring Domain Parameters
The Fibre Channel domain (fcdomain) feature performs principal switch selection, domain ID distribution, FC ID allocation, and fabric reconfiguration functions as described in the FC-SW-2 standards.
This chapter includes the following sections:
•
Information About Fibre Channel Domains
•
•
•
•
Information About Fibre Channel Domains
The Fibre Channel domain (fcdomain) feature performs principal switch selection, domain ID distribution, FC ID allocation, and fabric reconfiguration functions as described in the FC-SW-2 standards. The domains are configured on a per VSAN basis. If you do not configure a domain ID, the local switch uses a random ID.
This section describes each fcdomain phase:
•
•
•
•
Caution
Figure 10-1 shows a sample fcdomain configuration.
Figure 10-1 Sample fcdomain Configuration
This section includes the following topics:
•
Domain Restart
Fibre Channel domains can be started disruptively or nondisruptively. If you perform a disruptive restart, reconfigure fabric (RCF) frames are sent to other switches in the fabric and data traffic is disrupted on all the switches in the VSAN (including remotely segmented ISLs). If you perform a nondisruptive restart, build fabric (BF) frames are sent to other switches in the fabric and data traffic is disrupted only on the switch.
If you are attempting to resolve a domain ID conflict, you must manually assign domain IDs. A disruptive restart is required to apply most configuration changes, including manually assigned domain IDs. Nondisruptive domain restarts are acceptable only when changing a preferred domain ID into a static one (and the actual domain ID remains the same).
Note
Tip
You can apply most of the configurations to their corresponding runtime values. Each of the following sections provide further details on how the fcdomain parameters are applied to the runtime values.
The fcdomain restart command applies your changes to the runtime settings. Use the disruptive option to apply most of the configurations to their corresponding runtime values, including preferred domain IDs (see the "Domain IDs" section).
Domain Manager Fast Restart
As of Cisco MDS SAN-OS Release 3.0(2), when a principal link fails, the domain manager must select a new principal link. By default, the domain manager starts a build fabric (BF) phase, followed by a principal switch selection phase. Both of these phases involve all the switches in the VSAN and together take at least 15 seconds to complete. To reduce the time required for the domain manager to select a new principal link, you can enable the domain manager fast restart feature.
When fast restart is enabled and a backup link is available, the domain manager needs only a few milliseconds to select a new principal link to replace the one that failed. Also, the reconfiguration required to select the new principal link only affects the two switches that are directly attached to the failed link, not the entire VSAN. When a backup link is not available, the domain manager reverts to the default behavior and starts a BF phase, followed by a principal switch selection phase. The fast restart feature can be used in any interoperability mode.
Tip
Switch Priority
Any new switch can become the principal switch when it joins a stable fabric. During the principal switch selection phase, the switch with the highest priority becomes the principal switch. If two switches have the same configured priority, the switch with the lower WWN becomes the principal switch.
The priority configuration is applied to runtime when the fcdomain is restarted (see the "Domain Restart" section). This configuration is applicable to both disruptive and nondisruptive restarts.
fcdomain Initiation
By default, the fcdomain feature is enabled on each switch. If you disable the fcdomain feature in a switch, that switch can no longer participate with other switches in the fabric. The fcdomain configuration is applied to runtime through a disruptive restart.
Incoming RCFs
You can configure the rcf-reject option on a per-interface, per-VSAN basis. By default, the rcf-reject option is disabled (that is, RCF request frames are not automatically rejected).
The rcf-reject option takes effect immediately. No fcdomain restart is required.
Autoreconfiguring Merged Fabrics
By default, the autoreconfigure option is disabled. When you join two switches belonging to two different stable fabrics that have overlapping domains, the following cases apply:
•
•
The autoreconfigure option takes immediate effect at runtime. You do not need to restart the fcdomain. If a domain is currently isolated due to domain overlap, and you later enable the autoreconfigure option on both switches, the fabric continues to be isolated. If you enabled the autoreconfigure option on both switches before connecting the fabric, a disruptive reconfiguration (RCF) will occur. A disruptive reconfiguration may affect data traffic. You can nondisruptively reconfigure the fcdomain by changing the configured domains on the overlapping links and eliminating the domain overlap.
Domain IDs
Domain IDs uniquely identify a switch in a VSAN. A switch may have different domain IDs in different VSANs. The domain ID is part of the overall FC ID.
The configured domain ID can be preferred or static. By default, the configured domain ID is 0 (zero) and the configured type is preferred.
Note
If you do not configure a domain ID, the local switch sends a random ID in its request. We recommend that you use static domain IDs.
When a subordinate switch requests a domain, the following process takes place (see Figure 10-2):
1.
2.
Figure 10-2 Configuration Process Using the preferred Option
The behavior for a subordinate switch changes based on three factors:
•
•
•
In specific situations, the changes are as follows:
•
•
•
–
–
If you change the configured domain ID, the change is only accepted if the new domain ID is included in all the allowed domain ID lists currently configured in the VSAN. Alternatively, you can also configure zero-preferred domain ID.
Tip
Note
In an IVR NAT configuration, if one VSAN in the IVR topology is configured with static domain IDs, then the IVR domains that can be exported to that VSAN must also be assigned static domains.
Caution
Note
Specifying Static or Preferred Domain IDs
When you assign a static domain ID type, you are requesting a particular domain ID. If the switch does not get the requested address, it will isolate itself from the fabric. When you specify a preferred domain ID, you are also requesting a particular domain ID; however, if the requested domain ID is unavailable, then the switch will accept another domain ID.
While the static option can be applied at runtime after a disruptive or nondisruptive restart, the preferred option is applied at runtime only after a disruptive restart (see the "Domain Restart" section).
Allowed Domain ID Lists
By default, the valid range for an assigned domain ID list is from 1 to 239. You can specify a list of ranges to be in the allowed domain ID list and separate each range with a comma. The principal switch assigns domain IDs that are available in the locally configured allowed domain list.
Use allowed domain ID lists to design your VSANs with non-overlapping domain IDs. This helps you in the future if you need to implement IVR without the NAT feature.
CFS Distribution of Allowed Domain ID Lists
You can enable the distribution of the allowed domain ID lists configuration information to all Cisco MDS switches in the fabric using the Cisco Fabric Services (CFS) infrastructure. This feature allows you to synchronize the configuration across the fabric from the console of a single MDS switch. Since the same configuration is distributed to the entire VSAN, you avoid possible misconfiguration and the likelihood that two switches in the same VSAN have configured incompatible allowed domains.
Use CFS to distribute the allowed domain ID list to ensure consistency in the allowed domain ID lists on all switches in the VSAN.
Note
For more information about CFS, see Chapter 2 "Using the CFS Infrastructure".
Contiguous Domain ID Assignments
By default, the contiguous domain assignment is disabled. When the subordinate switches request the principal switch for two or more domains and the domains are not contiguous, the following cases apply:
•
•
Locking the Fabric
The first action that modifies the existing configuration creates the pending configuration and locks the feature in the fabric. Once you lock the fabric, the following conditions apply:
•
•
Committing Changes
To apply the pending domain configuration changes to other MDS switches in the VSAN, you must commit the changes. The pending configuration changes are distributed and, on a successful commit, the configuration changes are applied to the active configuration in the MDS switches throughout the VSAN and the fabric lock is released.
Clearing a Fabric Lock
If you have performed a domain configuration task and have not released the lock by either committing or discarding the changes, an administrator can release the lock from any switch in the fabric. If the administrator performs this task, your pending changes are discarded and the fabric lock is released.
The pending changes are only available in the volatile directory and are discarded if the switch is restarted.
FC IDs
When an N or NL port logs into a Cisco MDS 9000 Family switch, it is assigned an FC ID. By default, the persistent FC ID feature is enabled. If this feature is disabled, the following consequences apply:
•
•
•
•
–
–
Persistent FC IDs
When persistent FC IDs are enabled, the following consequences apply:
•
•
Persistent FC ID Configuration
When the persistent FC ID feature is enabled, you can enter the persistent FC ID submode and add static or dynamic entries in the FC ID database. By default, all added entries are static. Persistent FC IDs are configured on a per-VSAN basis. Follow these requirements to manually configure a persistent FC ID:
•
•
•
•
Note
About Unique Area FC IDs for HBAs
Note
Some HBA ports require a different area ID than storage ports when they are both connected to the same switch. For example, if the storage port FC ID is 0x6f7704, the area for this port is 77. In this case, the HBA port's area can be anything other than 77. The HBA port's FC ID must be manually configured to be different from the storage port's FC ID.
Switches in the Cisco MDS 9000 Family facilitate this requirement with the FC ID persistence feature. You can use this feature to preassign an FC ID with a different area to either the storage port or the HBA port.
Persistent FC ID Selective Purging
Persistent FC IDs can be purged selectively. Static entries and FC IDs currently in use cannot be deleted. Table 10-1 identifies the FC ID entries that are deleted or retained when persistent FC IDs are purged.
Table 10-1 Purged FC IDs
Static
In use
Not deleted
Static
Not in use
Not deleted
Dynamic
In use
Not deleted
Dynamic
Not in use
Deleted
Guidelines and Limitations
•
•
Default Settings
Table 10-2 lists the default settings for all fcdomain parameters.
Configuring Fibre Channel Domains
This section describes the fcdomain feature and includes the following topics:
•
•
Restarting a Domain
Domain Configuration Scenarios:
Switch Configuration
Irrespective of how the switches in VSAN 6 are configured, fcdomain restart disruptive vsan 6 causes all devices of all switches in VSAN 6 to log out, causing data traffic disruption.
Configured domain and the runtime domain are the same
Assuming that the configured domain and the runtime domain are the same on all switches, fcdomain restart vsan 6 does not cause any devices in VSAN 6 to log out.
Configured domain and runtime domain are not the same
Assuming that on some switches in VSAN 6 the configured domain and the runtime domain are not the same, fcdomain restart vsan 6 causes the devices in VSAN 6 attached to the switches whose statically configured and runtime domain differ to log out, causing data traffic disruption.
Detailed Steps
To restart the fabric disruptively or nondisruptively, follow these steps:
Enabling Domain Manager Fast Restart
Detailed Steps
To enable the domain manager fast restart feature in Cisco SAN-OS Release 3.0(2) or later, or MDS NX-OS Release 4.1(1a) or later, follow these steps:
Configuring Switch Priority
Restrictions
•
Detailed Steps
To configure the priority for the principal switch, follow these steps:
Enabling or Disabling fcdomains
Detailed Steps
To disable or reenable fcdomains in a single VSAN or a range of VSANs, follow these steps:
Configuring Fabric Names
Detailed Steps
To set the fabric name value for a disabled fcdomain, follow these steps:
Rejecting Incoming RCFs
Detailed Steps
To reject incoming RCF request frames, follow these steps:
Enabling Autoreconfiguration
Detailed Steps
To enable automatic reconfiguration in a specific VSAN (or range of VSANs), follow these steps:
Configuring Domain IDs
Domain IDs uniquely identify a switch in a VSAN. A switch may have different domain IDs in different VSANs. The domain ID is part of the overall FC ID.
The configured domain ID can be preferred or static. By default, the configured domain ID is 0 (zero) and the configured type is preferred.
This section includes the following topics:
•
•
•
•
Specifying Static or Preferred Domain IDs
Restrictions
•
Note
Detailed Steps
To specify a static or preferred domain ID, follow these steps:
Configuring Allowed Domain ID Lists
Prerequisites
An allowed domain ID list must satisfy the following conditions:
•
•
•
•
If you configure an allowed list on one switch in the fabric, we recommend that you configure the same list in all other switches in the fabric to ensure consistency or use CFS to distribute the configuration.
Detailed Steps
To configure the allowed domain ID list, follow these steps:
Enabling Allowed Domain ID Distribution
CFS distribution of allowed domain ID lists is disabled by default. You must enable distribution on all switches to which you want to distribute the allowed domain ID lists.
Prerequisites
•
Detailed Steps
To enable (or disable) allowed domain ID list configuration distribution, follow these steps:
Committing Changes
Detailed Steps
To commit pending domain configuration changes and release the lock, follow these steps:
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# fcdomain commit vsan 10
Commits the pending domain configuration changes.
Discarding Changes
At any time, you can discard the pending changes to the domain configuration and release the fabric lock. If you discard (abort) the pending changes, the configuration remains unaffected and the lock is released.
Detailed Steps
To discard pending domain configuration changes and release the lock, follow these steps:
Step 1
switch# config t
switch(config)#
Enters configuration mode.
Step 2
switch(config)# fcdomain abort vsan 10
Discards the pending domain configuration changes.
Enabling Contiguous Domain ID Assignments
Detailed Steps
To enable contiguous domains in a specific VSAN (or a range of VSANs), follow these steps:
Configuring FC IDs
When an N or NL port logs into a Cisco MDS 9000 Family switch, it is assigned an FC ID.
This section includes the following topics:
•
•
•
Enabling the Persistent FC ID Feature
If you connect to the switch from an AIX or HP-UX host, be sure to enable the persistent FC ID feature in the VSAN that connects these hosts.
A persistent FC ID assigned to an F port can be moved across interfaces and can continue to maintain the same persistent FC ID.
Restrictions
•
•
•
Note
vsan 1000 wwn 33:e8:00:05:30:00:16:df fcid 0x070128 and should be changed to vsan 1000 wwn 33:e8:00:05:30:00:16:df fcid 0x070100.
Detailed Steps
To enable the persistent FC ID feature, follow these steps:
Configuring Persistent FC IDs
Detailed Steps
To configure persistent FC IDs, follow these steps:
Configuring Unique Area FC IDs for an HBA
Detailed Steps
To configure a different area ID for the HBA port, follow these steps:
Note
Step 1
switch# show flogi database------------------------------------------------------------------------------------------INTERFACE VSAN FCID PORT NAME NODE NAME------------------------------------------------------------------------------------------fc1/9 3 0x6f7703 50:05:08:b2:00:71:c8:c2 50:05:08:b2:00:71:c8:c0fc1/10 3 0x6f7704 50:06:0e:80:03:29:61:0f 50:06:0e:80:03:29:61:0f
Note
Step 2
switch# conf tswitch(config)# interface fc1/9switch(config-if)# shutdownswitch(config-if)# endswitch#Step 3
switch# show fcdomain vsan 1...Local switch configuration information:State: EnabledFCID persistence: DisabledIf this feature is disabled, continue with this procedure to enable the persistent FC ID.
If this feature is already enabled, skip to Step 7.
Step 4
switch# conf tswitch(config)# fcdomain fcid persistent vsan 1switch(config)# endswitch#Step 5
switch# conf tswitch(config)# fcdomain fcid databaseswitch(config-fcid-db)# vsan 3 wwn 50:05:08:b2:00:71:c8:c2 fcid 0x6fee00 areaStep 6
switch# conf tswitch(config)# interface fc1/9switch(config-if)# no shutdownswitch(config-if)# endswitch#Step 7
switch# show flogi database------------------------------------------------------------------------------------------INTERFACE VSAN FCID PORT NAME NODE NAME------------------------------------------------------------------------------------------fc1/9 3 0x6fee00 50:05:08:b2:00:71:c8:c2 50:05:08:b2:00:71:c8:c0fc1/10 3 0x6f7704 50:06:0e:80:03:29:61:0f 50:06:0e:80:03:29:61:0f
Note
Purging Persistent FC IDs
Detailed Steps
To purge persistent FC IDs, follow this step:
Step 1
switch# purge fcdomain fcid vsan 4
Purges all dynamic and unused FC IDs in VSAN 4.
switch# purge fcdomain fcid vsan 3-5
Purges dynamic and unused FC IDs in VSAN 3, 4, and 5.
Clearing a Fabric Lock
To release a fabric lock, issue the clear fcdomain session vsan command in EXEC mode using a login ID that has administrative privileges.
switch# clear fcdomain session vsan 10Verifying FC Domain Configuration
To display the domain ID configuration information, perform the following tasks:
For detailed information about the fields in the output from these commands, refer to the Cisco MDS 9000 Family Command Reference.
This section includes the following topics:
•
•
Displaying CFS Distribution Status
You can display the status of CFS distribution for allowed domain ID lists using the show fcdomain status command.
switch# show fcdomain statusCFS distribution is enabledDisplaying Pending Changes
You can display the pending configuration changes using the show fcdomain pending command:
switch# show fcdomain pending vsan 10Pending Configured Allowed Domains----------------------------------VSAN 10Assigned or unallowed domain IDs: 1-9,24,100,231-239.[User] configured allowed domain IDs: 10-230.You can display the differences between the pending configuration and the current configuration using the show fcdomain pending-diff command.
switch# show fcdomain pending-diff vsan 10Current Configured Allowed Domains----------------------------------VSAN 10Assigned or unallowed domain IDs: 24,100.[User] configured allowed domain IDs: 1-239.Pending Configured Allowed Domains----------------------------------VSAN 10Assigned or unallowed domain IDs: 1-9,24,100,231-239.[User] configured allowed domain IDs: 10-230.Displaying Session Status
You can display the status of the distribution session using the show fcdomain session-status vsan command.
switch# show fcdomain session-status vsan 1Last Action: Distribution EnableResult: SuccessDisplaying fcdomain Information
Use the show fcdomain command to display global information about fcdomain configurations. See Example 10-1.
Note
Example 10-1 Displays the Global fcdomain Information
switch# show fcdomain vsan 2The local switch is the Principal Switch.Local switch run time information:State: StableLocal switch WWN: 20:01:00:0b:46:79:ef:41Running fabric name: 20:01:00:0b:46:79:ef:41Running priority: 128Current domain ID: 0xed(237)Local switch configuration information:State: EnabledFCID persistence: DisabledAuto-reconfiguration: DisabledContiguous-allocation: DisabledConfigured fabric name: 20:01:00:05:30:00:28:dfConfigured priority: 128Configured domain ID: 0x00(0) (preferred)Principal switch run time information:Running priority: 128No interfaces available.Use the show fcdomain domain-list command to display the list of domain IDs of all switches belonging to a specified VSAN. This list provides the WWN of the switches owning each domain ID. Example 10-2 shows the following:
•
•
•
Example 10-2 Displays the fcdomain Lists
switch# show fcdomain domain-list vsan 76Number of domains: 3Domain ID WWN--------- -----------------------0xc8(200) 20:01:00:05:30:00:47:df [Principal]0x63(99) 20:01:00:0d:ec:08:60:c1 [Local]0x61(97) 50:00:53:0f:ff:f0:10:06 [Virtual (IVR)]Use the show fcdomain allowed vsan command to display the list of allowed domain IDs configured on this switch. See Example 10-3.
Example 10-3 Displays the Allowed Domain ID Lists
switch# show fcdomain allowed vsan 1Assigned or unallowed domain IDs: 1-96,100,111-239.[Interoperability Mode 1] allowed domain IDs: 97-127.[User] configured allowed domain IDs: 50-110.
Tip
Use the show fcdomain fcid persistent command to display all existing, persistent FC IDs for a specified VSAN. You can also specify the unused option to view only persistent FC IDs that are still not in use. See Examples 10-4 and 10-5.
Example 10-4 Displays Persistent FC IDs in a Specified VSAN
switch# show fcdomain fcid persistent vsan 1000Total entries 2.Persistent FCIDs table contents:VSAN WWN FCID Mask Used Assignment---- ----------------------- -------- ----------- ---- ----------1000 11:11:22:22:11:11:12:23 0x700101 SINGLE FCID NO STATIC1000 44:44:33:33:22:22:11:11 0x701000 ENTIRE AREA NO DYNAMICExample 10-5 Displays All Persistent FC IDs in the fcdomain
switch# show fcdomain fcid persistentTotal entries 2.Persistent FCIDs table contents:VSAN WWN FCID Mask Used Assignment---- ----------------------- -------- ----------- ---- ----------1000 11:11:22:22:11:11:22:22 0x700501 SINGLE FCID NO STATIC1003 44:44:33:33:22:22:11:11 0x781000 ENTIRE AREA YES DYNAMICUse the show fcdomain statistics command to display frame and other fcdomain statistics for a specified VSAN or PortChannel. See Example 10-6 and Example 10-7.
Example 10-6 Displays fcdomain Statistics for a Specified VSAN
switch# show fcdomain statistics vsan 1VSAN StatisticsNumber of Principal Switch Selections: 5Number of times Local Switch was Principal: 0Number of 'Build Fabric's: 3Number of 'Fabric Reconfigurations': 0Example 10-7 Displays fcdomain Statistics for a Specified PortChannel
switch# show fcdomain statistics interface port-channel 10 vsan 1Interface Statistics:Transmitted Received----------- --------EFPs 13 9DIAs 7 7RDIs 0 0ACCs 21 25RJTs 1 1BFs 2 2RCFs 4 4Error 0 0Total 48 48Total Retries: 0Total Frames: 96----------- --------Use the show fcdomain address-allocation command to display FC ID allocation statistics including a list of assigned and free FC IDs. See Example 10-8.
Example 10-8 Displays FC ID Information
switch# show fcdomain address-allocation vsan 1Free FCIDs: 0x020000 to 0x02fdff0x02ff00 to 0x02fffeAssigned FCIDs: 0x02fe00 to 0x02feff0x02ffffReserved FCIDs: 0x020100 to 0x02f0ff0x02fe00 to 0x02feff0x02ffffNumber free FCIDs: 65279Number assigned FCIDs: 257Number reserved FCIDs: 61697Use the show fcdomain address-allocation cache command to display the valid address allocation cache. The cache is used by the principal switch to reassign the FC IDs for a device (disk or host) that exited and reentered the fabric. In the cache content, VSAN refers to the VSAN that contains the device, WWN refers to the device that owned the FC IDs, and mask refers to a single or entire area of FC IDs. See Example 10-9.
Example 10-9 Displays Address Allocation Information
switch# show fcdomain address-allocation cacheCache content:line# VSAN WWN FCID mask----- ---- ----------------------- -------- -----------1. 12 21:00:00:e0:8b:08:a2:21 0xef0400 ENTIRE AREA2. 6 50:06:04:82:c3:a1:2f:5c 0xef0002 SINGLE FCID3. 8 20:4e:00:05:30:00:24:5e 0xef0300 ENTIRE AREA4. 8 50:06:04:82:c3:a1:2f:52 0xef0001 SINGLE FCIDFeature History for Domain Parameters
Table 10-3 lists the release history for this feature. Only features that were introduced or modified in Release 3.x or a later release appear in the table.
