IP Routing

Components Used

Refer to Hardware Requirements for Catalyst 6000/Catalyst 6500 Redundancy for more information about the hardware requirements for redundancy on Catalyst 6000 series switches. Refer to Understanding Internal MSFC Redundancy on Hybrid Mode Catalyst 6000 Switches for more information about the minimum software release required on the Supervisor and MSFC for SRM.

This configuration was developed and tested using the software and hardware versions below.

• Catalyst 1 in the diagram below is a Catalyst 6509 with two 1000BaseX Supervisor WS-X6K-SUP1A-2GE and two MSFCs WS-F6K-MSFC2.

• Catalyst 2 in the diagram below is a Catalyst 6506 with two 1000BaseX Supervisor WS-X6K-SUP1A-2GE and two MSFCs WS-F6K-MSFC.

• Both Catalyst switches are running cat6000-supk8.8-1-2.bin OS software.

• The MSFC2 on Catalyst 1 is running c6msfc2-psv-mz.121-13.E10 and the MSFC on Catalyst 2 is running c6msfc-psv-mz.121-13.E10 software.

• The devices in AS10 and AS20 in the diagram are Cisco 2500 series routers running Cisco IOS^® Software Release 12.3(3).

The information presented in this document was created from devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If you are working in a live network, ensure that you understand the potential impact of any command before using it.

Refer to the Product Literature for more information on Catalyst 6000/6500 series switches.

Background Theory

In SRM redundancy, only the designated router (MSFC) is visible to the network at any given time. The non-designated router is booted up completely and participates in configuration synchronization, which is automatically enabled when entering SRM. Unlike the MSFC high availability method, the configuration of the non-designated router is exactly the same as the designated router, but its interfaces are kept in a "line down" state and are not visible to the network. Processes, such as routing protocols, are created on the non-designated router and the designated router, but all non-designated router interfaces are in a "line down" state; they do not send or receive updates from the network.

When the designated router fails, the non-designated router changes its state to become the designated router and its interface state changes to "link up." It builds its routing table while the existing Supervisor engine switch processor entries are used to forward Layer 3 traffic. After the newly designated router builds its routing table, the entries in the switch processor are updated.

Because only one MSFC is visible to the network at a given time, you do not need to establish multiple BGP peering between two MSFCs. (Other redundancy methods require peering between two MSFCs in one Catalyst chassis and two MSFCs in another Catalyst chassis.) In the event of a failure of the designated MSFC, the non-designated MSFC establishes BGP peering. Therefore, it always appears as a single BGP peer to the network and simplifies the network design, but it gives the same level of redundancy in case an MSFC has a failure.

The SRM redundancy feature is supported only on Catalyst 6000 series switches running CatOS System Software. For configuring redundancy on Catalyst 6000 switches operating Cisco IOS System Software, refer to Configuring EHSA Supervisor Engine Redundancy and Configuring RPR and RPR+ Supervisor Engine Redundancy.

Configure

In this section, you are presented with the information to configure the features described in this document.

Note: To find additional information on the commands used in this document, use the Command Lookup Tool (registered customers only) . You can also refer the Catalyst 6500 Series Command Reference and Catalyst 6500 Series Cisco IOS Command Reference for more information on these commands.

The basic steps of configuration are:

1. Configure SRM redundancy on Catalyst 1 and Catalyst 2 switches. For step-by-step instructions, see the Single Router Mode Redundancy section at Configuring Redundancy.

2. Configure BGP peering and policies on designated routers (MSFCs) for both Catalyst 1 and Catalyst 2 switches. For more information about how to configure BGP in dualhomed and multihomed situations, refer to Sample Configurations for Load Sharing with BGP in Single and Multihomed Environments.

Network Diagram

This document uses the network setup shown in the diagram below.

Click on the figure above to display it in a separate browser window, for use as reference later in this document.

In this network, Autonomous System AS4 is multihomed to AS10 and AS20 via VLAN 10 (Vlan10) and VLAN 20 (Vlan20), respectively. Within AS4, the network has two Catalyst 6000 switches (CAT 1 and CAT 2), each with two Supervisors and two MSFCs. Both of the switches are configured to provide Layer 2 and Layer 3 redundancy using a SRM configuration. CAT 1 does external BGP (eBGP) peering with AS10 and internal BGP (iBGP) peering with CAT 2. CAT 2 does eBGP peering with AS20 apart from the iBGP peering with CAT 1, thus providing redundancy in case the connection to one ISP fails.

Configurations

This document uses the configurations shown below.

CAT1> (enable) show running-config

!--- Output suppressed.

begin
!
# ***** NON-DEFAULT CONFIGURATION *****
!
!
#version 8.1(2)
!
#system
set system name CAT1
set system highavailability enable

!--- Enables high availability on Catalyst switch.

!
#vtp
set vtp domain cisco

!--- Output suppressed.

set vlan 1,10,20
!

!--- Output suppressed.

#set boot command
set boot config-register 0x2102
set boot system flash bootflash: cat6000-supk8.8-1-2.bin
!
# default port status is enable!!#module 1 : 2-port 1000BaseX Supervisor
!
#module 2 : 2-port 1000BaseX Supervisor
!
#module 3 : 48-port 10/100BaseTX Ethernet
set vlan 10 3/2

!--- Port 3/2 is connected to AS10 on VLAN 10.

!
#module 4 empty!
#module 5 : 8-port 1000BaseX Ethernet
set trunk 5/2 on isl 1-1005,1025-4094

!--- Port 5/2 is connected to CAT 2 as Trunk.

!

!--- Output suppressed.

!
#module 15 : 1-port Multilayer Switch Feature Card

!--- MSFC #1 on Port 1, Mod 15.

!
#module 16 : 1-port Multilayer Switch Feature Card

!--- MSFC #2 on Port 2, Mod 16.

CAT2 (enable) show running-config

!--- Output suppressed.

begin
!
# ***** NON-DEFAULT CONFIGURATION *****
!
!
#version 8.1(2)
!
#system
set system name CAT2
set system highavailability enable

!--- Enables high availability on Catalyst switch.
 
!
#vtp
set vtp domain cisco

!--- Output suppressed.

set vlan 1,10,20
!

!--- Output suppressed.

!
#set boot command
set boot config-register 0x2102
set boot system flash bootflash: cat6000-supk8.8-1-2.bin
!
!
# default port status is enable!!#module 1 : 2-port 1000BaseX Supervisor
!
#module 2 : 2-port 1000BaseX Supervisor
!
#module 3 : 48-port 10/100BaseTX Ethernet
set vlan 20 3/3

!--- Port 3/3 is connected to AS20 on VLAN 20.
 
! 
#module 4 : 8-port 1000BaseX Ethernet
set trunk 4/2 on isl 1-1005,1025-4094

!--- Port 4/2 is connected to CAT 1 as Trunk.
 
!

!--- Output suppressed.

!
#module 15 : 1-port Multilayer Switch Feature Card

!--- MSFC 1 on Port 1, Mod 15.

!
#module 16 : 1-port Multilayer Switch Feature Card

!--- MSFC #2 on Port 2, Mod 16.

CAT1> (enable)
CAT1> (enable) session 15
Trying Router-15...
Connected to Router-15.
Escape character is '^]'.

MSFC-CAT1> enable
MSFC-CAT1# show running-config
Building configuration...

Current configuration : 868 bytes
!
version 12.1
!

!--- Output suppressed.

!
hostname MSFC-CAT1
!
boot system bootflash: c6msfc2-psv-mz.121-13.E10
boot bootldr bootflash:c6msfc2-boot-mz.121-13.E10
!
ip subnet-zero
!
!
!
redundancy 

!--- This command enables redundancy.

 high-availability 

!--- This command enables high availability.

 single-router-mode 

!--- This command enables SRM.

!
interface Vlan10
 ip address 192.168.1.1 255.255.255.0

!--- Interface Vlan10 is acting as the routed interface for Vlan 10 on CAT1
 

!--- and is used for eBGP peering with AS10.

!
interface Vlan20
 ip address 192.168.2.3 255.255.255.0

!--- Interface Vlan20 is acting as the routed interface for Vlan 20 on CAT1


!--- and is used for iBGP peering with CAT2 via the trunk.

!
router bgp 4
 no synchronization
 bgp log-neighbor-changes
 network 192.168.0.0 mask 255.255.0.0
 neighbor 192.168.1.2 remote-as 10

!--- This command establishes eBGP peering with AS10.

 neighbor 192.168.2.1 remote-as 4

!--- This command establishes iBGP peering with interface VLAN 20 on CAT2.

 neighbor 192.168.2.1 next-hop-self

!--- This command assigns a next-hop value to 192.168.2.3
 

!--- for updates sent to 192.168.2.1 (the iBGP peer).

!
ip classless
ip route 192.168.0.0 255.255.0.0 Null0

!--- Output suppressed.

CAT2 (enable)
CAT2 (enable) session 15
Trying Router-15...
Connected to Router-15.
Escape character is '^]'. 
MSFC-CAT2> enable
MSFC-CAT2# show running-config
Building configuration...

Current configuration : 890 bytes
!
version 12.1
!

!--- Output suppressed.

!
hostname MSFC-CAT2
!
boot system flash bootflash: c6msfc-psv-mz.121-13.E10
boot bootldr bootflash:c6msfc-boot-mz.121-13.E10
!
ip subnet-zero
!
! 
redundancy 

!--- This command enables redundancy.

 high-availability 

!--- This command enables high availability.

 single-router-mode 

!--- This command enables SRM.

!
interface Vlan10
 ip address 192.168.1.3 255.255.255.0

!--- Interface Vlan10 is acting as the routed interface for Vlan 10 on CAT2

!
interface Vlan20
 ip address 192.168.2.1 255.255.255.0

!--- Interface Vlan20 is acting as the routed interface for Vlan 20 on CAT2


!--- and is used for iBGP peering with CAT1 as well as eBGP peering with AS20.

!
router bgp 4
 no synchronization
 bgp log-neighbor-changes
 network 192.168.0.0 mask 255.255.0.0
 neighbor 192.168.2.2 remote-as 20

!--- This command establishes eBGP peering with AS20.
 
 neighbor 192.168.2.3 remote-as 4

!--- This command establishes iBGP peering with interface Vlan20 on CAT1.

 neighbor 192.168.2.3 next-hop-self

!--- This command assigns a next-hop value to 192.168.2.1
 

!--- for updates sent to 192.168.2.3 (the iBGP peer).
 
!
ip classless
ip route 192.168.0.0 255.255.0.0 Null0

!--- Output suppressed.

Verify

This section provides information you can use to confirm your configuration is working properly.

Certain show commands are supported by the Output Interpreter Tool (registered customers only) , which allows you to view an analysis of show command output.

1. Identify and verify which MSFC module on CAT1 is in the active state and which is in the standby state. Use the show module command to verify the state. The highlighted output below identifies that Mod 15 is in the active state (Status = ok) and Mod 16 is in the standby state.

   CAT1> (enable) show module
   Mod Slot Ports Module-Type               Model               Sub Status
   --- ---- ----- ------------------------- ------------------- --- --------
   1   1    2     1000BaseX Supervisor      WS-X6K-SUP1A-2GE    yes ok
   15  1    1     Multilayer Switch Feature WS-F6K-MSFC2        no  ok
   2   2    2     1000BaseX Supervisor      WS-X6K-SUP1A-2GE    yes standby
   16  2    1     Multilayer Switch Feature WS-F6K-MSFC2        no  standby
   3   3    48    10/100BaseTX Ethernet     WS-X6248-RJ-45      no  ok
   5   5    8     1000BaseX Ethernet        WS-X6408A-GBIC      no  ok
   
   !--- Output suppressed.
   
   

2. Connect to the active MSFC and verify the redundancy state. In our case, the active MSFC is in Mod 15. To connect to Mod 15, use either the session or switch console command.

   CAT1> (enable) session 15
   Trying Router-15...
   Connected to Router-15.
   Escape character is '^]'.
   MSFC-CAT1>
   
   

   Once connected, use the show redundancy command to verify the redundancy status.

   MSFC-CAT1> enable
   MSFC-CAT1# show redundancy
   Designated Router: 1 Non-designated Router: 2
   Redundancy Status: designated 
   Config Sync AdminStatus : enabled
   Config Sync RuntimeStatus: enabled
   Single Router Mode AdminStatus : enabled
   Single Router Mode RuntimeStatus: enabled
   Single Router Mode transition timer : 120 seconds

   This confirms that MSFC in Mod 15 is in the active state and its redundancy state is designated. SRM status is enabled. When SRM is enabled, the Config Sync feature is automatically enabled which synchronizes the running and startup-configurations between the active and standby MSFCs on the switch.

3. Connect to the standby MSFC and verify the redundancy state. MSFC in Mod 16 is in the standby state. To connect to it, we use the session command, and then we use the show redundancy command to verify the state.

   CAT1> (enable) session 16
   Trying Router-16...
   Connected to Router-16.
   Escape character is '^]'.
   
   MSFC-CAT1>
   MSFC-CAT1> enable
   MSFC-CAT1# show redundancy
   Designated Router: 1 Non-designated Router: 2
   
   Redundancy Status: non-designated
   Config Sync AdminStatus : enabled
   Config Sync RuntimeStatus: enabled
   Single Router Mode AdminStatus : enabled
   Single Router Mode RuntimeStatus: enabled
   Single Router Mode transition timer : 120 seconds

   The above output confirms that Mod 16 is in the non-designated state and SRM status is enabled. In the standby state, all MSFC interfaces are in the down state. Use the show ip interface brief command to confirm this as shown below.

   MSFC-CAT1# show ip interface brief
   Interface                  IP-Address      OK? Method Status   Protocol
   Vlan10                     192.168.1.1     YES NVRAM  down     down
   Vlan20                     192.168.2.3     YES NVRAM  down     down

   Note: Step 4 to Step 6 are the same as Step 1 to Step 3, but on CAT2, and are used to verify the proper functioning of the SRM.

4. First check that the MSFC on Mod 15 is in the active state (Status = ok) and the MSFC in Mod 16 is in the standby state.

   CAT2> (enable) show module
   Mod Slot Ports Module-Type               Model               Sub Status
   --- ---- ----- ------------------------- ------------------- --- --------
   1   1    2     1000BaseX Supervisor      WS-X6K-SUP1A-2GE    yes ok
   15  1    1     Multilayer Switch Feature WS-F6K-MSFC         no  ok
   2   2    2     1000BaseX Supervisor      WS-X6K-SUP1A-2GE    yes standby
   16  2    1     Multilayer Switch Feature WS-F6K-MSFC         no  standby
   3   3    48    10/100BaseTX Ethernet     WS-X6248-RJ-45      no  ok
   4   4    8     1000BaseX Ethernet        WS-X6408-GBIC       no  ok
   
   !--- Output suppressed.
   
   

5. Connect to the active MSFC and verify the redundancy state.

   CAT2 (enable) session 15
   Trying Router-15...
   Connected to Router-15.
   Escape character is '^]'.
   
   MSFC-CAT2> enable
   MSFC-CAT2# show redundancy
   Designated Router: 1 Non-designated Router: 2
   
   Redundancy Status: designated
   Config Sync AdminStatus : enabled
   Config Sync RuntimeStatus: enabled
   Single Router Mode AdminStatus : enabled
   Single Router Mode RuntimeStatus: enabled
   Single Router Mode transition timer : 120 seconds

   The above output confirms that redundancy is enabled and this MSFC (Mod 15) is in the designated state.

6. Connect to the standby MSFC and verify the redundancy state.

   CAT2 (enable) session 16
   Trying Router-16...
   Connected to Router-16.
   Escape character is '^]'.
   
   MSFC-CAT2> enable
   MSFC-CAT2# show redundancy
   Designated Router: 1 Non-designated Router: 2
   
   Redundancy Status: non-designated
   Config Sync AdminStatus : enabled
   Config Sync RuntimeStatus: enabled
   Single Router Mode AdminStatus : enabled
   Single Router Mode RuntimeStatus: enabled
   Single Router Mode transition timer : 120 seconds

   The above output confirms the redundancy state of MSFC in Mod 16 as non-designated.

Troubleshoot

Now that you have verified the high availability SRM configuration, you need to verify the BGP status on MSFC on CAT1 and CAT2.

1. Use the show ip bgp summary command on the active MSFC on CAT1 to verify the neighbor establishment. The output below confirms successful eBGP and iBGP peering with AS10 and the CAT2 MSFC respectively.

   MSFC-CAT1# show ip bgp summary 
   BGP router identifier 192.168.2.3, local AS number 4 
   BGP table version is 4, main routing table version 4 
   3 network entries and 4 paths using 435 bytes of memory 
   4 BGP path attribute entries using 240 bytes of memory 
   2 BGP AS-PATH entries using 48 bytes of memory 
   0 BGP route-map cache entries using 0 bytes of memory 
   0 BGP filter-list cache entries using 0 bytes of memory 
   BGP activity 3/14 prefixes, 4/0 paths, scan interval 15 secs
     
   Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
   192.168.1.2     4    10      90      92        4    0    0 01:26:02        1
   192.168.2.1     4     4      91      91        4    0    0 01:25:38        2
   

2. Verify the BGP state on the active MSFC in CAT2. The output below confirms successful eBGP and iBGP peering with AS20 and CAT1 respectively.

   MSFC-CAT2# show ip bgp summary  
   BGP router identifier 192.168.2.1, local AS number 4 
   BGP table version is 4, main routing table version 4 
   3 network entries and 4 paths using 435 bytes of memory 
   4 BGP path attribute entries using 240 bytes of memory 
   2 BGP AS-PATH entries using 48 bytes of memory 
   0 BGP route-map cache entries using 0 bytes of memory 
   0 BGP filter-list cache entries using 0 bytes of memory 
   BGP activity 3/0 prefixes, 4/0 paths, scan interval 15 secs 
    
   Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
   192.168.2.2     4    20     110     112        4    0    0 01:46:43        1
   192.168.2.3     4     4     101     101        4    0    0 01:35:09        2
   

3. In the event the designated MSFC (Mod 15 on CAT1) experiences a failure, the standby MSFC (Mod 16 on CAT1) takes over the role of active MSFC. It is important to understand that with the failure of the designated MSFC and fail over to the non-designated MSFC, packet switching continues to work with the Supervisor engine switch processor entries programmed by the old designated MSFC. The newly designated MSFC eventually re-programs the Supervisor engine switch processor with its own entries and the old entries expire. The switch processor entries are either Multilayer Switching (MLS) shortcuts in the case of Supervisor 1A module with PFC, or stored in the Cisco Express forwarding (CEF) table in a Supervisor 2 module with PFC2.

   Failure of designated MSFC is simulated by resetting Mod 15 on CAT1 as shown below.

   CAT1> (enable) reset 15
   This command will reset module 15.
   Unsaved configuration on module 15 will be lost
   Do you want to continue (y/n) [n]? y
   2002 May 16 11:52:54 %SYS-5-MOD_RESET:Module 15 reset from Console//
   Resetting module 15...

   Ping traffic was generated from AS10 to a destination in AS20 during the fail over on CAT1 from designated to non-designated MSFC. Note that there was minimal packet loss during this fail over and the packet forwarding continued to work with the old MLS shortcuts programmed by the Mod 15 MSFC until the Mod 16 MSFC re-programs the switch processor with its own MLS entries.

   AS10-Router# ping
   Protocol [ip]:
   Target IP address: 172.16.1.1
   Repeat count [5]: 1000
   Datagram size [100]:
   Timeout in seconds [2]:
   Extended commands [n]:
   Sweep range of sizes [n]:
   Type escape sequence to abort.
   Sending 1000, 100-byte ICMP Echos to 172.16.1.1, timeout is 2 seconds:
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !!!!!!!!!!!!!!!!!!!!!
   Success rate is 99 percent (999/1000), round-trip min/avg/max = 1/3/116 ms

   Compared to other redundancy options for Catalyst 6000 MSFC, the SRM redundancy provides the advantages of conserving IP address by using the same address on both the active and standby MSFCs, thereby reducing routing protocol peering and making configuration simpler while providing the same failover capability as shown in this example.

Related Information

• Understanding Internal MSFC Redundancy on Hybrid Mode Catalyst 6000 Switches
• Hardware Requirements for Catalyst 6000/Catalyst 6500 Redundancy
• Sample Configurations for Load Sharing with BGP in Single and Multihomed Environments
• Achieve Optimal Routing and Reduce BGP Memory Consumption
• How to Use HSRP to Provide Redundancy in a Multihomed BGP Network
• Configuring Redundancy
• Cisco IOS Configuration Guide for BGP
• BGP Technology Support Page
• Technical Support - Cisco Systems