The data center is central to IT strategy and houses the computational power, storage resources, and applications necessary to support an enterprise business. A flexible data center infrastructure than can support and quickly deploy new applications can result in significant competitive advantage, but designing such a data center requires solid initial planning and thoughtful consideration of port density, access-layer uplink bandwidth, true server capacity, oversubscription, mobility, and other details.
The new Cisco Nexus® 9000 Series Switches provide features optimized specifically for the data center: high 10 and 40-Gbps port densities, reliability, performance, scalability, programmability, and ease of management. With exceptional performance and a comprehensive feature set, the Cisco Nexus 9000 Series offers versatile platforms that can be deployed in multiple scenarios, such as layered access-aggregation-core designs, leaf-and-spine architecture, and compact aggregation solutions.
With new business services and applications emerging, causing enterprises to seek new data center infrastructure designs, organizations need to understand the implications of these changes on current business services and applications. Careful consideration of such factors as Layer 2 mobility, bandwidth and latency, and symmetrical paths through load balancers and firewalls is important to helping ensure successful migration of business services from your current data center infrastructure to your new one.
This document guides you through the planning, design, and deployment of data center infrastructure based on the Cisco Nexus 9500 platform, helping you migrate your data center to this new platform.
Purpose of This Guide
This document is intended to help network planners, engineers, and managers who are deploying Cisco Nexus 9000 Series Switches in a data center as replacements for Cisco Catalyst® 6500 Series Switches.
Migration Overview
As business services evolve to meet increasing customer expectations, IT has become the focal point for not only enabling differentiated services, but also for reducing operating costs. To address these expectations, some common IT initiatives are data center consolidation, cloud enablement, new application deployment, and infrastructure modularity.
New applications being deployed are also adopting distributed processing models, increasing the need for an agile and programmable data center switching platform. The Cisco Nexus 9000 Series addresses this need with switches that are designed to support agile, distributed, bandwidth-intensive applications.
At the time of this writing, the Cisco Nexus 9000 Series delivers a comprehensive Cisco® NX-OS Software data center switching feature set in the form factors listed in Table 1.
Table 1. Cisco Nexus 9000 Series Models
Device Model
Line Cards and Expansion Modules
Description
Deployment
Cisco Nexus 9508 Switch
N9K-X9636PQ
36-port 40-Gbps Enhanced Quad Small Form-Factor Pluggable (QSFP+)
End of row (EoR), middle of row (MoR), aggregation layer, and core
N9K-X9564TX
48-port 1/10GBASE-T plus 4-port 40-Gbps QSFP+
N9K-X9564PX
48-port 1/10-Gbps SFP+ plus 4-port 40-Gbps QSFP+
Cisco Nexus 9396PX Switch
N9K-C9396PX
Cisco Nexus 9300 platform with 48-port 1/10-Gbps SFP+
Top of rack (ToR), EoR, MoR, aggregation layer, and core
Cisco Nexus 93128TX Switch
N9K-C93128TX
Cisco Nexus 9300 platform with 96-port 1/10GBASE-T
ToR, EoR, MoR, aggregation layer, and core
The following sections of this document discuss factors you should consider when migrating from the Cisco Catalyst 6500 Series Switches currently in your data center to the Cisco Nexus 9000 Series Switches.
Data Center Facilities
The availability of servers with 10-Gbps LAN-on-motherboard (LoM) technology is increasing the need for high-density 10GBASE-T ports on access-layer switches. Most data centers that have the cabling required to support 10GBASE-T can save costs by preparing to upgrade their servers and access switches while maintaining their physical deployment models.
Servers dictate how access-layer switches are deployed; Figure 1 shows some popular deployments.
Figure 1. Common Switch Deployments
The EoR model is commonly used with Cisco Catalyst 6500 Series Switches deployed as data center access-layer switches. Servers with 1-Gbps network interface cards (NICs) then have cables running to the EoR or MoR, with Cisco Catalyst 6500 Series Switches providing connectivity.
The form factors listed in Table 2 make migration to the Cisco Nexus 9500 platform consistent with current EoR Cisco Catalyst 6500 Series deployment models while providing enough capacity to accommodate growth of 10-Gbps-based LoM servers. Cisco fabric extender (FEX) technology can also be used to accommodate 1-Gbps ports if a Cisco Catalyst 6513 Switch is being replaced.
Table 2. Cisco Catalyst 6500 Series and Cisco Nexus 9500 Platform Form Factors
Cisco Catalyst 6509 Switch
Cisco Catalyst 6513 Switch
Cisco Nexus 9508 Switch
Line-card slots
7
11
8
Height
14 rack units (14RU)
19RU
13RU
Weight
60 lb (27.3 kg)
79.1 lb (35.9 kg)
-
1-Gbps ports per card
48
48
48
10-Gbps ports per card
16
16
64
Maximum number of 1-Gbps ports per chassis
384
576
384
Maximum number of 10-Gbps ports per chassis
130
192
512
This migration can provide significant cost savings in traditional environments in which existing cabling infrastructure can be used. Figures 2 and 3 show some of the scenarios in which you might consider migrating from existing traditional Cisco Catalyst 6500 Series Switches to Cisco Nexus 9500 and 9300 platform switches.
Figure 2. Scenario 1 for Migration from Cisco Catalyst 6500 Series to Cisco Nexus 9000 Series
Figure 3. Scenario 2 for Migration from Cisco Catalyst 6500 Series to Cisco Nexus 9000 Series
Data Center Switching Systems
Data center network deployments for enterprises and for services providers vary in their design, purpose, scale, application connectivity, etc., but Cisco Nexus 9000 Series Switches are flexible enough for use in many commonly deployed traditional and new environments. Depending on business and application requirements, current Cisco Catalyst 6500 Series deployments can be migrated to the Cisco Nexus 9000 Series in many new topologies, as shown in Figure 4.
Figure 4. Migration Topologies: Migration from Cisco Catalyst 6500 Series to Cisco Nexus 9000 Series
With so many variations possible, no two data center network migrations are alike or proceed in the same way. Therefore, data center network migration should be approached holistically.
When migrating data center switches, you should first analyze the functions that the current switches perform to assess the various components of the data center. For this purpose, the concept of a data center switching system (DCSS) can be useful, helping simplify the process of migrating to Cisco Nexus 9000 Series Switches.
A DCSS consists of one or many switches (of any kind) that are interconnected so that they collectively provide Layer 1 through 3 connectivity to servers and Layer 4 through 7 devices, and the applications that connect to them. Even in the case of Cisco Catalyst 6500 Series service modules such as the Cisco Catalyst 6500 Series Firewall Services Module (FWSM), Cisco Application Control Engine (ACE), Cisco Catalyst 6500 Series Network Analysis Module (NAM), etc. that share the chassis, an internal connection still exists between the switch and the service modules.
From a holistic viewpoint, a DCSS provides network connectivity at Open Systems Interconnection (OSI) Layers 1 through 3 between the end devices using VLANs, switch virtual interfaces (SVIs), Virtual Routing and Forwarding (VRF) instances, routing, access control lists (ACLs), etc. (Figure 5). It consists of:
• One or more switches
• Inter-Switch Links (ISLs; at Layer 2 or 3)
• Interfaces
– Uplinks to data center core and edge
– Downlinks to servers and hosts
– Services links to Layer 4 through 7 devices
– DCI links to peer DCSSs
Figure 5. Holistic View of DCSS
When migrating the data center to Cisco Nexus 9000 Series Switches, you should not only consider compatibility with existing servers and devices; you should also consider incorporation of next-generation capabilities, including 10 and 40-Gbps connectivity, Layer 2 mobility, new features, higher performance, and programmability (Figure 6).
Figure 6. Migrate to Next-Generation Capabilities
You can apply the DCSS concept whether you are replacing one Cisco Catalyst 6500 Series Switch or a combination of many Cisco Catalyst 6500 Series Switches.
Many changes to the current Cisco Catalyst 6500 Series DCSS configuration are needed to deploy new features on the target Cisco Nexus 9000 Series DCSS. Table 3 summarizes the changes needed during migration from the current to the target DCSS.
Table 3. Configuration Changes Needed for Migration
Type
Number of Links
Target DCSS State
Change Analysis
Methodology
Internal links
Medium
Many changes
Depends on design
Specify new configurations
Global Configurations
-
Few changes plus new features
Match customer's old and new standards
Convert old and specify new feature configurations
Uplinks
Low
New links: cabling and speed
Match uplink Layer 2 and 3 configurations
Specify new configurations
Peer and DCI Links
Low
New links: cabling and speed
Match peer Layer 2 and 3 configurations
Specify new configurations
Services Layer (Load Balancing, Firewall, etc.)
Low
New links: cabling and speed
Match new high-performance device configurations
Specify new configurations
Downlinks
High
Few changes
Match host, server, and endpoint configurations
Convert configurations
Migration Process
Migration from a Cisco Catalyst 6500 Series DCSS to a Cisco Nexus 9000 Series DCSS involves the following high-level steps:
• Plan, design, and build Cisco Nexus 9000 Series DCSS.
• Interconnect current DCSS to Cisco Nexus 9000 Series DCSS.
• Migrate end-device connection points to Cisco Nexus 9000 Series DCSS.
The example in this document presents the process for migrating from access-layer EoR Cisco Catalyst 6500 Series Switches EoR Cisco Nexus 9500 platform switches. Figure 7 shows the existing setup; the current DCSS that needs to be migrated (a pair of EoR access-layer Cisco Catalyst 6500 Series Switches) is highlighted.
Figure 7. Example: Existing DCSS to Be Migrated
Step 1: Plan, Design, and Build EoR Access-Layer DCSS Based on Cisco Nexus 9000 Series
In this example, the new EoR DCSS consists of a pair of Cisco Nexus 9508 Switches populated with 48-port 1/10-Gbps cards. Here are some of the elements to consider during the planning phase:
• Facilities for EoR deployment model
– Power
– Cooling
– Rack space
– Server cabling (Cat-5, Cat-6, and Cat-6a)
– Interswitch cabling and use of 40-Gbps bidirectional (Bidir) QSFP
• Interoperability
– VLANs and VRF instances
– Default gateway and First Hop Redundancy Protocol (FHRP)
– Security policies
– ACLs
– Hardware use
• Capacity planning
– Port density and need for 1/10/40/100-Gbps speeds
– East-west and north-south traffic patterns
– Oversubscription
– Between downlinks and internal links
– Between downlinks and uplinks
– Between downlinks and services links
– Between downlinks and data center interconnect (DCI) links
• Availability and failover
– NIC teaming
– Active-active Layer 2 and 3 PortChannel links
– Latency requirements
• Interoperability of protocol optimizations and new features
– Spanning Tree Protocol
– Interior Gateway Protocol (IGP)
– Exterior Gateway Protocol (EGP)
– Multicast support
– Quality-of-service (QoS) requirements
– Classification and marking
– Policing
• DCI
– Layer 2
– Layer 3
Most of the new and enhanced Cisco Nexus 9000 Series features, such as 10/40-Gbps ports, virtual PortChannel (vPC), Virtual Extensible LAN (VXLAN) bridging and gateways, routing protocol enhancements, Rapid per VLAN Spanning Tree Protocol (PVSTP+), and role-based access control (RBAC), are internal to the target DCSS and can be used when the new Cisco Nexus 9000 Series DCSS is deployed.
Perform the following steps to build the new EoR DCSS using the Cisco Nexus 9508:
1. Specify the global system configuration. Secure Shell (SSH); Simple Network Management Protocol (SNMP); Authentication, Authorization, and Accounting (AAA); virtual terminal (vty); other system management functions are used by data center operations team to efficiently manage switches, and some of these features are common to both the Cisco Catalyst 6500 Series DCSS and the Cisco Nexus 9508 DCSS. You can use the Cisco Nexus Migration Tool, at http://tools.cisco.com/nxmt, to migrate common configurations to the Cisco Nexus 9508. Figure 8 shows a sample system configuration.
Figure 8. Sample System Configuration Output
2. Configure and set up vPC. A virtual PortChannel, or vPC, allows links that are physically connected to two different Cisco Nexus 9000 Series Switches to appear as a single PortChannel to a third device. A vPC can provide Layer 2 multipathing, which allows you to create redundancy by increasing bandwidth, enabling multiple parallel paths between nodes and load-balancing traffic where alternative paths exist (Figure 9). For more information, refer to the vPC design and best practices guides.
Figure 9. vPC Configuration
To configure vPC on Cisco Nexus 9500 Series Switches as shown in Figure 9, follow these steps:
a. Configure the peer keepalive interface IP address:
N9500-One# configure terminal
N9500-One(config)# vrf context vpcpeerkeepalive
N9500-One(config-vrf)# int Eth1/1
N9500-One(config-if)# no switchport
N9500-One(config-if)# ip vrf member vpcpeerkeepalive
N9500-One(config-if)# ip address 10.1.1.1/24
N9500-One(config-if)# no shutdown
b. Enable vPC and Link Aggregation Control Protocol (LACP):
N9500-One(config)# feature vpc
N9500-One(config)# feature lacp
c. Create a VLAN:
N9500-One(config)#vlan 201
d. Create the vPC domain:
N9500-One(config)# vpc domain 101
e. Configure the vPC role priority (optional):
N9500-One(config-vpc-domain)# role priority 1000
f. Configure the peer keepalive link using the address on N9500-Two Eth1/1 interface (10.1.1.2):
g. Configure the vPC peer link. Note that, as for a regular interswitch trunk, trunking must be turned on for the VLANs to which the vPC member port belongs.
N9500-One(config-vpc-domain)# int e1/49, e2/49, e3/49, e4/49
N9500-One(config-if-range)# channel-group 1 mode active
After this step, the Cisco Nexus 9508 DCSS should be fully staged and ready for interconnectivity, as shown in Figure 10.
Figure 10. Example: New DCSS Ready for Connection
Step 2: Interconnect Current DCSS to Cisco Nexus 9000 Series DCSS
After the Cisco Nexus 9508 DCSS has been staged and is operational as a standalone system, the next step is to assess the needs of connecting end devices and configure the required interconnectivity.
Some of the elements that need to be analyzed and configured to help ensure interoperability are:
• Spanning Tree Protocol: The Cisco Nexus 9508 supports Rapid PVST+ and Multiple Spanning Tree (MST). Rapid PVST+ uses the same Bridge Protocol Data Unit (BPDU) format as specified in IEEE 802.1D and is backward compatible. It is difficult to convert all data center switches at the same time to Rapid PVST+. Because of the backward compatibility, you can implement Rapid PVST+ in phases. In this example, because the current Cisco Catalyst 6500 Series Switches act as aggregation switches, they can remain as Spanning Tree Protocol root switches as well as run FHRP to be the default gateways.
• Segmentation and security policies: Applications and security requirements mandate segmentation, which normally is achieved through VLAN, VRF, and ACL configurations. VLANs, VRF instances, and ACLs that need to be migrated from the Cisco Catalyst 6500 Series DCSS to the new Cisco Nexus 9508 DCSS need to be configured. The Cisco Nexus Migration Tool can help in this process.
You also need to consider new features specific to the Cisco Nexus 9508, including unicast, multicast, QoS, and VXLAN capabilities, and deploy them according to your application and business requirements.
After this step, the Cisco Nexus 9508 DCSS should be fully operational and ready for downlink migration to provide connectivity to the servers (Figure 11).
Figure 11. New DCSS Ready for Migration
Step 3: Migrate End-Device Connection Points to Cisco Nexus 9000 Series DCSS
Although migration is a straightforward process of matching Cisco Nexus 9508 downlink interface configurations with Cisco Catalyst 6500 Series downlink interface configurations, the migration of a hundred or more servers can cause change-management challenges that make this process time consuming.
Servers being migrated can be categorized according to whether the servers be keeping their current teaming configurations or taking advantage of new features such as 10GBASE-T, vPC, etc. If the server configuration will remain unchanged, then the Cisco Nexus Configuration Tool can be used to convert the current Cisco Catalyst 6500 Series configuration to the Cisco Nexus 9508 interface configuration.
If the servers being migrated will use vPC, you can use the following sample template for Cisco Nexus 9508 downlink interface configuration:!
interface port-channel 101
description vPC PortChannel to Server#1
switchport mode access
switchport access vlan 201
spanning-tree port type edge
vpc 101
!
interface Ethernet2/1
description vPC PortChannel Member#1 to Server#1
switchport mode access
switchport access vlan 201
spanning-tree port type edge
channel-group 101 mode active
!
During this step, the topology will look similar to Figure 12.
Figure 12. Configuring vPC for the Migration
After all the servers are migrated, the topology would look like Figure 13, and the Cisco Catalyst 6500 Series EoR access switches can be decommissioned.
Figure 13. The New DCSS Topology
Conclusion
The Cisco Nexus 9000 Series provides versatile platforms designed to meet the needs of distributed and bandwidth-intensive next-generation applications. The Cisco Nexus 9000 Series is an agile platform with a comprehensive feature set with 1/10/40-Gbps port densities and form factors that are well suited for enterprises seeking to migrate from their existing Cisco Catalyst 6500 Series deployments. Although migration of data center switches is a challenging task due to the business-critical nature of hosted applications, the risk can be significantly reduced by using Cisco Nexus 9000 Series Switches and by using DCSS migration methodology.
For More Information
If your enterprise wants to use the expertise of Cisco Advanced Services, please contact us at as-aci-support@cisco.com.
