Cisco on Cisco

Cisco is recentralizing and virtualizing its data center infrastructure to increase business agility, a trend that the company calls Data Center 3.0. Data Center 1.0 refers to centralized mainframes, and Data Center 2.0 describes highly distributed architectures with reduced IT control (Figure 1).

The Data Center 3.0 initiative at Cisco supports the following business and IT goals:

• Increase server and storage utilization: When every Cisco branch had its own servers, server utilization hovered at 20 percent, and CPU utilization averaged 5 to 15 percent for Windows and Linux servers and 10 to 30 percent for Unix servers. Consolidating and virtualizing the Cisco data center environment have increased CPU and storage utilization to approximately 70 percent. Increased utilization reduces both capital and operational expense, for a lower total cost of ownership.
• Reduce operational costs: Cisco IT manages more than 15,000 physical and virtual servers, and wanted to improve the administration ratio from 1:72 to 1:250. A major strategy is converting physical servers in branch offices to virtual servers in Cisco data centers. This strategy eliminates the costs of dedicated IT personnel at branch offices to manage servers, network storage, and backup. A consolidated data center environment has also enabled Cisco IT to increase its use of automation, which addresses another source of operational costs.
• Accelerate deployment of new applications and services: Cisco IT is aggressively working towards decreasing provisioning times. In an automated, centralized data center, new services are deployed just once, in the data center, instead of once in each branch office.
• Facilitate disaster recovery: A centralized data center architecture offloads the responsibility for backup and recovery from branch office personnel.
• Standardize across all Cisco lines of business: “It’s more challenging to enforce Cisco IT data center standards In distributed data centers,” says John Nelsen, network and data center services architect, Cisco. “Bringing servers, storage, and applications back to our core data centers significantly reduces redundant efforts and nonstandard deployments, which tend to be costly.”
• Provide adequate power and cooling for new systems: New servers require more power and cooling than most branch offices can provide. Moving branch infrastructure to centralized data centers enables Cisco IT to take advantage of the Cisco Unified Computing System™, which combines compute, network, storage access, and virtualization in a single cohesive system, reducing space, power, and cooling requirements.
"We are experiencing the same performance increases with Cisco WAAS in more than 200 offices as we did with 30 offices," Palermo says. "The solution scales with no performance degradation."

Jim Palermo
IT Project Manager, Cisco
• Continue the journey to becoming a Borderless Enterprise: The Cisco global workforce increasingly works from branch offices and home. “It has become imperative for Cisco IT to provide remote office and mobile users with the same collaboration experience and application performance available to our employees based in campus sites,” says Jawahar Sivasankaran, senior manager, IT Customer Strategy and Success.

Cisco IT had to overcome two challenges to consolidate the data center environment:

• Accelerating performance of centralized applications delivered over the WAN, so that branch-office employees would be as productive as coworkers with offices next door to the data center.
• Optimizing bandwidth to accommodate the increase in traffic when Cisco’s remote offices began accessing services over the WAN instead of hosting them on their office LANs.

The company wanted an application acceleration solution that did not change the application experience, by adding additional steps, for example. Cisco IT also wanted a solution that would accelerate all TCP applications, including those based on HTTP, HTTPS, Microsoft Exchange, and Common Internet File System (CIFS).

Cisco IT found its solution in Cisco Wide Area Application Services (WAAS), which the company uses to accelerate Oracle, SAP, Microsoft Exchange, and other applications (Figure 2). Cisco branch-office employees access centralized applications and file systems exactly as they did when the applications and file systems were hosted locally, because Cisco WAAS operates transparently in both the network and application layers.

Cisco currently deploys a single Cisco WAAS appliance or router module in each branch and one or more in data centers. If an employee in Bulgaria visits a webpage hosted in the Cisco San Jose data center, for example, the Cisco WAAS appliance in Bulgaria communicates with the appliance in the San Jose data center to optimize requests and responses. Behind the scenes, multiple optimization and compression techniques accelerate application performance and minimize bandwidth.

Cisco uses a combination of standalone Cisco WAAS appliances and Cisco Integrated Services Router (ISR) modules. Most remote offices receive a Cisco WAE-674 appliance, which Cisco IT chose because it supports virtual blades. Cisco IT currently uses the virtual blade capability for enterprise print services and will soon add content and delivery network services. “Virtual blades reduce TCO [total cost of ownership] for branch offices,” says Jim Palermo, Cisco IT project manager.

Smaller Cisco offices receive a Cisco WAAS Network Module NME 522, which fits into the existing office Cisco ISR alongside other modules for unified communications, security, and wireless LAN. Combining WAAS and other services in one platform increases operational efficiency and reduces branch power and cooling requirements.

Cisco routers use Web Cache Coordination Protocol (WCCP) to direct traffic to Cisco WAAS. This approach increases application availability. During maintenance of a Cisco WAAS appliance or module, employees simply continue to access centralized applications and services as they would ordinarily, just without acceleration. “Installing WAAS takes just 10 minutes,” says Palermo. “The technician simply turns on WCCP within the router’s Cisco IOS Software and conducts tests.”

Cisco IT deployed Cisco WAAS across the global enterprise in three phases.

Cisco IT began developing a WAAS business case and architecture in March 2007. Late that year, Cisco conducted a proof of concept in seven remote offices in Europe, Eastern Europe, and other emerging markets, and the Amsterdam data center. Cisco chose these locations for the pilot, because they have limited WAN bandwidth and bandwidth is relatively expensive, maximizing potential performance improvements and cost savings.

Previously, Cisco IT used the Cisco Application Content and Networking System (ACNS) for web content caching and rich media prepositioning in these locations. Cisco WAAS further increased available bandwidth by optimizing more types of application traffic. Even the limited pilot reduced TCP traffic by 20 percent, and the combination of Cisco WAAS and Cisco ACNS reduced WAN bandwidth consumption by 40 percent (Figure 3).

In June 2008, Cisco IT began a global pilot with 40 offices representing a cross-section of Cisco locations and network topologies. During the 8-week pilot, Cisco IT confirmed that Cisco WAAS worked well with Cisco ACNS. The company also confirmed that the Cisco Intrusion Prevention System (IPS) did not perceive the Cisco Wide Area Application Engine (WAE) as a malicious host when it intercepted traffic.

Cisco timed the enterprise deployment to coincide with its data center virtualization program, based on the Cisco Unified Computing System. Cisco IT engaged its usual services partner to deploy Cisco WAAS. “Implementing Cisco WAAS is as simple as replacing a router, and our partner was comfortable viewing WAAS as part of the Cisco IT Fleet Management Program to continually refresh the core infrastructure upgrades during regularly scheduled office visits,” says Palermo.

Offices are eligible for a Cisco WAAS solution if they have less than 45 Mbps bandwidth. Over 200 offices and four data centers used Cisco WAAS as of January 2010, and Cisco IT expects that more than 300 offices and about eight data centers will be operational within the next few quarters. “Installation and operation have been trouble-free,” Palermo says. “We haven’t opened even one support case.”

Cisco IT introduced Cisco WAAS Mobile during the enterprise deployment. “A couple of thousand Cisco employees, mostly sales teams and executives, use Cisco WAAS Mobile lightweight software on their PCs to accelerate the VPN connection from their home or hotel to the Cisco network,” says Sivasankaran. Cisco WAAS Mobile works whether the employee is using a hardware or software VPN solution, and with any connection, including public Wi-Fi, cellular air card, satellite broadband, and DSL. “We’ve noticed that downloading a large presentation from home takes a fraction of the time with Cisco WAAS Mobile,” Palermo says.

In a survey, 92 percent of employees using Cisco WAAS Mobile recommended that WAAS Mobile become part of the standard desktop. “Cisco IT considers Cisco WAAS Mobile a part of our next-generation desktop architecture for Borderless Networks, our strategy to deliver services anytime, anywhere, and on any device,” says Sivasankaran. Cisco IT has tested Cisco WAAS Mobile for interoperability with other standard elements of Cisco desktops, and has standardized deployment of Cisco WAAS Mobile Server Software on a virtual server.

Cisco currently uses Cisco WAAS to optimize TCP application traffic, including HTTP, HTTPS, FTP, CIFS, and print services. Internal WAN traffic has decreased by as much as 60 percent in some locations, and Cisco IT projects up to US$30 million savings over three years from deferring or eliminating bandwidth service upgrades.

“Part of Cisco IT’s Borderless Networks strategy is to make the remote office employees’ WAN experience comparable to campus employees’ LAN experience,” says Sivasankaran. “Cisco WAAS has helped us to achieve this cost-effectively, so that Cisco IT can invest in business and collaboration applications instead of avoidable WAN overhead costs.”

The Cisco WAE 674 is helping Cisco IT virtualize its remote offices. “Consolidating intelligence reduces the need for peripheral equipment in our branches, reducing capital and operational expense,” Palermo says.

CIFS traffic optimization has been so effective that Cisco is eliminating local filers at 120 remote offices, moving their content to a regional filer hub. When the full deployment is complete, Cisco IT projects approximately $1 million savings over three years for hardware service and maintenance at branch offices.

Cisco WAAS accelerates HTTP page viewing by approximately 30 seconds, and employees typically visit several hundred HTTP and HTTPS pages daily. Over three years, Cisco IT conservatively estimates the company will achieve $50 million in productivity gains as thousands of remote office workers experience faster application performance. Interestingly, the productivity savings are similar throughout the world. In developed countries with faster connectivity, for example, higher fully burdened salaries offset the somewhat lesser time savings. .

Table 1 shows typical improvements to application performance over the WAN at Cisco. “We are experiencing the same performance increases with Cisco WAAS in more than 200 offices as we did with 30 offices,” Palermo says. “The solution scales with no performance degradation The improvements varied somewhat from test to test, based on latency, bandwidth, and whether Cisco ACNS was used.

Table 1. Performance Improvement in Branch Offices Using Cisco WAAS
Activity	Performance Improvement for First Access (%)	Performance Improvement for Second Access, After Content has Been Cached (%)
View intranet page	86	90
View cisco.com page	49	78
Browse Livelink	59	60
Open a file from within Livelink	61	93
Upload a 3.11MB PowerPoint file with Common Internet File System (CIFS) protocol	32	84
Download 3.11MB PowerPoint file with CIFS	32	99
Download a 3.11MB PowerPoint file with Microsoft Exchange	9	42

Cisco IT surveyed internal users on their impressions of application acceleration. Most respondents indicated that they were impressed with how much more quickly they could download a document from the Cisco internal documentation management systems or a shared folder several times faster, according to Palermo. They also noted faster downloading of Microsoft Exchange attachments, and faster rendering of certain internal webpages. As a result, when files on a local server in the Moscow branch office were moved to a larger file server in the Amsterdam data center, Moscow employees had no indication their files were now 1300 miles away.

Notably, the application performance increases from Cisco WAAS have remained steady as the deployment scaled from 30 to more than 200 offices.

Cisco WAAS optimizes Remote Desktop Protocol (RDP), which thin clients use to connect to a cloud computing system. Cisco IT is taking advantage of RDP optimization to deploy virtual desktops hosted on the Cisco Unified Computing System, with the goal of increased security, faster provisioning of new services, and cost savings from lower desktop costs and support requirements.

Cisco IT plans to expand its Cisco WAAS program in the following ways:

• Increase the number of Cisco WAAS data center deployments in regional data centers that provide distributed services: Regional data centers host services such as Microsoft Exchange or IBM Rational ClearCase, and are consolidation points for the Cisco developer community. “WAAS allows us to reduce the number of data centers required to support the developers’ application infrastructure,” Palermo says.
• Deploy Cisco WAAS appliances or modules in extranet sites: Cisco plans to deploy Cisco WAAS in a few hundred extranet partner sites, to improve performance between those sites and Cisco data centers.
• Make Cisco WAAS Mobile a standard desktop feature: Cisco IT is collaborating with the Desktop Solutions Group to extend Cisco WAAS Mobile to all employees, enabling them to work even more productively from home or customer and partner offices.

Cisco IT offers the following suggestions to other IT organizations planning to implement Cisco WAAS:

• Take an architectural approach whenever possible. "I've talked to many Cisco customers who have successfully deployed WAN optimization as a standalone technology, and it works just fine," says Sivasankaran. "At Cisco, we increased the value of Cisco WAAS by taking an architectural approach and integrating it into our high-level Borderless Networks strategy. Our WAN team is using Cisco WAAS in the design to consolidate our remote office IT infrastructure into a virtualized environment on the Cisco Unified Computing System."
• Clearly define requirements for the traffic that you optimize, and perform due diligence on the architecture and design analysis.
• Conduct a pilot before moving to a full production deployment, even if you have conducted lab testing or a proof of concept. Although the high-level Cisco WAAS architecture is the same for every organization, the optimal deployment varies based on the IT environment and project lifecycle. Cisco, for example, aligned the timeline for the Cisco WAAS global deployment with its new production data center strategy that relies on the Cisco Unified Computing System as a virtualization platform.
• Clearly document the roles and responsibilities of the operational organizations that will support and manage the Cisco WAAS infrastructure. In many IT organizations, Cisco WAAS affects both the group responsible for the routers that redirect traffic to the Cisco WAAS devices, and the group directly responsible for the Cisco WAAS devices, themselves. "Help ensure that both groups understand each other's role to streamline troubleshooting and address problems with the best efficiency," Palermo says.