Cisco on Cisco

Cisco's competitive advantage is based not only on innovative technology, but also on innovative business processes that increase employees' productivity and enable collaboration. Cisco® Unified Communications creates the foundation for many of the company's business processes. For example, Cisco Unified Communications System 6.0 introduced presence, which enables Cisco employees to see if their coworkers are currently available and how they prefer to be reached. This capability has eliminated the considerable time that employees previously spent trying multiple phone numbers and leaving messages.

Cisco IT upgrades the company's Cisco Unified Communications Manager architecture whenever the company introduces a major release, both to improve the user experience and to take advantage of new, time-saving management features. In November 2007, for example, Cisco IT upgraded globally to Cisco Unified Communications Manager 6.1.1.

In 2008, Cisco IT began upgrading to Cisco Unified Communications Manager 7.0, to take advantage of the new capabilities shown in Table 1.

New Capability	Difference from Version 6.1	Benefit for Cisco
Intelligent bridge selection	Intelligence to select video or audio-only conference bridges based on whether participants have video-enabled endpoints	Enables cost-effective, multipoint video conferencing for employees who use Cisco Unified Personal Communicator and Cisco Unified Video Advantage
Mobility improvements	Integration with Cisco Unified Mobility Advantage and Cisco Unified Mobile Communicator	Enables Dial Via Office, call screening, and plus (+) dialing from mobile phones
Productivity features	Supports the same features whether employees are using IP phones with Session Initiation Protocol (SIP) or Skinny Call Control Protocol (SCCP)	Allows employees to take advantage of directed call pickup, Do Not Disturb, and Cisco WebEx™widgets
Enhanced interoperability	Additional interoperability with partners through SIP	Enables scalable deployment of IP public switched telephone network (PSTN), fax, SIP phones, and more
Improved management tools	New bulk administration tool, Call Detail Record (CDR) monitoring, and enhanced configuration tool	Reduces the time Cisco IT spends on upgrades, configuration, and monitoring
Trusted relay point feature	Trusted quality of service (QoS) and Call Admission Control (CAC), as well as trusted VLAN traversal for Cisco Unified Communications software clients	Increases security of extranet IP telephony deployments

Cisco IT planned the global upgrade to minimize risk, minimize disruption of unified communications services, and make the most useful services available to employees in the least amount of time. The upgrade affected more than 90,000 employees and contractors working together at 300 sites worldwide. These workers have 120,000 IP phones, 45,000 softphones (Cisco Unified Personal Communicator and Cisco IP Communicator), 40,000 mobile phones, and 70,000 voice mailboxes.

Deployment Team
Cisco assembled a team to collaborate on global planning, implementation, communications, and training. The team included 15 IT staff members as well as staff from the Unified Communications Business Unit.

Timeframe

The team spent four weeks formulating an upgrade strategy and recommendations, including which features to enable and when, and four weeks on design and engineering. In three months, the team implemented the upgrade on 15 Cisco Unified Communications Manager clusters, including several megaclusters with up to 19 servers. The last cluster was upgraded in February 2009.

Upgrade Process

To minimize resource requirements, Cisco upgraded geographic theaters one by one instead of all at once. "Rather than upgrading the software and enabling new features at the same time, we upgraded the software in all locations first, and are currently introducing new capabilities in phases," says Hubert Ochsenhofer, Cisco IT solution architect. This approach minimized risk and IT resource requirements.

Cisco IT began by upgrading the Cisco Unified Communications Manager cluster in Sydney, Australia, conducting an early field trial (EFT) from November 2008 through the end of December 2008. "The release was stable through EFT," says Ian Pudney, Cisco IT manager for unified communications in Europe. Next, Cisco IT upgraded the servers in Singapore and Research Triangle Park, North Carolina; followed by Amsterdam, Netherlands; London, United Kingdom; Brussels, Belgium; and San Jose, California. The smallest cluster had three servers, and the largest was a 19-server megacluster in San Jose.

For each site, Cisco IT downloaded the installer software ahead of time, cached it on the hard drive, and then performed the installation on a weekend. Cisco IT installed the new software remotely, using Secure File Transfer Protocol (SFTP) or FTP. Installation took approximately one hour, depending on the database size.

To minimize the amount of time that employees did not have phone service on the weekend, Cisco IT used the following process:

• Downloaded the installer software during the work week, caching it on the hard drive
• Upgraded the backup servers in the cluster over the weekend while the primary server remained online
• Failed over to the upgraded backup servers
• Upgraded the primary servers
• Failed back to the primary servers

Employees experienced little or no service loss during this process, although calls that were active when the server was rebooted might be dropped.

"No person was without dial tone for more than five minutes, and people who worked closest to the data center usually had their service back in less than a minute," says Chris Hartley, unified communications network engineer at Cisco.

Cisco IT staged the upgrade during the day and performed the reboot after hours. Employees could change user settings such as speed-dial numbers and call forwarding at any time during the upgrade. However, they could not use the extension mobility feature, which allows employees to log onto any Cisco Unified IP phone to customize it with their own phone number and preferences.

Platform

Cisco IT deployed Cisco Unified Communications Manager 7.0 on Cisco 7845 H2 Media Convergence Servers, the same platform the company used for the previous version. The team upgraded to Red Hat Linux 4, which improves performance on large clusters. Red Hat Linux enabled Cisco IT to back out of upgrades if needed. Red Hat Linux 4 maintains active and inactive partitions, and Cisco IT upgraded the nonactive partition while the previous version continued to operate from the active partition.

"When we rebooted the server on the weekend, it pointed to the new partition, which loaded the new Cisco Unified Communications Manager 7.0 software," says Hartley. "If we needed to back out, we could simply reboot the updated server to the old version, which was on the inactive partition."

New Features for Productivity, Collaboration, and Management

Cisco employees continue to enjoy major features that were introduced in Cisco Unified Communications Manager 6.1, such as presence, Cisco Unity® voicemail, web collaboration, and enhanced mobility. As of February 2009, Cisco IT had implemented the following new features introduced with Cisco Unified Communications Manager 7.0.

Simplified International Dialing To save time, many Cisco employees return calls by simply selecting the number from their call logs or missed call lists and pressing the dial button. Previously, they had to manually add plus (+) numbers such as the international calling code and country code. With the new Calling Party Normalization feature in Cisco Unified Communications Manager 7.0, employees can return calls to standard international (E164) numbers with the touch of a button. All of the necessary transformations happen behind the scenes. For example, if someone in Australia clicks to dial a team member in Singapore, the system adds the international dial code and Singapore's country code. But if someone in Singapore clicks to dial the same person, the system knows to not add those digits. Employees can take advantage of simplified international dialing with desktop phones as well as mobile phones.

Intelligent Bridge Selection for Customer Video Conferencing When Cisco employees set up a conference, Cisco Unified Communications Manager 7.0 uses a video-capable bridge only if two or more participants have video endpoints. Otherwise, it uses an audio-only bridge, which costs only 10 percent as much as video-capable bridges and uses only 7 to 17 percent of the bandwidth. This feature is enabling Cisco to expand its use of video with lower costs.

Do Not Disturb Enhancements Employees who select the Do Not Disturb (DND) option now have a choice: turn off the ringer and leave the flash on, or turn off both and only receive notification of a call on the display.

IP Phone Service Enhancements Cisco IT can now set up IP phone services, such as extension mobility and personal directory, as enterprise services that apply to all IP phones and softphones. This procedure saves time because Cisco IT no longer needs to individually assign these services to each phone.

Presence Information on Call Lists Cisco IT enabled the "busy" field for call lists. Now employees who view a list of missed calls can see whether the calling party's line is busy or not. This feature is like viewing presence information on the Cisco Unified IP phone itself.

Phone Personalization Cisco IT selected the phone personalization option so that employees can personalize their Cisco Unified IP phones with a picture and ringtone. Information about the current call now takes up only two lines on the display, so more of the image is visible.

New Service Parameters Cisco IT selected the following new service parameters:

• Remove the + sign on outbound calls.
• Suppress music on hold on conference bridges. Now, if a conference participant puts the call on hold to attend to another issue, the other participants can continue their conversation without the distraction of music in the background.
• Never drop the special conference connection when the meeting initiator leaves.
• Enable call control features for voice or video conference participants other than the controller.

Dial via Office Cisco IT has introduced the Dial Via Office feature to several departments and will gradually add others. With this feature, employees who use Cisco Mobile Communicator software on their mobile phones can make calls as if they were using their office phones. When they want to initiate a call, they send a message over a data channel to Cisco Unified Communications Manager 7.0, through the Cisco Unified Mobility Advantage server.. Cisco Unified Communications Manager connects the called endpoint to the nearby enterprise voice gateway, then connects the mobile client to that same gateway to complete the call. To the mobile user, this is a local call to the local voice gateway. The rest of the call is carried across the enterprise IP network, saving mobile long-distance charges. Another advantage is that the employee's office number appears on the call recipient's caller ID display so that the recipient knows who is calling.

WebEx Widgets Cisco IT is conducting a limited trial of widgets. Employees can select a phone number in their browser or Microsoft Office applications and then simply click the ClickToDial widget. The Voicemail widget lets them see information about the voicemail on the IP phone to help them decide which messages to play back first.

Enhanced User Experience
New features in Cisco Unified Communications Manager 7.0 help employees work the way they like to work: for example, by setting the Do Not Disturb option, personalizing their phones, and more easily dialing international numbers.

Cost Avoidance

The intelligent bridge selection feature has enabled Cisco to allow spontaneous video conferencing at lower cost. Without the feature, Cisco would have had to replace its audio-only bridges with Multipoint Control Units (MCUs), which might have been cost-prohibitive. Now, Cisco has extended the life of its audio bridges because they can still be used for calls without video participants.

Foundation for Future Services

Cisco Unified Communications Manager 7.0 provides the foundation that will enable Cisco IT to enrich the Cisco voice network with other services (Table 1). Especially valuable to Cisco will be:

• Interconnecting with other IP voice networks using SIP trunking
• Making it more cost effective to expand the use of video conferencing, through intelligent bridge selection
• Empowering mobile employees by extending more services to mobile phones and smartphones
• Enabling tighter integration with partners' IP voice networks

Cisco IT will continually upgrade Cisco Unified Communications Manager with the latest software, to empower employees with the latest collaboration and productivity tools. Upgrades to Cisco Unified Communications Manager 7.1.2 and 8.01 are already planned.

Additional Cisco Unified Communications Manager 7.0 features that Cisco IT plans to enable include:

• Local route groups and transformation patterns. Cisco IT will no longer need to create route patterns for each new site, which will reduce the number of entries and translation patterns to maintain. "This will lower management costs, especially for clusters that span multiple countries or multiple offices in a country," says Paul Harrison, IT engineer for unified communications. "For example, in the United Kingdom, where we have five offices, we'll configure the dial plan just once instead of five times, which will save two or three hours."
• Trusted relay points. Trusted relay points enable trusted QoS and CAC, as well as trusted VLAN traversal for Cisco Unified Communications software clients.
• Expanded SIP support for endpoints and trunks.
• Expanded SIP compatibility with Cisco Unified Communications applications.
• Mobility time-of-day enhancements. Employees who direct calls to their home phones, for example, will be able to specify that they not receive calls during normal sleeping hours, a hazard in enterprises with employees in different time zones.
• Directed call pickup. Employees will be able to pick up a call ringing another extension, for example, when a team member is out of the office.

"Like other enterprise IT groups, Cisco IT usually does not implement a 'x.0' release," says Ochsenhofer. "However, Cisco Unified Communications Manager 7.0 was so thoroughly tested that we felt confident deploying it for 90,000 employees and contractors. The upgrade proceeded very smoothly."

Cisco IT shares the following lessons learned with companies that are planning their own upgrades to Cisco Unified Communications Manager 7.0:

• First upgrade the clusters, and later introduce the software's new capabilities, in phases.
• Use a consistent global design and configuration. Cisco IT uses the same standards for all 15 global clusters, regardless of their size.
• Follow consistent processes for each cluster upgrade. Cisco IT produced an upgrade document for its global IT teams.