-
Cisco Active Network Abstraction Managing MPLS User Guide, 3.6.5
-
preface.fm
-
Introducing MPLS VPN Maps
-
Creating and Manipulating VPN MPLS Maps
-
Creating and Manipulating Cisco ANA Business Configuration
-
Viewing VPN Properties In Service View
-
Viewing MPLS Related Inventory Properties
-
Fault Management In MPLS Networks
-
Calculating Impact Analysis
-
Working with PathTracer in VPN Service View
-
Running a VPN Leak Report Command
-
Additional Alarms
-
Feedback
Table Of Contents
Fault Management In MPLS Networks
MPLS TE Tunnel Down and TE Tunnel Flapping
Pseudo Wire (L2 VPN) MPLS Tunnel Down
Fault Management In MPLS Networks
This chapter describes the alarms that Cisco ANA detects and reports for LDP, BGP, MPLS TE (using RSVP TE), MPLS Black Holes, as well as alarm reports for Layer 2 and Layer 3 VPNs:
•
MPLS Related Faults—Describes the "MPLS black hole found" and "Broken LSP discovered" alarms.
•
•
•
•
•
Cisco ANA supports the following alarms:
•
–
–
•
–
•
–
•
–
–
–
•
–
The alarms are displayed in the ticket pane of the Cisco ANA NetworkVision window. For more information about the ticket pane, see the Cisco Active Network Abstraction 3.6.5 User Guide.
MPLS Related Faults
This section includes descriptions of the following MPLS related faults:
•
•
Note
MPLS Black Hole Found Alarm
A MPLS "black hole" is defined as an abnormal termination of a MPLS path (LSP) inside a MPLS network. A MPLS "black hole" exists when on a specific interface there are untagged entries destined for a known PE router. It is assumed that a router functions as a PE router if there are services using the MPLS network, such as L3 VPNs or Pseudo Wire (L2 VPN) MPLS Tunnels. Note that the untagged interfaces may exist in the network in normal situations. For example, where the boundary of the MPLS cloud has untagged interfaces this is still considered normal.
The existence of a MPLS "black hole" results in a loss of all the MPLS labels on a packet including the VPN information which lies in the inner MPLS label. So if a packet goes through an untagged interface, the VPN information is lost. The VPN information loss translates directly to VPN sites losing connectivity.
A "MPLS Black Hole Found" alarm is detected actively by the system, namely, service alarms are generated whenever Cisco ANA discovers a MPLS interface that has at least one untagged LSP leading to a known PE router.
Black hole alarms are detected either:
•
•
Broken LSP Discovered Alarm
The "MPLS Black Hole Found" alarm activates a backward flow on the specific untagged entry in order to traverse the full path of the LSPs passing through it. If Cisco ANA locates services (VRFs, Pseudo Wire L2 tunnels) along this path that are using these LSPs a "Broken LSP Discovered" alarm is issued. Such services can only be found on PE routers and they can be found on more than one PE router. The source of the "Broken LSP Discovered" alarm is the PE router on which the service was discovered and in many cases this router is different from the router that issued the "MPLS Black Hole Found" alarm.
"Broken LSP Discovered" alarms are correlated to the "MPLS Black Hole Found" alarm (except in the case of a Black hole alarm due to a link down as described on page 6-4).
The "Broken LSP Discovered" alarm is detected actively by the system, namely, service alarms are generated.
An example of a MPLS black hole scenario follows.
In the network described in this example, the shortest path from PE2 to PE3 is PE2<->P2<->PE3. The link between P2 and PE3 is a MPLS link, meaning interfaces on both side of the link are configured as MPLS interfaces. Also assume that for some reason the MPLS configuration is incomplete or incorrect, namely:
•
•
In this case the label switching table on P2 and PE3 will have untagged entries for the LSPs between PE2 and PE3. If PE2 and PE3 have VPN services (VRFs, Pseudo Wire tunnels) the outcome will be that the data flow between PE2 and PE3 will be affected.
Figure 6-1 Example of a MPLS Black Hole Scenario
In this case Cisco ANA does the following:
•
•
•
–
–
•
Note
Using this functionality, once the "MPLS Black hole" alarm is cleared, then after a specific time interval (configured Gateway correlation timeout) has passed, the Gateway will be able to close the ticket since all the alarms correlated to "MPLS Black hole" are "Broken LSP" which are configured as auto cleared.
Black Hole To Link Down
In a case where a link down event in a MPLS network has caused an IP reroute and therefore LDP redistribution, a case may arise where new LSPs are now redirected through a non-MPLS segment thereby creating a black hole.
In this case the "Broken LSP Discovered" alarms are issued as described in Broken LSP Discovered Alarm, but all the broken LSPs that are found are correlated to the "Link Down" alarm and not to the "MPLS Black Hole Found" alarm.
LDP Related Faults
Label Distribution Protocol (LDP) enables neighboring P or PE routers acting as label switch routers (LSRs) to discover potential peers in an MPLS network with which they can establish LDP sessions, in order to negotiate and exchange labels to be used for forwarding packets.
If a session to an LDP neighbor goes down, an "LDP neighbor down" alarm is issued. This can happen as the result of a failure in the TCP connection used by the LDP session, or if the interface is no longer running MPLS. The "LDP neighbor down" alarm is cleared by a corresponding "LDP neighbor up" alarm.
The alarm is issued when a peer is removed from the table in the LDP Neighbors tab. The alarm runs a correlation flow to detect what event happened in the network core, and then performs a Root Cause Analysis to find its root cause. The alarm initiates an IP based flow towards the "Peer Transport Address" destination. If an alarm is found during the flow, it will correlate to it.
Note
BGP Related Faults
Cisco ANA monitors BGP neighbor information and makes correlation and impact analysis information available to users.
This section includes a description of the BGP related faults.
BGP Neighbor Loss
In IP/MPLS VPN networks, when BGP connectivity is lost to a specific device, the resulting BGP connection loss translates directly to VPN sites losing connectivity.
The VNE models the BGP connection between routers and actively monitors its state. A BGP neighbor loss alarm is generated from both sides of the connection in the case of a connectivity loss, resulting in alarms and tickets being issued and users viewing impact analysis information.
The correlation engine identifies various faults that affect the BGP connection and reports them as the root cause for the BGP neighbor loss alarm. For example, Link down, CPU over utilized, and Link data loss.
Note
The "BGP Neighbor Loss" alarm is detected actively by the system, namely, service alarms are generated.
The system also supports "BGP neighbor down" syslogs.
BGP Neighbor Loss
When the VNE's BGP component is polling the status of BGP neighbors (expedite or normal polling), and an entry for a neighbor does not exist any more or its state changed from Established state to any other state, the BGP component will issue an alarm for "BGP Neighbor Loss". This alarm causes the BGP component to issue a Root Cause Analysis (RCA) correlation flow to find its root cause.
If RCA does not find any alarm to correlate to, the VNE will send this alarm to the gateway as a ticket.
If this alarm is configured in the registry to issue "Look For Affected" flow, the following process will be issued:
•
•
•
BGP Process Down
A query checks the status of the BGP process when the VNE's BGP component is polling for the status and configuration of its BGP neighbors (expedite or normal polling).
If the BGP process is not running, the VNE's BGP component will issue an alarm "BGP Process Down".
This alarm will always be a ticket and will not try to correlate to other alarms.
All the "BGP Neighbors Down" alarms issued due to the "BGP Process Down" should correlate to this "BGP Process Down" ticket.
Traffic Engineering Faults
This section includes a description of the following Traffic Engineering related faults:
•
•
•
MPLS TE Tunnel Down and TE Tunnel Flapping
When a TE tunnel's operational status changes to down and the tunnel is not flapping, the system generates a "Tunnel Down" alarm.
The correlation engine identifies various faults that affect the TE tunnel's status and reports on them as the root cause for the TE "Tunnel Down" alarm, for example, Link down.
Multiple up and down alarms that are generated during a short time interval are suppressed and displayed as a "Tunnel Flapping" alarm (according to the specific flapping configuration).
The "MPLS TE Tunnel Down" and the "TE Tunnel flapping" alarms are detected actively by the system, namely, service alarms are generated.
The system also supports "MPLS TE Tunnel Down" syslogs, which are correlated to the service alarm.
For Cisco CRS-1 routers running Cisco IOS XR 3.6 software and using PBTS in MPLS or MPLS VPN networks, Cisco ANA supports the following 3 subalarms for the MPLE TE Tunnel Down alarm:
•
•
•
The specific subalarm that is generated depends on the EXP bit specified for the traffic. Cisco ANA maps the specified EXP bit to tunnel priority and uses that mapping to generate the resultant subalarm. The alarm description includes information about the EXP bit.
Tunnel Reoptimized
Tunnel reoptimization occurs when a tunnel is up and its route changes but the tunnel continues to remain up. When a TE tunnel is reoptimized to take a different path, the system parses the tunnel reoptimized syslog, if such a syslog is available, and displays this syslog as a ticket.
The "Tunnel Reoptimized" alarm is generated from a syslog message sent by the router.
Layer 2 VPN Faults
This section includes a description of the Layer 2 VPN fault, Pseudo Wire (L2 VPN) MPLS tunnel down.
Pseudo Wire (L2 VPN) MPLS Tunnel Down
A "Pseudo Wire MPLS Tunnel Down" alarm is issued when the pseudo wire link goes down, namely, the pseudo wire tunnel is reported as down from both the devices (based on the status of the tunnel), and the tunnel is not flapping.
The correlation engine identifies various faults that affect the Pseudo Wire tunnel status and reports on them as the root cause for the "Pseudo Wire MPLS Tunnel Down" alarm, for example, Link down.
Cisco ANA traces the LSE path to the edge of the PWE3 tunnel and marks the edges of the tunnel as affected.
The "Pseudo Wire MPLS Tunnel Down" alarm is detected actively by the system, namely, service alarms are generated.
Alarms Summary
The following section describes the alarms that may be displayed in the ticket pane of the Cisco ANA NetworkVision window for VPNs, including their severity and the up alarm for each:
For more information about the ticket pane, see the Cisco Active Network Abstraction 3.6.5 User Guide.
