User Guide for Mobile Wireless Transport Manager 5.0 - Troubleshooting MWTM and the Network [Cisco Mobile Wireless Transport Manager]

Table Of Contents

Troubleshooting MWTM and the Network

Clearing a Locked-Up MWTM Display

Investigating Data Problems

Understanding MWTM Client Start Error Messages

Data Model Mediator Service Error

Demand Poller Manager Service Error

Checking MWTM Server Start Processes

Viewing the MWTM Troubleshooting Log

Viewing MWTM Data on the Web

Viewing Detailed Troubleshooting Instructions for Events

Diagnosing a Typical Network Problem

Troubleshooting MWTM and the Network

This chapter provides the following information for troubleshooting basic MWTM and network problems:

•Clearing a Locked-Up MWTM Display

•Investigating Data Problems

•Understanding MWTM Client Start Error Messages

•Checking MWTM Server Start Processes

•Viewing the MWTM Troubleshooting Log

•Viewing MWTM Data on the Web

•Viewing Detailed Troubleshooting Instructions for Events

•Diagnosing a Typical Network Problem

Clearing a Locked-Up MWTM Display

In MWTM, events might cause message popups to remain in the background of your display, preventing you from interacting with other windows. If you suspect that your display has locked up, perform the following tasks:

•Make sure you are running MWTM on a supported operating system. For details on supported operating systems, see Chapter 1, "Preparing to Install MWTM" in the Cisco Mobile Wireless Transport Manager Installation Guide.

•Minimize windows and look for an MWTM message popup in the background.

Investigating Data Problems

If you suspect that there are problems with the data that MWTM is displaying, perform the following tasks:

•Enter equivalent show commands on the router. Is the data the same as that displayed by MWTM?

•Send SNMP queries to the nodes. Do all queries complete?

The results of these tasks can help you distinguish between a router problem and an MWTM problem.

Understanding MWTM Client Start Error Messages

If you encounter one of the following errors upon starting the MWTM client, follow the procedures below:

•DataModelMediatorService: Could not find service in RMI registry or the RMI Registry may be down.

•DemandPollerManagerService: Could not find service in RMI registry or the RMI registry may be down. Check the MWTM server and make sure it is running.

Data Model Mediator Service Error

If you have received the following message: "DataModelMediatorService: Could not find service in RMI registry or the RMI Registry may be down" either you have specified an incorrect port number when installing MWTM, or the server or RMI registry is unavailable.

To correct this problem, use the following procedure:

Step 1 Verify that you specified a correct port number.

Step 2 Enter the mwtm status command on the server to determine the status of all MWTM servers on the local host.

Step 3 Enter the mwtm restart command to restart any servers that are not running.

Demand Poller Manager Service Error

If you have received the following message: "DemandPollerManagerService: Could not find service in RMI registry or the RMI registry may be down. Check the MWTM server and make sure it is running" one or more of the MWTM server processes may not have started.

To diagnose and correct this problem, use the following procedure:

Step 1 Enter the mwtm status command on the server to determine the status of all MWTM processes.

Check the output to see if the sgmDataServer and sgmTrapReceiver processes do not appear in the Ready state. They may appear as follows:
PROCESS         STATE      PID    Last Message 
sgmDataServer    Starting    2586   Starting EventModelMediatorService
sgmMsgLogServer  Ready       2551   Running
sgmTrapReceiver  Initial 
Step 2 If the processes are not all in a Ready state, search the following log file /opt/CSCOsgm/logs/messageLog.txt for the following error message:
A java.IO.EOFException was encountered against the persisted.server.data file.
Step 3 Enter the mwtm cleandb command on the server, which will restore the persisted.server.data file to a valid state. The output should now show all processes running, as follows:
PROCESS         STATE    PID      Last Message 
sgmDataServer    Ready    2586      Running
sgmMsgLogServer  Ready    2551      Running
sgmTrapReceiver  Ready    2600         Running
Step 4 Start the MWTM client, then re-discover the network (for details, see Chapter 2, "Discovering Your RAN-O Networks Using MWTM.")

Checking MWTM Server Start Processes

When you run the mwtm start command, normal output appears as follows:
PROCESS         STATE    PID      Last Message 
sgmDataServer    Ready    2586      Running
sgmMsgLogServer  Ready    2551      Running
sgmTrapReceiver  Ready    2600         Running
If the sgmDataServer and sgmTrapReceiver process do not appear in the Ready state, see the "Demand Poller Manager Service Error" section for details on fixing this issue.

Viewing the MWTM Troubleshooting Log

MWTM stores troubleshooting information in the /opt/CSCOsgm/tmp/cisco_sgm_tshoot.log file on the MWTM server. This log, which is updated each time the MWTM Server Troubleshooting page is accessed or the mwtm tac command is run, contains information that might be requested by Cisco customer support personnel.

If you want to view the MWTM troubleshooting log, Cisco strongly recommends that you do so in a Web browser. To view the log in a Web browser, select System Troubleshooting from the MWTM Server Home Page.

You can also view the log from the command line, but this method displays the entire log, which can contain thousands of lines of output, line-by-line on your workstation screen. Therefore, Cisco strongly recommends that you view the log from the Web, as indicated above. However, if you are viewing the log at the request of the Cisco Technical Assistance Center (TAC), it is best to view the log from the command line.

To view the log from the command line:

Step 1 Log in as the root user, as described in the "Becoming the Root User (Server Only)" section on page 3-3, or as a super user, as described in the "Specifying a Super User (Server Only)" section on page 10-19.

Step 2 Enter the following commands:

# cd /opt/CSCOsgm/bin
# ./mwtm tac
This command might take a minute or more to complete. When it completes, MWTM displays the following message and prompt:

Output is in /opt/CSCOsgm/tmp/cisco_sgm_tshoot.log
Would you like to view it? [y]
Step 3 Press Enter. MWTM displays the contents of the /opt/CSCOsgm/tmp/cisco_sgm_tshoot.log file.

Viewing MWTM Data on the Web

MWTM provides an enormous amount of Web-based troubleshooting information. From the MWTM Server Home Page, you can access many Web pages containing MWTM data, including server status, network status, installation logs, message logs, product documentation, and other important troubleshooting information about MWTM. For full details, see the "Accessing MWTM Data from a Web Browser" section on page 13-1.

Viewing Detailed Troubleshooting Instructions for Events

MWTM provides extensive type-specific help and troubleshooting instructions for events. To see help and troubleshooting instructions for an event, right-click the event and select Help for Event.

You can also provide your own enterprise-specific instructions to operators in the event help. For more information, see the "Changing the Way MWTM Processes Events" section on page 5-17.

Diagnosing a Typical Network Problem

When you use MWTM to diagnose a problem in a RAN-O network, follow these basic steps:

1. Monitor the network using the MWTM Main Window and the Topology Window. For example, an object in the topology map that changes color from green to yellow or red indicates a problem in the network.

2. Use MWTM windows, especially the Details window, to begin investigating the problem.

3. As you identify the source of the problem, examine the messages logged by MWTM for more detailed information about the sequence of events that led to the problem.

4. Telnet to the problematic router, if necessary.

The following real-life example provides detailed information about using MWTM to diagnose a problem in a RAN-O network:

Step 1 A network operator (we'll call him Joe) is using MWTM to monitor a RAN-O network. Joe has customized his view, limiting it to only those nodes for which he is responsible.

(For more information about customizing views, see the "Working with Views" section on page 4-1.)

Step 2 In the topology map, Joe notices a node that has changed color from green to yellow. Yellow indicates a status of Warning, which means that one or more interfaces associated with that node is in Unknown or Warning status and is not flagged as Ignored.

(For more information about node status, see the "Node Table" section on page 6-8.)

Step 3 Joe single-clicks the node in the topology map.

MWTM highlights the node in the topology map and in the topology view table in the left pane of the Topology Window. With the node highlighted, Joe can easily see that the name of the node is BTS-1941a.

MWTM also displays all associated interfaces in the topology Connections table.

Joe clicks the node's name and the zoom button in the topology view table.

MWTM redraws the topology map, centered on BTS-1941a, making it easier for Joe to see the relevant portion of the map.

(For more information about the Topology Window and how to use it, see the "Viewing the Topology of the Network" section on page 8-1.)

Step 4 Joe notices that one of BTS-1941a's diamonds is red, indicating that the associated interface is either Unavailable or Unknown. Joe single-clicks the red diamond.

MWTM highlights the connection in the topology map and in the topology Connections table. The table entry indicates that the connection is Unavailable.

Step 5 Joe right-clicks the connection in the topology map and selects View > Configuration Details in the right-click menu.

MWTM opens the Details window (in the main MWTM window), showing detailed information for the connection. In the Details window, detailed information for the selected connection is displayed in the Configuration Data section.

Immediately, Joe sees that the Operational Status is Down but notices that the Operational Status for E1 1/0 is Up.

Step 6 Joe selects the Recent Events tab and notices that a Critical Alarm for E1 1/0 was recently added.

Joe logs into the BTS-1941a device (right-click on device name and select Router > Telnet To) and runs the show controller E1 1/0 command. He learns that the device recently loss physical connectivity.

Step 7 Joe goes to the device and discovers that the cable is physically damaged. He replaces the cable and returns to the MWTM server.

Step 8 Joe views the MWTM main window and observes that MWTM has already polled the device and changed the state color from yellow to green.

Step 9 Joe looks at the MWTM topology window again and verifies the interface status has changed from yellow to green.

	User Guide for Mobile Wireless Transport Manager 5.0
	Troubleshooting MWTM and the Network
User Guide for Mobile Wireless Transport Manager 5.0 About This Document MWTM Overview Discovering Your RAN-O Networks Using MWTM Getting Started with MWTM Working with Views Working with Events Working with Nodes Working with Interfaces Viewing the Topology of the Network Working with MWTM Statistics Reports Configuring MWTM Security Configuring MWTM for Your Network Troubleshooting MWTM and the Network Accessing MWTM Data from the Web Browser Appendix A--MWTM Status Definitions Appendix B--MWTM FAQs Appendix C--MWTM Command Reference Appendix D--MWTM MIB Reference Appendix E--MWTM Trap Reference Appendix F--Configuring MWTM to Run with Various Networking Options Appendix G--MWTM Statistics Export File Formats Appendix H--MWTM and Ports Appendix I--Open Source License Notices for the Cisco Mobile Wireless Transport Manager Glossary Index	Download this chapter Troubleshooting MWTM and the Network Download the complete book Cisco Mobile Wireless Transport Manager User Guide (PDF - 5 MB) Feedback Table Of Contents Troubleshooting MWTM and the Network Clearing a Locked-Up MWTM Display Investigating Data Problems Understanding MWTM Client Start Error Messages Data Model Mediator Service Error Demand Poller Manager Service Error Checking MWTM Server Start Processes Viewing the MWTM Troubleshooting Log Viewing MWTM Data on the Web Viewing Detailed Troubleshooting Instructions for Events Diagnosing a Typical Network Problem Troubleshooting MWTM and the Network This chapter provides the following information for troubleshooting basic MWTM and network problems: •Clearing a Locked-Up MWTM Display •Investigating Data Problems •Understanding MWTM Client Start Error Messages •Checking MWTM Server Start Processes •Viewing the MWTM Troubleshooting Log •Viewing MWTM Data on the Web •Viewing Detailed Troubleshooting Instructions for Events •Diagnosing a Typical Network Problem Clearing a Locked-Up MWTM Display In MWTM, events might cause message popups to remain in the background of your display, preventing you from interacting with other windows. If you suspect that your display has locked up, perform the following tasks: •Make sure you are running MWTM on a supported operating system. For details on supported operating systems, see Chapter 1, "Preparing to Install MWTM" in the Cisco Mobile Wireless Transport Manager Installation Guide. •Minimize windows and look for an MWTM message popup in the background. Investigating Data Problems If you suspect that there are problems with the data that MWTM is displaying, perform the following tasks: •Enter equivalent show commands on the router. Is the data the same as that displayed by MWTM? •Send SNMP queries to the nodes. Do all queries complete? The results of these tasks can help you distinguish between a router problem and an MWTM problem. Understanding MWTM Client Start Error Messages If you encounter one of the following errors upon starting the MWTM client, follow the procedures below: •DataModelMediatorService: Could not find service in RMI registry or the RMI Registry may be down. •DemandPollerManagerService: Could not find service in RMI registry or the RMI registry may be down. Check the MWTM server and make sure it is running. Data Model Mediator Service Error If you have received the following message: "DataModelMediatorService: Could not find service in RMI registry or the RMI Registry may be down" either you have specified an incorrect port number when installing MWTM, or the server or RMI registry is unavailable. To correct this problem, use the following procedure: Step 1 Verify that you specified a correct port number. Step 2 Enter the mwtm status command on the server to determine the status of all MWTM servers on the local host. Step 3 Enter the mwtm restart command to restart any servers that are not running. Demand Poller Manager Service Error If you have received the following message: "DemandPollerManagerService: Could not find service in RMI registry or the RMI registry may be down. Check the MWTM server and make sure it is running" one or more of the MWTM server processes may not have started. To diagnose and correct this problem, use the following procedure: Step 1 Enter the mwtm status command on the server to determine the status of all MWTM processes. Check the output to see if the sgmDataServer and sgmTrapReceiver processes do not appear in the Ready state. They may appear as follows: PROCESS STATE PID Last Message sgmDataServer Starting 2586 Starting EventModelMediatorService sgmMsgLogServer Ready 2551 Running sgmTrapReceiver Initial Step 2 If the processes are not all in a Ready state, search the following log file /opt/CSCOsgm/logs/messageLog.txt for the following error message: A java.IO.EOFException was encountered against the persisted.server.data file. Step 3 Enter the mwtm cleandb command on the server, which will restore the persisted.server.data file to a valid state. The output should now show all processes running, as follows: PROCESS STATE PID Last Message sgmDataServer Ready 2586 Running sgmMsgLogServer Ready 2551 Running sgmTrapReceiver Ready 2600 Running Step 4 Start the MWTM client, then re-discover the network (for details, see Chapter 2, "Discovering Your RAN-O Networks Using MWTM.") Checking MWTM Server Start Processes When you run the mwtm start command, normal output appears as follows: PROCESS STATE PID Last Message sgmDataServer Ready 2586 Running sgmMsgLogServer Ready 2551 Running sgmTrapReceiver Ready 2600 Running If the sgmDataServer and sgmTrapReceiver process do not appear in the Ready state, see the "Demand Poller Manager Service Error" section for details on fixing this issue. Viewing the MWTM Troubleshooting Log MWTM stores troubleshooting information in the /opt/CSCOsgm/tmp/cisco_sgm_tshoot.log file on the MWTM server. This log, which is updated each time the MWTM Server Troubleshooting page is accessed or the mwtm tac command is run, contains information that might be requested by Cisco customer support personnel. If you want to view the MWTM troubleshooting log, Cisco strongly recommends that you do so in a Web browser. To view the log in a Web browser, select System Troubleshooting from the MWTM Server Home Page. You can also view the log from the command line, but this method displays the entire log, which can contain thousands of lines of output, line-by-line on your workstation screen. Therefore, Cisco strongly recommends that you view the log from the Web, as indicated above. However, if you are viewing the log at the request of the Cisco Technical Assistance Center (TAC), it is best to view the log from the command line. To view the log from the command line: Step 1 Log in as the root user, as described in the "Becoming the Root User (Server Only)" section on page 3-3, or as a super user, as described in the "Specifying a Super User (Server Only)" section on page 10-19. Step 2 Enter the following commands: # cd /opt/CSCOsgm/bin # ./mwtm tac This command might take a minute or more to complete. When it completes, MWTM displays the following message and prompt: Output is in /opt/CSCOsgm/tmp/cisco_sgm_tshoot.log Would you like to view it? [y] Step 3 Press Enter. MWTM displays the contents of the /opt/CSCOsgm/tmp/cisco_sgm_tshoot.log file. Viewing MWTM Data on the Web MWTM provides an enormous amount of Web-based troubleshooting information. From the MWTM Server Home Page, you can access many Web pages containing MWTM data, including server status, network status, installation logs, message logs, product documentation, and other important troubleshooting information about MWTM. For full details, see the "Accessing MWTM Data from a Web Browser" section on page 13-1. Viewing Detailed Troubleshooting Instructions for Events MWTM provides extensive type-specific help and troubleshooting instructions for events. To see help and troubleshooting instructions for an event, right-click the event and select Help for Event. You can also provide your own enterprise-specific instructions to operators in the event help. For more information, see the "Changing the Way MWTM Processes Events" section on page 5-17. Diagnosing a Typical Network Problem When you use MWTM to diagnose a problem in a RAN-O network, follow these basic steps: 1. Monitor the network using the MWTM Main Window and the Topology Window. For example, an object in the topology map that changes color from green to yellow or red indicates a problem in the network. 2. Use MWTM windows, especially the Details window, to begin investigating the problem. 3. As you identify the source of the problem, examine the messages logged by MWTM for more detailed information about the sequence of events that led to the problem. 4. Telnet to the problematic router, if necessary. The following real-life example provides detailed information about using MWTM to diagnose a problem in a RAN-O network: Step 1 A network operator (we'll call him Joe) is using MWTM to monitor a RAN-O network. Joe has customized his view, limiting it to only those nodes for which he is responsible. (For more information about customizing views, see the "Working with Views" section on page 4-1.) Step 2 In the topology map, Joe notices a node that has changed color from green to yellow. Yellow indicates a status of Warning, which means that one or more interfaces associated with that node is in Unknown or Warning status and is not flagged as Ignored. (For more information about node status, see the "Node Table" section on page 6-8.) Step 3 Joe single-clicks the node in the topology map. MWTM highlights the node in the topology map and in the topology view table in the left pane of the Topology Window. With the node highlighted, Joe can easily see that the name of the node is BTS-1941a. MWTM also displays all associated interfaces in the topology Connections table. Joe clicks the node's name and the zoom button in the topology view table. MWTM redraws the topology map, centered on BTS-1941a, making it easier for Joe to see the relevant portion of the map. (For more information about the Topology Window and how to use it, see the "Viewing the Topology of the Network" section on page 8-1.) Step 4 Joe notices that one of BTS-1941a's diamonds is red, indicating that the associated interface is either Unavailable or Unknown. Joe single-clicks the red diamond. MWTM highlights the connection in the topology map and in the topology Connections table. The table entry indicates that the connection is Unavailable. Step 5 Joe right-clicks the connection in the topology map and selects View > Configuration Details in the right-click menu. MWTM opens the Details window (in the main MWTM window), showing detailed information for the connection. In the Details window, detailed information for the selected connection is displayed in the Configuration Data section. Immediately, Joe sees that the Operational Status is Down but notices that the Operational Status for E1 1/0 is Up. Step 6 Joe selects the Recent Events tab and notices that a Critical Alarm for E1 1/0 was recently added. Joe logs into the BTS-1941a device (right-click on device name and select Router > Telnet To) and runs the show controller E1 1/0 command. He learns that the device recently loss physical connectivity. Step 7 Joe goes to the device and discovers that the cable is physically damaged. He replaces the cable and returns to the MWTM server. Step 8 Joe views the MWTM main window and observes that MWTM has already polled the device and changed the state color from yellow to green. Step 9 Joe looks at the MWTM topology window again and verifies the interface status has changed from yellow to green.
	Terms & Conditions \| Privacy Statement \| Cookie Policy \| Trademarks