Feedback
Table Of Contents
Installation and Upgrade FAQs and Troubleshooting
Installation and Upgrade Troubleshooting
Core Functionality FAQs and Troubleshooting
Core Functionality Troubleshooting
Email Integration FAQs and Troubleshooting
Email Integration Troubleshooting
Email and Office Plug-in FAQs and Troubleshooting
Calendar FAQs and Troubleshooting
Video Calls FAQs and Troubleshooting
Search FAQs and Troubleshooting
Health and Performance Monitoring FAQs and Troubleshooting
Health and Performance Monitoring FAQs
Health and Performance Monitoring Troubleshooting
Director FAQs and Troubleshooting
Worker FAQs and Troubleshooting
Message Queue FAQs and Troubleshooting
Analytics FAQs and Troubleshooting
My Library FAQs and Troubleshooting
Framework FAQs and Troubleshooting
Streams FAQs and Troubleshooting
UC Integrations FAQs and Troubleshooting
UC Integrations Troubleshooting
Open API FAQs and Troubleshooting
OpenSocial FAQs and Troubleshooting
Synthetic Monitoring FAQs and Troubleshooting
Synthetic Monitoring Troubleshooting
Using Jabber with Cisco WebEx Social FAQs and Troubleshooting
Using Jabber with Cisco WebEx Social FAQs
SSL Termination FAQs and Troubleshooting
SSL Termination Troubleshooting
FAQs and Troubleshooting
This chapter provides FAQs and troubleshooting hints for various Cisco WebEx Social components.
This chapter is organized as follows:
•
Installation and Upgrade FAQs and Troubleshooting
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Installation and Upgrade FAQs and Troubleshooting
•
Installation and Upgrade Troubleshooting
This section provides the following troubleshooting information:
•
•
Symptom The Topology page on the Director is empty after upgrade.
Possible Cause After upgrading Cisco WebEx Social to a newer version, the System > Topology page on the Director displays an empty server list.
Recommended Action Refresh your web browser using F5 or Ctrl-F5 (for hard refresh).
Symptom After an upgrade, the Director's Health page displays Integrity check alerts.
Possible Cause This issue may arise if the scheduled run of the Integrity service coincides with the upgrade process in which case the integrity check expectably fails because files are changing.
Recommended Action Manually rerun the integrity check on the affected nodes. Take these steps:
Step 1
Step 2
sudo /opt/cisco/bin/integrity.sh --run-check
Step 3
Core Functionality FAQs and Troubleshooting
•
Core Functionality Troubleshooting
This section provides the following troubleshooting information:
•
•
Symptom File attached to an update is not visible in full page view.
Possible Cause This issue can appear when you have your browser idle in Cisco WebEx Social for a long period (for example: > 8 hours). More specifically, these conditions must have all been true:
•
•
•
Recommended Action To prevent this issue from appearing again, sign in to the Director, go to Application > Portal > Advanced Portal Properties and search for com.cisco.ecp.vdl.attachment.cleanup.job.interval. Set its value to be one hour longer than the session timeout. (Note that the com.cisco.ecp.vdl.attachment.cleanup.job.interval is in minutes while the session timeout is in hours.)
Symptom Community creation pauses indefinitely.
Possible Cause This issue may be due to you NFS storage being unreachable.
Recommended Action Check to see if your NFS storage is running and responding and that it is reachable from the network segment Cisco WebEx Social is in.
Email Integration FAQs and Troubleshooting
•
Email Integration FAQs
•
•
•
•
•
•
•
•
Q.
A.
•
•
In additional you may want to enable DEBUG level for the QUAD_EVENTING category on Worker nodes.
Collect the following log files from each Worker node:
•
•
See Setting Log Trace Levels for detailed steps.
Q.
A.
•
•
Collect the following log files from each Worker node:
•
•
See Setting Log Trace Levels for detailed steps.
Q.
A.
•
•
Collect the following log files from each Worker node:
•
•
See Setting Log Trace Levels for detailed steps.
Q.
A.
Step 1
Step 2
debug_peer_list = example.comwhere example.com is the domain of the SMTP server that Postfix is sending to.
Step 3
Step 4
It is recommenced to undo the changes after you are finished troubleshooting. To undo the changes:
Step 1
Step 2
Step 3
Step 4
Q.
A.
Q.
A.
Q.
A.
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Q.
A.
Q.
A.
For inbound email:
•
•
•
•
•
•
For inbound email:
•
•
•
•
•
•
•
•
•
•
Email Integration Troubleshooting
This section provides the following troubleshooting information:
•
•
•
•
•
•
•
Symptom A reply created using Outlook/OWA is added as first level comment in WebEx Social instead of as a reply.
Possible Cause Your organization is using Microsoft Exchange Server 2003 without the KB908027 fix applied.
Recommended Action Apply the fix or upgrade to the latest version of Microsoft Exchange Server 2003. For more information, see http://support.microsoft.com/kb/908027.
Symptom Out of the Office auto replies to WebEx Social email notifications are added as content in WebEx Social.
Possible Cause The user account sending the Out of Office message is hosted on Microsoft Exchange Server 2003.
Recommended Action Mail accounts running on later versions of Microsoft Exchange Server or IBM Lotus Domino should not run into this issue.
Symptom A number of (or all) users did not receive their email digests (summary of important updates).
Possible Cause You have recently increased or decreased worker.digestscheduler.mainJobRepeatInterval. After the value has been modified and saved, the Digest Scheduler waits for that interval before it runs again. Depending on a number of related factors some users may be skipped when creating digests.
Recommended Action Wait for worker.digestscheduler.mainJobRepeatInterval to run out (30 min. if left at its default), then the digests should start arriving on schedule.
Possible Cause The user has requested a daily digest and you have set worker.digestscheduler.mainJobRepeatInterval to a large value that makes the next run of the Digest Scheduler to fall into the next day.
Recommended Action When you are setting the worker.digestscheduler.mainJobRepeatInterval parameter, take the "Daily Digest Notification Time" value (Director > Application > Portal > Email Digest) in consideration. Set worker.digestscheduler.mainJobRepeatInterval to a value that allows the Digest Scheduler to run at least once in the time frame between "Daily Digest Notification Time" and the end of the day.
For example if you have set "Daily Digest Notification Time" to 23:00 (11 pm), that leaves the Digest Scheduler only 1 hour to start and complete its run; ensure this by setting worker.digestscheduler.mainJobRepeatInterval to less than an hour.
Possible Cause The user has recently changed their time zone. If the new time zone has already been notified, the affected user does not receive their daily report.
Recommended Action Keeping the time zone setting should allow the user to receive future daily reports on schedule.
Symptom Users receive multiple daily digests.
Possible Cause The Administrator has changed the Daily Digest Notification Time after the daily digest has been sent out for the day.
Recommended Action If you want to avoid duplicate daily digests when changing Daily Digest Notification Time to an earlier time, ensure you make the change before the original time comes for the day.
Possible Cause The user has recently changed their time zone. If the new time zone has not been notified yet, the affected user receives a second report.
Recommended Action Keeping the time zone setting should allow the user to receive future daily reports on schedule.
Symptom Users receive multiple weekly digests.
Possible Cause The Administrator has changed the Weekly Digest Notification Date forward after the weekly digest has been sent out for the week.
Recommended Action If you want to avoid duplicate weekly digests, ensure you make the change to Weekly Digest Notification Date before the original time comes for the week.
Symptom Inbound email does not appear as content in Cisco WebEx Social.
Possible Cause The Worker node is restarting or has just been restarted.
Recommended Action Wait for about 10 minutes. After that, the emails that have been sent should appear as content.
Symptom Some replies are saved as new posts containing the entire email thread as opposed to comments to the original content.
Possible Cause This issue can arise if the original email has not yet been processed by Cisco WebEx Social.
Recommended Action If you experience this issue often, check the Dashboard for the Worker nodes (Director GUI > Stats). If the App Server/Worker nodes are under heavy load—as indicated by the CPU and Load charts—then consider adding more App Server or Worker nodes, or both.
Possible Cause This issue can arise if some of the SMTP headers that Cisco WebEx Social uses to identify the message thread have been deleted or lost.
Recommended Action Check to see if the all these SMTP headers are present in the email and if they have a meaningful value:
•
•
•
If one or more of the headers are missing, investigate to find out why and at what stage these headers have been lost.
Symptom Users are not receiving any emails generated by Cisco WebEx Social.
Possible Cause Your email relay is not relaying messages coming from Cisco WebEx Social.
Recommended Action Configure your email relay host to properly relay messages coming from Worker nodes.
Symptom Video thumbnails do not display in email notifications.
Possible Cause Worker nodes do not have the Show and Share (SNS) security certificates installed. Because email notifications are generated on the Worker role, all Worker nodes should have access to the Show and Share server to be able to access video thumbnail and URL data.
Recommended Action Ensure that you have installed the SNS security certificates on all App Server and Worker nodes. For installing the certificates on App Server nodes, see "Cisco Show And Share" in the Cisco WebEx Social Administration Guide. To install the certificates on Worker nodes, complete the following steps on each Worker node:
Step 1
Step 2
Step 3
cd /usr/java/default/lib/security/
Step 4
sudo ../../bin/keytool -import -keystore cacerts -alias <certificate alias> -file /home/admin/<certificate filename>
where certificate alias is an arbitrary name such as the SNS server hostname; certificate filename is the name of the PEM file containing the certificate.
Step 5
Step 6
service worker restart
Symptom Email notifications about alerts and announcements stop sending out.
Possible Cause This issue may appear after the Worker node is restarted. Alerts and announcements are then held in pending state because the main alerts/announcements job has entered an ERROR state.
Recommended Action Take the following steps to restore the notifications:
Note
Step 1
Step 2
select count(*)from worker_triggers wtwhere wt.trigger_state = 'ERROR';Step 3
update worker_triggers wt
set wt.trigger_state = 'WAITING'
where wt.strigger_state = 'ERROR';
Email and Office Plug-in FAQs and Troubleshooting
•
Email and Office Plug-in FAQs
•
•
•
Q.
A.
•
•
•
Q.
A.
•
•
Q.
A.
•
•
See Setting Log Trace Levels for detailed steps.
In addition, if Docx4j appears in any stack traces, then increase the Log4J logging level for org.docx4j.
Calendar FAQs and Troubleshooting
Calendar FAQs
•
•
•
Q.
A.
•
•
•
•
When viewing logs, look for the AGGREGATED_CALENDAR log key.
See Setting Log Trace Levels for detailed steps.
Q.
A.
Q.
A.
Calendar Troubleshooting
This section provides the following troubleshooting information:
•
•
Symptom LDAP user cannot connect to Microsoft Exchange Server through WebDAV.
Possible Cause (Only if "Use LDAP Directory Synchronization" is checked in the Calendar Configuration under Server > Common Configurations.) The user email address has been changed. Because WebEx Social uses the prefix of the email address to construct the WebDAV URL, the user can be prevented from connecting to Microsoft Exchange.
Recommended Action Ask the user to complete these steps:
Step 1
Step 2
Step 3
1.
2.
Step 4
The connection should succeed.
Symptom Domino users who connect through SSL cannot connect after upgrading WebEx Social.
Possible Cause The IBM Lotus Domino SSL security certificate has been invalidated by WebEx Social.
Recommended Action Reimport the SSL security certificate. See the Administration Guide for detailed instructions.
Symptom I added a new attendee to a recurring meeting using Microsoft Outlook but the new name does not appear in the Calendar application in Cisco WebEx Social.
Possible Cause There is a know issue with some versions of Microsoft Exchange Server when calendaring information is fetched over WebDAV.
Recommended Action Use Microsoft Outlook Web Access to edit the recurring meeting instead of Microsoft Outlook.
Video Calls FAQs and Troubleshooting
Video Calls FAQs
•
•
•
•
•
•
•
Q.
A.
http://<WS base url>/plugin/cwc/CWICPluginVersion
where <WS base url> is the URL you use to access WebEx Social.
Q.
A.
Q.
A.
Windows 7: %USERPROFILE%\AppData\Local\Temp\WebCommunicator.LOG
Windows XP: %USERPROFILE%\Local Settings\Temp\WebCommunicator.LOG
Mac OSX: /private/var/log/install.log
Q.
A.
Windows 7: %USERPROFILE%\AppData\Local\softphone.log
Windows XP: %USERPROFILE%\Local Settings\Application Data\softphone.log
Mac OSX: /Users/{$USER}/Library/Application Support/softphone.log
Q.
A.
Q.
A.
Q.
A.
•
Example:
<DialRule BeginsWith="408902" NumDigits="10" DigitsToRemove="2" PrefixWith=""/><DialRule BeginsWith="1408902" NumDigits="11" DigitsToRemove="3" PrefixWith=""/><DialRule BeginsWith="408525" NumDigits="10" DigitsToRemove="3" PrefixWith="8"/><DialRule BeginsWith="408526" NumDigits="10" DigitsToRemove="3" PrefixWith="8"/>•
Example:
<DialRule BeginsWith="22101" NumDigits="10" DigitsToRemove="1" PrefixWith="2"/><DialRule BeginsWith="902" NumDigits="7" DigitsToRemove="0" PrefixWith="+1408"/><DialRule BeginsWith="8256" NumDigits="8" DigitsToRemove="1" PrefixWith="+1206"/><DialRule BeginsWith="8525" NumDigits="8" DigitsToRemove="1" PrefixWith="+1408"/>Q.
Video Calls Troubleshooting
This section provides the following troubleshooting information:
•
•
Symptom I choose to send my video but the remote device does not display it.
Possible Cause A network/Internet security software on your computer is blocking the outbound connection.
Recommended Action The security software may or may not notify you of blocked connections. In both cases the solution is to whitelist the WebEx Social Call Plug-in in your security software.
Symptom Video originating from WebEx Social does not utilize the entire screen on some hardware communication devices.
Possible Cause The device does not have RTCP enabled. RTCP allows devices connected to CUCM to negotiate the best possible video resolution between endpoints. The option is enabled on the WebEx Social Call Plug-in by default.
Recommended Action In your Cisco Unified Communications Manager, ensure RTC is enabled for any devices that receives video from WebEx Social.
Symptom Error appears when the WebEx Social Call Plug-in is trying to load: "The Call Plug-in loaded successfully but there was a problem registering your phone."
Possible Cause Another device of the same type (Client Services Framework/ECP) has already registered onto the Cisco Unified Communications Manager (CUCM).
Recommended Action To see if another device has registered, go to the CUCM > Device > Phone and search for ECP<username> where <username> is the screen name of the user.
The IP Address column shows the IP address from which the ECP device is currently registered. If the Status column displays "Unregistered", IP Address is the last address that have been registered with that device. If the status is Registered or the IP Address does not correspond to the last location from where you used your Cisco WebEx phone, then some other application is using the ECP device. Unregister the device and reload the Cisco WebEx Social Call Plug-in before retrying a call.
Symptom Call cannot be placed with the Cisco WebEx Social Call Plug-in.
Possible Cause The Cisco WebEx Social Call Plug-in cannot download the device configuration file from the CUCM. The Call Plug-in reports the following:
0x107dd0000] csf.ecc: libXML2 msg: "Namespace prefix xsi for type on device is not defined"22-Jun-2012 16:12:55,207 -0700 ERROR [0x107dd0000] csf.ecc: insecureRetrieveConfig() file tftp://example.com/ecpapkshirs.cnf.xml requires security22-Jun-2012 16:12:55,207 -0700 ERROR [0x107dd0000] csf.ecc.api: fetchDeviceConfig() could not obtain config for ecpapkshirs22-Jun-2012 16:12:55,207 -0700 INFO [0x107dd0000] csf.ecc.api: getLastTFTPServerUsed() =22-Jun-2012 16:12:55,207 -0700 INFO [0x107dd0000] csf.ecc.api: getLastCCMCIPServerUsed() =22-Jun-2012 16:12:55,207 -0700 INFO [0x107dd0000] csf.ecc.api: connect(eSoftPhone, ecpapkshirs, )22-Jun-2012 16:12:55,207 -0700 ERROR [0x107dd0000] csf.ecc: doConnect() failed - No local IP address set! : eNoLocalIpConfiguredThe Call Plug-in only supports unsecured devices. If the CUCM device is configured otherwise, this may prevent the Call Plug-in from downloading the device configuration file.
Recommended Action Check if there is network connectivity between the Call Plug-in and the CUCM. Also check if the device configuration file contains this line: <capfAuthMode>0</capfAuthMode>. If the value is different than 0, then the device in CUCM has been set up as secured, which is not supported by the Call Plug-in. To remedy this, change Device > Phone > <username> > Certification Authority Proxy Function (CAPF) Information > Certificate Operation to No Pending Operation.
Search FAQs and Troubleshooting
Search Troubleshooting
This section provides the following troubleshooting information:
•
•
Symptom When I do a global or local search I get the "Internal Server 500" error.
Possible Cause The service is not operational.
Recommended Action Check if the master Search Store, all slave Search Store nodes and the Index Store (if enabled) are operational. These are actions you can take:
On Search Store nodes, run this command as admin:
sudo service search status
On the Index Store node, run this command as admin:
sudo service searchcache status
Check if Solr administration console on each of the above nodes is accessible—see Accessing the Search Store Administration Console, and Accessing the Index Store Administration Console.
Possible Cause Misconfiguration.
Recommended Action Check if the master Search Store, all slave Search Store nodes and the Index Store (if enabled) are properly configured in portal-ext.properties on the App Server. These parameters must be set in accordance with your specific deployment:
solr.masters
solr.slave.region.1 (and other slaves if solr.slave.regions > 1)
search.cache.url
search.cache.post.url
search.cache.video.url
search.cache.social.url
search.cache.follower.url
Possible Cause Not enough disk space.
Recommended Action Take these steps:
Step 1
•
•
df -h
In the output, find the line that has "/" in the "Mounted on" column. A volume that is out of disk space will show 100% in the "Use%" column.
Step 2
service search stop
Step 3
Step 4
service search start
Possible Cause Server errors (500 Internal Server Error).
Recommended Action If you are getting "500 Internal Server Error" in the logs (the App Server logs, the master/slave Search Store logs, or the Index Store request logs) instead of 200 status codes for each request, then the machine may be out of disk space or the indexes may be corrupt.
If the machine is out of disk space, see the "Out of disk space" Possible Cause above.
Otherwise the indexes may be corrupt. Take these corrective steps:
Step 1
•
•
•
Another symptom is to see a core or more missing in the Index Store administrator portal. There should be a total of 5 cores linked as "Admin post", "Admin video", "Admin social", "Admin follower", and "Admin autocomplete". If any of those cores is missing, chances are that it is corrupt and you should see 404 error messages in the Index Store logs for the missing core.
Step 2
•
sudo service search stop
•
sudo service searchcache stop
Step 3
Step 4
•
sudo service search start
•
sudo service searchcache start
Symptom When I open My Library I get the "An unexpected error occurred" message.
Possible Cause For possible causes and recommended actions, see When I do a global or local search I get the "Internal Server 500" error. and take all steps related to Index Store.
Health and Performance Monitoring FAQs and Troubleshooting
•
•
Health and Performance Monitoring FAQs
•
Q.
A.
Health and Performance Monitoring Troubleshooting
This section provides the following troubleshooting information:
•
•
Symptom I restarted monit but monitoring does not seem to be working for that node.
Possible Cause The initialization of monit has not completed.
Recommended Action Wait for the initialization delay of monit (about 2 minutes).
Symptom I do not receive health data for a node.
Possible Cause If a node is marked as "Disabled" in the Topology page on the Director, monit does not perform checks on that node.
Recommended Action Enable the node.
Symptom This error appears for some nodes "CRITICAL: STATUS integrity: status failed (1) for /var/monit/check_integrity.sh"
Possible Cause This error is known to sometimes appear on new deployments on the Director's Health page. It may appear multiple times for multiple hosts and is caused by a pair of missing directories (/misc and /net), part of the autofs package, as evidenced by these log messages in the messages log:
Mar 21 12:49:26 ce2app1 integrity.sh: S.5....T. /opt/cisco/quad_synthetic/MonitorTest.pyMar 21 12:49:26 ce2app1 integrity.sh: missing /miscMar 21 12:49:26 ce2app1 integrity.sh: missing /netMar 21 12:49:26 ce2app1 integrity.sh: (FAIL) 69.52s Verifying RPM packages -> 3 package verification errorsMar 21 12:49:26 ce2app1 integrity.sh: Total time elapsed: 82.72sMar 21 12:49:26 ce2app1 integrity.sh: Total Tests Performed: 18Mar 21 12:49:26 ce2app1 integrity.sh: Tests Passed: 17Mar 21 12:49:26 ce2app1 integrity.sh: Tests Failed: 1Mar 21 12:49:26 ce2app1 integrity.sh: Compliance Rating: 94%Recommended Action Reinstall the autofs package on the affected nodes. Take these steps:
Step 1
Step 2
sudo yum reinstall autofs
Logs FAQs and Troubleshooting
Logs FAQs
•
•
•
•
•
Q.
A.
http://<director>/logs
Where <director> is the URL you use to access the Director web UI.
Use user admin and your unified access password to log in.
Alternatively, if you need to perform advanced actions with logs such as tracing logs in real time, log in to the Director node, go to /opt/logs and then enter the directory for the date you need.
Q.
A.
Note, however, that you can enable debugging in the App Server logs to cause the same logging to show in the normal App Server application logs.
Q.
A.
•
•
•
•
•
•
Q.
A.
Date/time Date and time the message was logged.
Host Originating host.
Process Name: quad
Log Level: Is always INFO.
Category: What type of security event this is. See What message categories are defined in the security and auditing log?
Thread Name: What thread within Tomcat did the event originate in.
Principal: User account this message pertains to.
Source: Where the message comes from, for example the IP address of the system performing the action.
Component: What area is affected.
Action: What type of action is taking place on the resource.
Resource: What is being affected (for example: Post, Message Boards).
Status: Success or Failure.
Reason: Additional information.
Q.
A.
Q.
A.
Q.
A.
Log messages about remote log settings appear in:
•
•
•
Logs Troubleshooting
Symptom I used "Apply All" or "Reset All" to modify the log trace levels of all App Server and Worker nodes but the changes does not seem to propagate to all nodes of those types.
Possible Cause Possible reasons include:
•
•
•
Director FAQs and Troubleshooting
Director FAQs
•
•
Q.
A.
Q.
A.
In the current version the following functional areas are managed by Certificate Management:
•
•
•
Director Troubleshooting
This section provides the following troubleshooting information:
Symptom I have uploaded a new security certificate using Application > Security but it does not seem to be taking effect.
Possible Cause Puppet did not restart the nodes that the certificates were pushed to.
Recommended Action Manually restart all App Server and Worker nodes.
Symptom After adding a node to the Topology page and deploying that node the node displays ERROR under Version Info ("Last config update: ERROR").
Possible Cause The last Puppet run has failed.
Recommended Action Log in to the node in question as admin and run the following command before you refresh the note statuses by clicking Refresh All on the Topology page:
sudo service puppet debug
If this command doesn't bring the status to normal, try the remaining Recommended Actions.
Possible Cause There is a significant time difference between the Director's clock and other nodes' clocks.
Recommended Action Synchronize the clocks on all nodes (preferably to a central NTP server). Do the following:
•
Aug 21 13:49:31 ds-director ntpd[17795]: time correction of 25374 seconds exceeds sanity limit (1000); set clock manually to the correct UTC time.To correct significant time differences, set the clocks manually using the date command (see next bullet).
•
Step 1
Step 2
Step 3
sudo date -s "DD MMM YYYY HH:MM:SS"
where DD MMM YYYY HH:MM:SS is the date followed by the time in 24-hour format. FOr example if you want to set the clock to Apr 19th 2012, 11:14:00 pm, type "19 APR 2012 11:14:00"
Or if you need to only set the time:
sudo date +%T -s "HH:MM:SS"
where HH:MM:SS is the time in 24-hour format.
After you have synchronized the clocks on all nodes, run sudo service puppet debug on all nodes that are showing ERROR on the Topology page.
Possible Cause There is a certificate mismatch between the node and the Director.
Recommended Action Take these steps to resolve the mismatch:
Step 1
salt-key -d <node FQDN>; service salt-master restart
puppetca --clean <node FQDN>; service puppetmaster restart
where <node FQDN> is the fully qualified domain name of the new node.
Step 2
rm -rf /etc/salt/pki/* ; service salt-minion restart
rm -rf /var/lib/puppet/ssl/* ; service puppet restart
Step 3
salt '<node FQDN>' cmd.run 'echo -n $(cat /opt/cisco/version.info)'
where <node FQDN> is the fully qualified domain name of the new node.
Step 4
quad.ecp-deploy.com: 3.4.0.09000.739,2013-06-06 11:57 UTC,OK
Worker FAQs and Troubleshooting
Worker FAQs
•
Q.
A.
•
•
•
•
•
Message Queue FAQs and Troubleshooting
•
Message Queue Troubleshooting
This section provides the following troubleshooting information:
•
•
•
Symptom Executing "service rabbitmq-server stop" doesn't seem to stop RabbitMQ.
Possible Cause The described case is a known defect.
Recommended Action Try executing sudo killall -u rabbitmq as admin.
Symptom I removed a node from a cluster and now rabbitmq is not functioning correctly.
Possible Cause The described case is a known defect.
Recommended Action As admin, stop rabbitmq on the remaining cluster nodes, then execute "rm -rf /opt/cisco/rabbitmq/data" and finally restart rabbitmq on all nodes.
Symptom RabbitMQ fails to start and shows this error "ERROR: failed to load application amqp_client: {"no such file or directory","amqp_client.app"}"
Possible Cause A RabbitMQ plug-in has frozen.
Recommended Action Run the following commands as admin to reset the amqp_client plug-in:
sudo rabbitmq-plugins disable rabbitmq_management
sudo service rabbitmq-server stop
sudo service rabbitmq-server start
sudo rabbitmq-plugins enable rabbitmq_management
Symptom On a fresh install, RabbitMQ fails to start with the following error the RabbitMQ logs: "Can't set short node name!\nPlease check your configuration\n"
Full log message:
/var/log/rabbitmq/startup_logActivating RabbitMQ plugins ...0 plugins activated:{error_logger,{{2012,12,1},{0,56,34}},"Can't set short node name!\nPlease check your configuration\n",[]}{error_logger,{{2012,12,1},{0,56,34}},crash_report,[[{initial_call,{net_kernel,init,['Argu ment__1']}},{pid,<0.20.0>},{registered_name,[]},{error_info,{exit,{error,badarg},[{gen_ser ver,init_it,6},{proc_lib,init_p_do_apply,3}]}},{ancestors,[net_sup,kernel_sup,<0.10.0>]},{ messages,[]},{links,[<0.17.0>]},{dictionary,[{longnames,false}]},{trap_exit,true},{status, running},{heap_size,377},{stack_size,24},{reductions,180}],[]]}Possible Cause The RabbitMQ rpm package was upgraded and RabbitMQ was stopped or restarted before puppet could replace the configuration.
Recommended Action Log in to the Message Queue node and then run the following commands as user admin:
sudo sed -i 's/sname/name/g' /usr/lib/rabbitmq/lib/*/sbin/*
sudo service puppet debug
This should start RabbitMQ without errors.
Symptom Both ActiveMQ nodes appear to be running as Master.
Possible Cause After NFS issues, the file locking required for the Master/Slave mechanism to function may not work properly, resulting in both ActiveMQ nodes running as Master at the same time. To be sure that you are experiencing this scenario, run the following command on both AMQ nodes:
netstat -an | grep 8161
You should see one of the nodes have output like this:
tcp 0 0 :::8161 :::* LISTENIf you see this on both nodes, you are running into this scenario.
Recommended Action Take this steps:
Step 1
Step 2
a.
sudo service monit stop
sudo service puppet stop
sudo service activemq stop
b.
sudo service monit stop
sudo service puppet stop
sudo service activemq stop
Step 3
•
•
a.
b.
activemq 5441 1 0 Nov30 ? 00:00:15 /opt/cisco/activemq/bin/linux-x86-64/wrapper /opt/cisco/activemq/bin/linux-x86-64/wrapper.conf wrapper.syslog.ident=ActiveMQ wrapper.pidfile=/opt/cisco/activemq/bin/linux-x86-64/./ActiveMQ.pid wrapper.daemonize=TRUE wrapper.lockfile=/var/lock/subsys/ActiveMQ......org.tanukisoftware.wrapper.WrapperSimpleApp org.apache.activemq.console.Main startroot 26063 10937 0 00:25 pts/0 00:00:00 grep activemqc.
d.
root 26063 10937 0 00:25 pts/0 00:00:00 grep activemq
Step 4
sudo service activemq start
Step 5
netstat -an | grep 8161
•
tcp 0 0 :::8161 :::* LISTEN•
The expected output should be similar to the following:
activemq 5441 1 0 Nov30 ? 00:00:15 /opt/cisco/activemq/bin/linux-x86-64/wrapper /opt/cisco/activemq/bin/linux-x86-64/wrapper.conf wrapper.syslog.ident=ActiveMQ wrapper.pidfile=/opt/cisco/activemq/bin/linux-x86-64/./ActiveMQ.pid wrapper.daemonize=TRUE wrapper.lockfile=/var/lock/subsys/ActiveMQactivemq 5443 5441 0 Nov30 ? 00:29:01 java -Dactivemq.home=../.. -Dactivemq.base=../.. -Djavax.net.ssl.keyStorePassword=password -Djavax.net.ssl.trustStorePassword=password -Djavax.net.ssl.keyStore=../../conf/broker.ks -Djavax.net.ssl.trustStore=../../conf/broker.ts -Dcom.sun.management.jmxremote -Dorg.apache.activemq.UseDedicatedTaskRunner=true -Dderby.storage.fileSyncTransactionLog=true -Dcom.sun.management.jmxremote.port=8002 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.cisco.ecp.Role=Messaging -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/cisco/activemq -XX:ErrorFile=/opt/cisco/activemq/diagnostic-info.quadjms-crash.txt -XX:OnOutOfMemoryError=../../bin/quadjms_diagnostics.sh -XX:OnError=../../bin/quadjms_diagnostics.sh -Xms128m -Xmx4096m -Djava.library.path=../../bin/linux-x86-64/ -classpath ../../bin/wrapper.jar:../../bin/run.jar -Dwrapper.key=apcwlYMEtZ4ACu7S -Dwrapper.port=32000 -Dwrapper.jvm.port.min=31000 -Dwrapper.jvm.port.max=31999 -Dwrapper.pid=5441 -Dwrapper.version=3.2.3 -Dwrapper.native_library=wrapper -Dwrapper.service=TRUE -Dwrapper.cpu.timeout=10 -Dwrapper.jvmid=1 org.tanukisoftware.wrapper.WrapperSimpleApp org.apache.activemq.console.Main startroot 26063 10937 0 00:25 pts/0 00:00:00 grep activemqStep 6
sudo service activemq start
Step 7
netstat -an | grep 8161
You should not get a return from this command. If you do, then you have run back into the issue again and you need to troubleshoot NFS to find the root cause.
Step 8
sudo service monit start
sudo service puppet start
Analytics FAQs and Troubleshooting
Analytics FAQs
•
Q.
A.
•
•
See Setting Log Trace Levels for detailed steps.
To log events such as sending or receiving to/from Message Queue, also set the QUAD_EVENT category to DEBUG.
Be sure to restart the Worker nodes for the log level to go into effect on that role.
Analytics log messages can appear in *_appserver.log, *_worker.log, or *_analyticsmrscheduler.log.
My Library FAQs and Troubleshooting
My Library Troubleshooting
This section provides the following troubleshooting information:
Symptom The My Library page does not seem to be responding to user actions: dialogs are not opening up, the Delete button does nothing, and so on.
Possible Cause A Javascript error has occurred on the page. Possible Javascript errors include:
•
•
$LAB.wait() error caught:SyntaxError: missing ; before statementRecommended Action Try reloading the page; if the problem persists, try loading the page with another web browser.
Framework FAQs and Troubleshooting
Framework FAQs
•
Q.
A.
•
•
See Setting Log Trace Levels for detailed steps.
Streams FAQs and Troubleshooting
Streams FAQs
•
Q.
A.
•
•
•
UC Integrations FAQs and Troubleshooting
•
UC Integrations Troubleshooting
This section provides the following troubleshooting information:
Symptom These errors appear when the user tries to switch the Cisco Call Plug-in from computer audio to desktop phone mode: [cwic] eUnknownFailure, [cwic] Login Error, and [cwic] unregisterPhone
Possible Cause The list of Unified Communications Manager (UCM) servers contains an IP address that does not correspond to a UCM server.
Recommended Action Sign in to Cisco WebEx Social as Administrator, go to Account Settings > Server > Common Configurations > WebDialer, find the offending entry in the list of Registered UCM Clusters and correct or remove it. Use the Cisco Call Plug-in log on the user computer to identify the offending UCM entry.
Open API FAQs and Troubleshooting
Open API FAQs
•
Q.
A.
•
•
See Setting Log Trace Levels for detailed steps.
OpenSocial FAQs and Troubleshooting
OpenSocial FAQs
•
Q.
A.
OpenSocial Troubleshooting
This section provides the following troubleshooting information:
Symptom When adding a OpenSocial application, the application title is displayed but the application content is not visible.
Possible Cause Unknown.
Recommended Action Refresh the browser window and the contents should appear.
Synthetic Monitoring FAQs and Troubleshooting
•
Synthetic Monitoring Troubleshooting
This section provides the following troubleshooting information:
•
•
•
Symptom Posts created during synthetic testing are not removed.
Possible Cause This issue can arise when you manually run the MonitorTest.py script with parameters "-l INFO" and then interrupt the script.
Recommended Action Sign in to Cisco WebEx Social using the ciscosyntheticmonitoruser user, go to My Library and delete the posts manually. Another options is to rebuild the Cisco WebEx Social indexes using these steps:
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
Step 7
Step 8
Symptom Posts created manually using the ciscosyntheticmonitoruser are not removed.
Possible Cause The Cisco WebEx Social indexes were not updated properly.
Recommended Action See the recommended action for Posts created during synthetic testing are not removed.
Symptom Synthetic monitoring reports "Search Created Post Not Found" or similar search related failure.
Possible Cause The value of the batch.manager.fetch.interval.secs advanced portal property exceeds 600 (10 min).
Recommended Action Lower the value of batch.manager.fetch.interval.secs to below 600. The recommended value is 300 (5 min).
Possible Cause An App Server node is running LDAP synchronization with a large LDAP database. In this case you see a corresponding alarm in the Director's Health page.
Recommended Action Wait for a few hours and the error should stop appearing. If it does not, then there is probably a different possible cause.
Using Jabber with Cisco WebEx Social FAQs and Troubleshooting
•
Using Jabber with Cisco WebEx Social FAQs
•
•
Q.
A.
•
•
•
•
•
In either case, the issue will need to be investigated between Cisco Jabber and the chat or unified communications server.
•
•
•
•
Q.
A.
•
•
After you do that, open the browser's console and start monitoring the output.
If you want to enable Dojo logging, execute the following in the browser's console: dojo.config.isDebug=true
SSL Termination FAQs and Troubleshooting
•
SSL Termination FAQs
•
•
Q.
A.
Q.
A.
•
•
SSL Termination Troubleshooting
This section provides the following troubleshooting information:
•
Symptom SSL sessions cannot be open to one or more App Server nodes.
Possible Cause The SSL termination settings on the Director (Application > Security > Cluster Security) are not applied to all App Server nodes.
Recommended Action Log in to each App Server node as admin and run this command:
sudo service puppet debug
Running this command normally corrects the problem. If it does not, it outputs information that can help you understand the reason for the failure.
Possible Cause The security certificates you uploaded using the Director (Application > Security > HTTPS/SSL) were not propagated to all App Server nodes or the quad service was not restarted.
Recommended Action Log in to each App Server node as admin and run this command:
sudo service puppet debug
If this does not help, manually restart the quad service on each App Server node. Run this command:
sudo service quad restart
Chat FAQs and Troubleshooting
Chat Troubleshooting
This section provides the following troubleshooting information:
•
Symptom A user cannot change their availability using Cisco Jabber after setting it in Cisco WebEx Social.
Possible Cause This may happen when the WebEx IM option has been selected as Chat server and the session priority in the Cisco WebEx Social chat server settings has been set higher than or equal to Cisco Jabber's default session priority value (127 as of this writing).
Recommended Action As administrator, go to Account Settings > Server > Common Configurations > Chat and lower Session Priority to below 127.
