Troubleshooting Jabber Like a TAC Engineer Robbie Horgan Test Engineer BRKUCC-2347

Transcription

1

2 Troubleshooting Jabber Like a TAC Engineer Robbie Horgan Test Engineer BRKUCC-2347

3 Agenda Troubleshooting Logins Troubleshooting Jabber Features Tools Used By Jabber

4 Which Jabber client generates the highest number of cases for TAC? Jabber for Mac Jabber for ios Jabber for Android Jabber for Windows

5 Agenda Troubleshooting Logins Service Discovery Logins Certificates Troubleshooting Jabber Features Tools Used By Jabber

6 Service Discovery Overview Enables clients to automatically detect and locate services Queries network for DNS SRV records Allows administrators to centrally configure Jabber with server side changes. Removes the need to use bootstrap file or having to instruct users to manually configure client Goal of discovery is to find a primary source of authentication

7 Typical Service Discovery Questions Seen By TAC The Jabber client is unable to discover our SRV record

8 Service Discovery Flow User Launches Client Client Determines Services Domain Client checks if network is inside or outside the firewall, and if Expressway is deployed Client sends cloud HTTP request to check for WebEx service Client starts monitoring for network changes No Yes Connect to Cisco WebEx Messenger service Connect to available services on corporate network Client looks for _cisco-uds Found? No Yes Connect to available services on corporate network Client looks for _cuplogin Found? No Yes Connect to available services on corporate network Is Expressway Available? Prompt user to manually enter connection details No Yes Omitted

9 Jabber Client Showing Service Discovery Failed

10 Service Discovery Failure Code and Logs All detailed discovery-related logs have [service-discovery] logger name in them Jabber client csf-unified.log Example: :07:16,829 WARN [0x ] [vices\impl\discoveryhandlerimpl.cpp(414)] [service-discovery] [DiscoveryHandlerImpl::callOnFailedDiscoveryResultOnDispatcherThread] - Discovery Failure -> (id) name :: (1005) ServiceDiscoveryNoSRVRecordsFound

11 Service Discovery Failure Codes ID Name UI message Description 1001 ServiceDiscoveryFailure Failed to discover network services. Unknown discovery failure 1002 ServiceDiscoveryAuthenticationFailure Your username or password is not correct. Failed to authenticate with CUCM (9.0+) 1003 ServiceDiscoveryCannotConnectToCucmServer Cannot communicate with the server. Cannot connect to CUCM (9.0+) 1004 ServiceDiscoveryNoCucmConfiguration Failed to discover network services. CUCM server is misconfigured 1005 ServiceDiscoveryNoSRVRecordsFound Failed to discover network services. No SRV records are found 1006 ServiceDiscoveryCannotConnectToEdge Cannot communicate with the server. Cannot connect to EDGE server 1007 ServiceDiscoveryNoNetworkConnectivity Cannot communicate with the server. No network connectivity

12 Failed nslookup of SRV record

13 Packet Capture of Service Discovery Failed

14 Configure SRV in DNS

15 Configure SRV in DNS

16 Configure SRV in DNS

17 Configure SRV in DNS

18 Successful nslookup of SRV record

19 Successful Service Discovery

20 Packet Capture of Successful Service Discovery

21 Debugging Service Discovery Cache File Located in %appdata%\cisco\unified Communications\Jabber\CSF\Config\service-location.xml <?xml version="1.0"?> <UCServices> <DomainName>ciscolive.com</DomainName> <UCService> <type>cucm</type> <connectioninformation> <name>_cisco-uds</name> <scope>unknown</scope> <address>cucm9.ciscolive.com</address> <protocol>_tcp</protocol> <port>8443</port> </connectioninformation> </UCService> </UCServices>

22 Reset Jabber to Install state ProgramData Users Local Profile Roaming Profile [USERNAME] AppData Local Unified Communications Jabber Roaming Jabber can be reset to Install state by removing both roaming and local profiles. Jabber will reinitialize on next startup CSF Unified Communications Jabber CSF Contacts History Logs Photo Cache Config Credentials CustomEmoticons Security Erase Jabber folders

23 Agenda Troubleshooting Logins Service Discovery Logins Certificates Troubleshooting Jabber Features Tools Used By Jabber

24 Common Login Questions Seen By TAC Users are unable to login for a new deployment. Only this one user is unable to login. All users suddenly can not login.

25 Incorrect Password Entered by End User

26 Check End User Page Login

27 Check End User Page Login

28 What Do the Logs Show? User is not licensed IM&P Server - EPASSoapXXXXX.log (Client Profile Agent Logs) :21:54,169 INFO [http-bio-443-exec-13] handlers.loginhandler - prelogin:querystring=execute PROCEDURE ucsoapprelogin('estaal','cup9',' ','cup9.ciscolive.com'); :21:54,170 DEBUG [http-bio-443-exec-13] imdb.imdbgeneralaccessorutil - getvalidappusersresultset -- query: SELECT * FROM validappusers WHERE userid='estaal'; :21:54,188 WARN [http-bio-443-exec-13] handlers.loginhandler - prelogin:prelogin reasoncode=failure. User either not CUP licensed or not found in database

29 Ensure User is Licensed For CUP 8 Users do not have proper licensing configured In Communications Manager 8.x this is under System > Licensing > Capabilities Assignment Verify in CUCM for the specific user:

30 Ensure User is Licensed For IM&P 9.x/10.x In Communications Manager 9.x this is under User Management > End User Verify in CUCM for the specific user:

31 Verify Connection To CUCM & Services Are Started Ensure System > CUCM Publisher shows green checkmarks. If not correct issue accordingly. Verify the Cisco UP Sync Agent is started on the IM&P Publisher Server. Verify this on the serviceability page under Tools > Control Center Feature Serivces

32 Verify User Licensing Has Synced From CUCM Verify licensing has replicated to IM&P server (under User Management > End User): If the information in IM&P does not match what is in CUCM, try restarting the Cisco UP Sync Agent to force a sync. Check Sync Status System (System > CUCM Publisher)

33 Check End User Page Login

34 What Do the Logs Show? IM&P Server - EPASSoapXXXXX.log (Client Profile Agent Logs) Wrong Password <ns1:login client-version=" " client-type="cupc" force="true"><ns1:username>estaal</ns1:username><ns1:password>...</ns1:password></ns1:login> :26:10,417 INFO [http-bio-443-exec-17] handlers.loginhandler - prelogin:querystring=execute PROCEDURE ucsoapprelogin('estaal','cup9',' ','cup9.ciscolive.com'); :26:10,417 DEBUG [http-bio-443-exec-17] imdb.imdbgeneralaccessorutil - getvalidappusersresultset -- query: SELECT * FROM validappusers WHERE userid='estaal'; :26:10,549 INFO [http-bio-443-exec-17] handlers.loginhandler - Wrong credential for : estaal IMS result code:1

35 IMS Result Codes 1) Wrong Credentials :26:10,549 INFO [http-bio-443-exec-17] handlers.loginhandler - Wrong credential for : estaal IMS result code:1 2) Account locked by Admin :34:37,576 INFO [http-bio-443-exec-7] handlers.loginhandler - Administratively locked for : estaal IMS result code:2 3) Account Hack lock :38:16,259 INFO [http-bio-443-exec-20] handlers.loginhandler - Hack locked for : estaal IMS result code:3 4) Account locked due to inactivity 7) User inactive in LDAP :42:53,284 INFO [http-bio-443-exec-12] handlers.loginhandler - End user status is INACTIVE in LDAP for : estaal IMS result code:7

36 Make Sure LDAP Authentication User Connects CUCM > System > LDAP > LDAP Authentication

37 Jabber Unable to Communicate With Server

38 Make Sure DNS Works Client PC must resolve what is configured under System > Cluster Topology

39 Verify Correct Services Started For Login Verify the XCP Connection Manager and XCP Authentication Server are started on the IM&P Publisher Server. Verify this on the serviceability page under Tools > Control Center Feature Services

40 Client Profile Agent Logs For Successful Login IM&P Server - EPASSoapXXXXX.log (Client Profile Agent Logs) Successful Login <ns1:login client-version=" " client-type="cupc" force="true"><ns1:username>estaal</ns1:username><ns1:password>...</ns1:password></ns1:login> :10:01,564 INFO [http-bio-443-exec-12] handlers.loginhandler - prelogin:querystring=execute PROCEDURE ucsoapprelogin('estaal','cup9',' ','cup9.ciscolive.com'); :10:01,709 INFO [http-bio-443-exec-12] handlers.loginhandler - dologin:ims login result is success for estaal IMS result code: :10:01,713 INFO [http-bio-443-exec-12] handlers.loginhandler - SOAP Login was successful

41 Agenda Troubleshooting Logins Service Discovery Logins Certificates Troubleshooting Jabber Features Tools Used By Jabber

42 Common Certificate Questions Seen By TAC I got my certificate signed by a CA; why do I still get a prompt?

43 Certificate Error Message As Seen By End User

44 How to see it in the logs Jabber client csf-unified.log :35:50,729 DEBUG [0x00000ae4] [src\cert\common\certificatedata.cpp(130)] [csf.cert] [cert::certificatedata::parsesubjectcnfield] - Subject CN field : CUP9.ciscolive.com :35:50,745 DEBUG [0x00000ae4] [ls\src\cert\utils\altnameparser.cpp(331)] [csf.cert.utils] [cert::altnameparser::verify] - Looking for match with CUP :35:50,745 ERROR [0x00000ae4] [ls\src\cert\utils\altnameparser.cpp(375)] [csf.cert.utils] [cert::altnameparser::verify] - No Match Found :35:50,745 ERROR [0x00000ae4] [ls\src\cert\common\certverifier.cpp(267)] [csf.cert] [cert::certverifier::buildcertresult] - Verification of identity: 'CUP9' failed :35:50,745 INFO [0x00000ae4] [mmon\platformverificationhandler.cpp(42)] [csf.cert] [cert::platformverificationhandler::handleplatformverificationresultsynchronously] - Verification result : FAILURE reason : [CN_NO_MATCH]

45 Login Certificate Flow Port 8443 Tomcat Certificate (Client Validates Certificate) Continue Login Process

46 CUCM Tomcat Certificate CUCM > System > Server

47 Login Certificate Flow Port 8443 Tomcat Certificate (Client Validates Certificate) Continue Login Process

48 Instant Messaging & Presence Tomcat Certificate CUCM User Management > User Settings > UC Service

49 Login Certificate Flow Port 5222 cup-xmpp Certificate (Client Validates Certificate) Continue Login Process

50 Instant Messaging & Presence XMPP certificate IM&P > System > Cluster Topology

51 Login Certificate Flow Port 8443 Tomcat Certificate (Client Validates Certificate) Complete Connection

52 Unity Connection Tomcat Certificate CUCM User Management > User Settings > UC Service

53 Agenda Troubleshooting Logins Troubleshooting Jabber Features Directory Integration Desk Phone Control and Presence Jabber for ios, Android and Mac Tools Used By Jabber

54 Common Directory Questions Seen By TAC Why am I unable to search for users? Why is the IM address incorrect? All my users show offline, even though they are online. When I add a contact, they disappear immediately. My contacts phone number does not show.

55 Directory Not Connecting

56 Explanation of jabber-config.xml file Centrally configure parameters for Jabber Allows administrators to make changes to client Prevent the need of a COP file

57 Sample jabber-config.xml file <?xml version="1.0" encoding="utf-8"?> <config version="1.0"> <Directory> <DirectoryServerType>EDI</DirectoryServerType> <PrimaryServerName> </PrimaryServerName> <ServerPort1>3268</ServerPort1> <UseWindowsCredentials>0</UseWindowsCredentials> <SearchBase1>dc=example, dc=com</searchbase1> </Directory> </config>

58 Directory Search Working

59 Unable to add contact If you try to add a contact, but they immediately disappear from the contact list, this points to an issue with the subscriber message on the IM&P server. There are 2 common issues that TAC sees User trying to be added to the contact list is not licensed for IM&P User A tried to add User B, User A is associated to server A and User B is associated to server B. Server A and B and in different subnets. If the communication type under System > Cluster Topology > Settings in IM&P is set to MDNS, users on the different servers will not be able to add each other. The router communication type should be set to Router-to-Router when the servers are in different subnets.

60 Incorrect IM Address

61 Sample jabber-config.xml file <?xml version="1.0" encoding="utf-8"?> <config version="1.0"> <Directory> <DirectoryServerType>EDI</DirectoryServerType> <PrimaryServerName> </PrimaryServerName> <ServerPort1>3268</ServerPort1> <UseWindowsCredentials>0</UseWindowsCredentials> <SearchBase1>dc=example, dc=com</searchbase1> </Directory> </config> <UserAccountName>telephoneNumber</UserAccountName>

62 Correct IM Address

63 Incorrect Phone Number

64 Sample jabber-config.xml file <?xml version="1.0" encoding="utf-8"?> <config version="1.0"> <Directory> <DirectoryServerType>EDI</DirectoryServerType> <PrimaryServerName> </PrimaryServerName> <ServerPort1>3268</ServerPort1> <UseWindowsCredentials>0</UseWindowsCredentials> <SearchBase1>dc=example, dc=com</searchbase1> </Directory> </config> <BusinessPhone>ipPhone</BusinessPhone>

65 Correct Phone Number

66 Agenda Troubleshooting Logins Troubleshooting Jabber Features Directory Integration Desk Phone Control and Presence Jabber for ios, Android and Mac Tools Used By Jabber

67 Typical Desk Phone Questions Seen By TAC All users are unable to get phone control. This one new user has no phone control. We do not see phone presence for our users.

68 Jabber Client Showing No Phone Control

69 Jabber Desk Phone Control CUCM > User Management > End User

70 Jabber Desk Phone Control CUCM > Device > Phone

71 Jabber Controlling 99xx/89xx phone CUCM Server - SDL001_200_XXXXX.txt (CTI Manager Logs) AppInfo [CTI-APP] [CTIHandler::OutputCtiMessage ] CTI FailureResponse ( seq#=3 result= description=user not configured to access device that supports Connected Transfer and Conference feature) CUCM > User Management > End User

72 CTI Logs Showing Login Timeout CUCM Server - SDL001_200_XXXXX.txt (CTI Manager Logs) 14:32: AppInfo CTIHandler(2,200,22, ) [CTI-APP] [CTIHandler::OutputCtiMessage ] CTI ProviderOpenCompletedEvent (seq#=370) provider id= CM Version= error code= description=directory login failed - timeout enableipv6=0 NoOfDaysPwdToExp=

73 Check LDAP Authentication Port System > LDAP > LDAP Authentication

74 Check LDAP Authentication Port System > LDAP > LDAP Authentication

75 Jabber Client Showing Phone Control

76 Phone Presence Not Working?

77 Check the SIP Trunk to IM&P Ensure that the SIP Trunk to IM&P on CUCM is resolvable CUCM > Device > Trunk

78 Is PublishEPA being created? CUCM Server - SDL001_100_XXXXX.txt (CCM Manager Logs) SdlSig PublishInd restart0 PublishEPA(1,100,106,2) PublishManager(1,100,105,1) 1,100,13,18.14^ ^SEP006440B580CE [R:N- H:0,N:1,L:0,V:0,Z:0,D:0] users.size()=1 users=estaal; pattern= numplanpkid=5fe08c87-d701- ddb9-38f7-eaab51652ae0 devicepkid=681733d4-c a e4552d mobilenumber= model=cisco-cp8961/9.3.2 state=2 isdnd=f firstregisterdevice=f devicemac=006440b580ce If not, check the line association in CUCM CUCM > User Management > End User

79 Phone Presence Working

80 Agenda Troubleshooting Logins Troubleshooting Jabber Features Directory Integration Desk Phone Control and Presence Jabber for ios, Android and Mac Tools Used By Jabber

81 Common Questions Seen By TAC Why is my mobile client unable to log in? Why can I search with Jabber for Windows, but none of the other clients?

82 Jabber for ios/android Unable to Login

83 What The Logs Show Jabber client Jabber log INFO [3b09218c] - [JabberWerx][log] [LoginMgr]: #1, connectonretrievedserverinfo login, cup:cup9.ciscolive.com INFO [3b09218c] - [JabberWerx][log] [CupSoapCli]: login cup async, server:cup9.ciscolive.com, user:******, ver: INFO [7dc0000] - [JabberWerx][log] [CupSoapCli]: CupSoapClientImpl::Login, endpoint: INFO [7dc0000] - [JabberWerx][log] [CupSoapCli]: Login() result, prim:, backup:, reason:soap 1.2 fault: SOAP-ENV:Sender[no subcode] "Host not found Detail: get host by name failed in tcp_connect()

84 Where Mobile Client Gets Server Information Mobile device must resolve what is configured under System > Cluster Topology

85 User Logged In

86 Unable to Search Directory

87 Be Sure jabber-config.xml BDI Section Is Configured <?xml version="1.0" encoding="utf-8"?> <config version="1.0"> <!-- LDAP Directory configuration for non-windows platform clients --> <Directory> <DirectoryServerType>BDI</DirectoryServerType> <BDIPrimaryServerName>XX.XX.XX.XX</BDIPrimaryServerName> <BDIServerPort1>3268</BDIServerPort1> <BDIPresenceDomain>example.com</BDIPresenceDomain> <BDISearchBase1>dc=example, dc=com</bdisearchbase1> </Directory> </config>

88 Working Directory Search

89 Which information will TAC ask for when you open a case? a) Problem Report b) Version numbers (server and client) c) Time the problem occurred d) UserID of the user who is having an issue. All of the Above!

90 Agenda Troubleshooting Logins Troubleshooting Jabber Features Tools Used By Jabber Jabber-config.xml Understanding the Problem Report

91 Check if jabber-config.xml is Formatted Properly

92 jabber-config.xml Correct Formatting

93 jabber-config.xml File Generator

94 jabber-config.xml File Generator

95 jabber-config-user.xml This file takes precedence over the jabber-config.xml file Is meant for testing purposes only

96 Agenda Troubleshooting Logins Troubleshooting Jabber Features Tools Used By Jabber Jabber-config.xml Understanding the Problem Report

97 How To Create A PRT Windows Android

98 How To Create A PRT Mac ios

99 Jabber for Windows PRT Install Parameter

100 Jabber for Windows PRT Local and Downloaded Config

101 Jabber for Windows PRT Local and Internal memory Cache

102 Jabber for Windows PRT Low level Library logs

103 Jabber for Windows PRT General System Info

104 Jabber for Windows PRT Main Jabber Logs

105 Jabber for Windows PRT Jabber Version and logged in User Info

106 Jabber for Windows PRT Crash Dump

107 Jabber for Windows PRT How the PRT was created

108 Jabber for Windows PRT End-users Comments

109 Jabber for Mac PRT

110 Jabber for ios PRT

111 Conclusion

112 Participate in the My Favorite Speaker Contest Promote Your Favorite Speaker and You Could be a Winner Promote your favorite speaker through Twitter and you could win $200 of Cisco Press products (@CiscoPress) Send a tweet and include Your favorite speaker s Twitter handle <rohorgancisco> Two hashtags: #CLUS #MyFavoriteSpeaker You can submit an entry for more than one of your favorite speakers Don t forget to View the official rules at

113 Complete Your Online Session Evaluation Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card. Complete your session surveys though the Cisco Live mobile app or your computer on Cisco Live Connect. Don t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online

114 Continue Your Education Demos in the Cisco Campus Walk-in Self-Paced Labs Table Topics Meet the Engineer 1:1 meetings

115 Thank you

116

117 Troubleshooting Video

118 Common Video Questions Seen By TAC I am unable to make/receive video calls. Video works for everyone, except one user.

119 Video Call Not Working

120 Medianet Not Installed

121 Medianet Installed, Still No Video Ensure computer is tethered to phone If controlling 99xx/89xx, unplug camera Check for CDP/CAST traffic via packet capture Ensure video is enabled

122 Video Shows Connected, Still No Video Only H.264 is supported Ensure call is internal Check location/region settings on CUCM

123 Video Call Working

124 Contact Photos

125 Common Contact Photo Questions Seen By TAC Where does Jabber get the contact photo from? Why can t I see contact photos?

126 Packet Capture of Contact Photos Acquired Via URL

127 Contact Photos via Web Server <?xml version="1.0" encoding="utf-8"?> <config version="1.0"> <Directory> <PhotoUriSubstitutionEnabled>true</PhotoUriSubstitutionEnabled> <PhotoUriSubstitutionToken>sAMAccountName</PhotoUriSubstitutionToken> <PhotoUriWithToken> </Directory> </config>

128 Contact Photos via jpegphoto Jabber client csf-unified.log Working :28:20,255 DEBUG [0x ] [2g-person\src\main\person\Person.cpp(48)] [csf.person] [person::person::person] - Person (00F559A8) is created from record 03501E18 (Roster:asemin@ciscolive.com). Total(created/live): 2/ :28:20,520 DEBUG [0x000004e0] [ices\impl\contacts\personwatcher.cpp(30)] [personwatcherlogger] [PersonWatcher::onPersonDataChanged] - Person 00F559A8. hasphoto?: :28:20,520 DEBUG [0x ] [e\src\services\impl\contactimpl.cpp(456)] [csf-contact-impl] [ContactImpl::setHasPhoto] - Setting hasphoto to true for asemin@ciscolive.com Non-working :41:18,226 DEBUG [0x00000a64] [2g-person\src\main\person\Person.cpp(48)] [csf.person] [person::person::person] - Person (00F56128) is created from record 035BA1A8 (ActiveDirectory:jpeters@ciscolive.com). Total(created/live): 7/ :41:18,226 DEBUG [0x ] [ce\src\services\impl\contactimpl.cpp(89)] [csf-contact-impl] [ContactImpl::setPerson] - Set to person 00F56128(Justin Peters). Has photo? 0

129 Soft Phone Control

130 Soft Phone Not Registering TFTP server not set under Application > Legacy Clients > Settings in IM&P TFTP Traffic is Blocked (port TCP 6970 and UDP 69) Client is not able to resolve CUCM hostname in Config File SIP traffic being blocked (port 5060 and 5061) No CCMCIP profile No device association

131 Jabber Soft Phone Call Fails Out To PSTN Even though Never start calls with video is selected, video is only muted Video capabilities are sent to Telco Telco refuses the call Transfer Capability = Unrestricted Digital Need to add bearer-cap speech to the voice-port for outbound calls.

132 ISDN Se0/0/0:23 Q931: TX -> SETUP pd = 8 callref = 0x03BB Bearer Capability i = 0x8890 Standard = CCITT Transfer Capability = Unrestricted Digital Transfer Mode = Circuit Transfer Rate = 64 kbit/s Channel ID i = 0xAAAAAA Exclusive, Channel 22 Progress Ind i = 0x Origination address is non-isdn Calling Party Number i =, ' ' Plan:ISDN, Type:National Called Party Number i =, ' ' Plan:ISDN, Type:National ISDN Se0/0/0:23 Q931: RX <- CALL_PROC pd = 8 callref = 0x83EE Channel ID i = 0xAAAAAA Exclusive, Channel 22 ISDN Se0/0/0:23 Q931: RX <- DISCONNECT pd = 8 callref = 0x83EE Cause i = 0x82AF - Resource unavailable, unspecified ISDN Se0/0/0:23 Q931: TX -> RELEASE pd = 8 callref = 0x03BB