SECURING DOMINO LDAP Open Mic June 10th 2015
AGENDA Background Domino Directory Assistance Domino LDAP Server Domino LDAP in a Post-Poodle World Questions 2
BACKGROUND We consider this presentation a continuation on the LDAP Configuration OpenMic Powell Pendergraft and Brandon Kutsch provided last summer The team strongly recommends folks go back and review previous presentation for additional Domino LDAP configurations / performance considerations Open Mic Webcast: LDAP Configuration - 30 July 2014 http://www.ibm.com/support/docview.wss?uid=swg27042283 3
We combed IBM for expertise across multiple product lines, Surfaced technical experts from different disciplines: Support, Development, Swat Across 3 different continents to represent the World Class team we assemble for the call today. Many thanks to the Software Engineers who contributed to make today's presentation possible!
DOMINO DIRECTORY ASSISTANCE Brandon Kutsch
DOMINO DIRECTORY ASSISTANCE Basic Directory Assistance setup SSL Secure DA setup Troubleshooting Additional Resources 6
DOMINO DIRECTORY ASSISTANCE Directory Server 7
DOMINO DIRECTORY ASSISTANCE Note: no x509 credentials allowed for binds Protocol Version selection within DA is no longer applicable after recent POODLE fixes sslv2 is disabled.. Please use the SSLCipherSpec= notes.ini to configure ciphers used during negotiation http://www-10.lotus.com/ldd/dominowiki.nsf/dx/sslcipherspec 8
9 SECURE DIRECTORY ASSISTANCE 1)Prior to attempting secure setup, configure DA over unencrypted 389 2)Obtain LDAP directory server's SSL trusted root certificate ask the LDAP admin or extract via openssl> s_client -connect ldapserver.com:636 3)Install trusted root into Domino KYR file Use Server Certifcate Admin DB for SHA1 https://www.ibm.com/support/docview.wss?uid=swg21449281 Or the new kyrtool (9x) SHA1andSHA2 based certs http://www-10.lotus.com/ldd/dominowiki.nsf/dx/kyrtool 4)For either SHA1/SHA2 roots, specify the KYR in Server's Ports->Internet Ports We use the Domino Server Doc to specify the SSL keystore when Domino acts as the SSL client, regardless if using Internet Sites OR Web Configuration view for Domino Server configurations. NOTE The Directory Assistance Test/Verify wizards are Java agents that utilize CACerts (ikeyman) and NOT the Domino Server KYR file
LDAPDebug=512 (when Domino acts as an LDAP Client) Debug_namelookup=16 (only collects DA/LDAP lookups) "LDAP GW" output showing lookups to LDAP servers Debug_directory_assistance=1 (requires server restart) Useful if directory assistance is not loading Webauth_verbose_trace=1 Can identify why a login failed or succeeded, very verbose NAMELOOKUP_PING_LDAP_RETRY=1 helpful in troubleshooting remote lookups Debug_SSL_All=1 DEBUG_SSL_HANDSHAKE=2 when troubleshooting 636 secure connections show xdir r (reloads DA) / show xdir d (outputs DA config currently in memory) console.log copy of Directory Assistance.nsf TROUBLESHOOTING DIRECTORY ASSISTANCE Manual NSD http://www-01.ibm.com/support/docview.wss?uid=swg21204263 10
ADDITIONAL RESOURCES How to allow Directory Assistance to communicate with an external LDAP server using SSL encryption http://www.ibm.com/support/docview.wss?uid=swg21249483 How can Domino be set up to work with Microsoft's Active Directory? http://www.ibm.com/support/docview.wss?uid=swg21293255 Problems using Directory Assistance LDAP wizards with SSL http://www.ibm.com/support/docview.wss?uid=swg21303960 http://www-10.lotus.com/ldd/dominowiki.nsf/dx/extract_the_root_certificate_from_a_signed_stamped_ssl_server_certificate http://www-10.lotus.com/ldd/dominowiki.nsf/xpviewcategories.xsp?lookupname=domino security Open Mic Webcast: LDAP Configuration - 30 July 2014 http://www.ibm.com/support/docview.wss?uid=swg27042283 11
DOMINO LDAP SERVER Bradley Ineichen
DOMINO LDAP SERVER Configuring Domino LDAP SSL Server setup LDAP Server Debug / References 13
DOMINO LDAP SERVER 14
CONFIGURING DOMINO LDAP The LDAP task runs automatically on the administration server for the primary IBM Lotus Domino Directory. If you wish to run LDAP on other servers, you must, run the LDAP task manually. Create a Server Configuration Document" and for the field "Use these settings as the default setting for all servers", choose Yes Customize the default LDAP service configuration. In most cases, the LDAP service default settings are adequate If you wish to allow clients to connect to the LDAP service over the Internet, you must register the servers DNS name and IP address with the Internet Service Provider that runs the LDAP service. To check whether you set up the LDAP service correctly, use an LDAP search utility such as ldapsearch provided with IBM Lotus Notes and Domino, to issue a query to the LDAP service. 15
CONFIGURING DOMINO LDAP Port and port security settings - Controls the ports LDAP clients can use to connect to the LDAP service, and the authentication methods enabled for each port This is set in the server document. Default: TCP/IP port 389/636 enabled for name-and-password authentication and for anonymous access. Choose fields that anonymous users can query via LDAP" - If the port settings allow anonymous access, controls which attributes anonymous LDAP users can search. "Allow LDAP users write access Controls whether LDAP users can modify a directory. By default LDAP modifications not allowed. "Rules to follow when this directory..." - Controls how the LDAP service responds when it encounters more than one entry or naming rule that applies to an LDAP add, modify, or compare operation, the default is don't carry out the operation. "Timeout" Controls the maximum time allowed to process an LDAP search, there is not time limit set by default. 16
CONFIGURING DOMINO LDAP Maximum number of entries returned" - Controls the maximum number of entries that the LDAP service can return in response to an LDAP search query. By default there are no limits. "Minimum characters for wildcard search" - Controls the minimum number of characters users must place before the first wildcard in a substring search filter, must use at least 1 character "Enforce schema" - This controls whether directory modifications through LDAP must conform to the schema. By default the current schema is enforced "DN Required on Bind" Controls whether the LDAP service requires clients to log on with distinguished names for name-and-password authentication.. Distinguished logon names not required by default "Encode results in UTF8 for LDAP-v2 clients" - This setting controls how the LDAP service returns results to LDAP v2 clients, either OUTFIT or UTF8. Results are returned in OUTFIT to v2 clients by default. "Allow dereferencing of aliases on search requests" This setting Enables limited alias dereferencing for LDAP search requests. This setting is disabled by default. 17
SSL SERVER SETUP Use Server Certifcate Admin DB for SHA1 http://www.ibm.com/support/docview.wss?uid=swg21268695 Or the new kyrtool (9x) SHA1andSHA2 based certs http://www-10.lotus.com/ldd/dominowiki.nsf/dx/kyrtool Ask the Experts session: Ask us anything about SSL and Certificates - December 2014 http://www-01.ibm.com/support/docview.wss?uid=swg27044211 Note: There may be issues with old Domino 85x MD5 based certs consider using OpenSSL and Kyrtool to create SHA1 certs https://www.ibm.com/support/docview.wss?uid=swg21680147 http://www-10.lotus.com/ldd/dominowiki.nsf/xpviewcategories.xsp?lookupname=domino security 18
LDAP SERVER DEBUG / REFERENCES LDAPDebug=7 (LDAP server) This shows all LDAP server activity debug_namelookup=1 console command show stat ldap http://ibm.co/1dq4hxj Test it with a ldap client like LDAPSEARCH.exe http://www-01.ibm.com/support/docview.wss?uid=swg27002627nsd Open Mic Webcast: LDAP Configuration - 30 July 2014 http://www.ibm.com/support/docview.wss?uid=swg27042283 19
DOMINO LDAP IN A POST-POODLE WORLD Analyn Policarpio Powell Pendergraft David Workman
DOMINO LDAP IN A POST-POODLE WORLD "I fixed POODLE but broke LDAP" Interoperability (Sametime Case Study) Debug/Troubleshooting 21
I FIXED POODLE BUT BROKE LDAP POODLE fixes may result in mismatched hash algorithms, protocols, or ciphers between LDAP server and client Upgrading Domino and third party products/configuration to implement new security features POODLE fixes - some key changes: Disabled SSLv2 protocol Option to disable SSLv3 connection Introduced TLS 1.0 (Nov 2014) - for 8.5.x and 9.x. TLS 1.2 included in Domino 9.0.1 FP3 IF2 (May 2015) SHA-2 certificates Introduced in Domino 9.x Some LDAP servers upgraded their certificates to SHA-2 SSL Ciphers - Strong ciphers were introduced and weak ciphers were removed These are configured via notes.ini parameter - SSLCipherSpec= 22
I FIXED POODLE BUT BROKE LDAP Verify that the protocols used by the server and client match (SSLv2, SSLv3, TLS1.0, TLS1.2) Examples: Domino LDAP client offers TLS 1.0 while the LDAP server only uses TLS 1.2. LDAP server or client only uses SSLv3 Solutions: Upgrade Domino to a version that supports TLS 1.2 to match the LDAP server Update LDAP servers or appliances that use SSLv2 Upgrade the third party LDAP server or client side to disable SSLv2 Re-enable SSLv2 handshake on Domino (SSL_ENABLE_INSECURE_SSLV2_HELLO=1) Option available in 8.5.3 FP6 IF7 and 9.0.1 FP3 IF1 Not a recommended option 23
Update Cipher suite in use - 9.x - stronger cipher suites introduced - 9.0.1 FP3 IF2 includes stronger TLS 1.2 ciphers I FIXED POODLE BUT BROKE LDAP - 8.5.x and 9.x - We strongly recommend against using the RC4 ciphers in order to protect against the "RC4 Bar Mitzvah" attack. - RC4-MD5 and DES-CBC-SHA have been added to the list of weak ciphers. Import certificates from remote LDAP server - Customers may be updating certificates to implement POODLE fixes - Third party servers may now require connection using LDAPS (port 636) instead of LDAP (389) Update to SHA2 if possible, or recreate the keyring file of the server You may need to use OpenSSL and kyrtool.exe (available in latest versions of Notes Admin/Domino) to create a new SHA1 certificate to resolve MD5 cert issue. (SHA2 only possible in ND9x) 24
I FIXED POODLE BUT BROKE LDAP The Poodle updates remove SSLv2 from the Server code, but from the Directory assistance LDAP Tab there are still options for sslv2 handshakes when Domino is acting as the ssl client. We do not honor these options anymore and current versions of Notes/Domino will not make an outbound (client-side) connection with an SSLv2 ClientHello, as that is highly insecure and explicitly forbidden by RFC 6176. Please use the SSLCipherSpec= notes.ini instead http://www-10.lotus.com/ldd/dominowiki.nsf/dx/sslcipherspec LO85203 / SPR # DWON9X5L53 / TN1959341 http://www-01.ibm.com/support/docview.wss?uid=swg21959341 Domino reported a problem connecting as an LDAP client after applying IF7 for Domino 8.5.3 FP6 25
SAMETIME ACCESS TO DOMINO LDAP Having successfully implemented fixes to secure Domino internet protocols from POODLE vulnerabilities over SSL some administrators found that user authentication began to fail for Sametime community servers configured to use Domino as the LDAP server. Research and testing began to find that Domino LDAP servers rejected Sametime server requests for authentication over SSL, resetting the connection on each attempt. Further testing and Wireshark logging discovered that Sametime requests secured transactions initiated with SSLv2 handshakes. You can find Domino LDAP server versions disabling SSLv2 protocol support at the links below. Keep abreast of developments for Domino 9.0.x for further TLS news as noted above. http://www-10.lotus.com/ldd/dominowiki.nsf/dx/tls_1.2 http://www-10.lotus.com/ldd/dominowiki.nsf/dx/ibm_domino_tls_1.0 26
SAMETIME ACCESS TO DOMINO LDAP Sametime uses its own LDAP client code by design to initialize SSL handshakes with SSLv2. Domino LDAP servers upgraded to deal with the POODLE threats by design disallow initializing a handshake with SSLv2 resulting the continued refusal and packet resets. The solution lies in an upgrade to Sametime 9.0 HF1 for all Sametime 8.5x and 9.0 servers. Migrating Sametime: http://www-01.ibm.com/support/knowledgecenter/ssktxq_9.0.0/admin/migrate/upgr_st852x_intro.dita Sametime 9.0 HF1 link: http://www-01.ibm.com/support/docview.wss?uid=swg21656751 Use Sametime 9.0 HF1 protocol versions TLS1, TLS12 which can be set as default values in the TLS configuration setting via Sametime System Console (SSC). You will find the steps in the wiki link below. http://ibm.co/1cg8ugd Much less secure, not recommended you may set Domino LDAP to accept an SSLv2 ClientHello by upgrading to Domino LDAP to 9.0.1 FP3 IF1, and set the following flag in the notes.ini of the Domino LDAP server : SSL_ENABLE_INSECURE_SSLV2_HELLO=1 See the Open Mic on Sametime and POODLE issues. http://www-01.ibm.com/support/docview.wss?uid=swg27045127 27
DEBUG and TROUBLESHOOTING DEBUG commands that needs to be enabled in the Domino server captured in Server Console: DEBUG_SSL_ALL=x (0 = Debug Off, 1 = Little Information, 2 = More information, 3 = Full Information ) DEBUG_SSL_HANDSHAKE=2 IBM Domino (r) Server (64 Bit), Release 9.0.1, October 14, 2013 [14155850:00002-00001] 10/31/2014 17:23:41.07 SSL_Handshake> Protocol Version = TLS1.0 (0x301) [14155850:00002-00001] 10/31/2014 17:23:41.07 SSL_Handshake> TLS/SSL Handshake completed successfully Other troubleshooting steps to consider: -Get/install the LDAP server certificate from the LDAP admin or via OpenSSL - install using the Kyrtool.exe introduced in 9.x -Test SSL connections using OpenSSL openssl s_client -connect ldapserver:636 -SSL3 openssl s_client -connect ldapserver:636 -TLS1 openssl s_client -connect ldapserver:636 -TLS1_2 -Third party site to test the SSL of an LDAP server https://www.ssllabs.com/ssltest/index.html 28
DEBUG and TROUBLESHOOTING This shows the ciphers and a full ssl handshake: Example of a TLS 1.0 successful handshake. [03EC:000F-1AF0] 06/01/2015 11:04:34.89AM SSL_Handshake> Protocol Version = TLS1.0 (0x301) [03EC:000E-120C] 06/01/2015 11:04:34.89AM CompleteNTIRequest> Exit [03EC:000F-1AF0] 06/01/2015 11:04:34.89AM SSL_Handshake> KeySize = 128 bits [03EC:000F-1AF0] 06/01/2015 11:04:34.89AM SSL_Handshake> Current Cipher = 0x002F(RSA_WITH_AES_128_CBC_SHA) [03EC:000F-1AF0] 06/01/2015 11:04:34.89AM SSL_Handshake> SSLErr = 0 [03EC:000F-1AF0] 06/01/2015 11:04:34.89AM SSL_Handshake> Using resumed SSL/TLS session [03EC:000E-120C] 06/01/2015 11:04:34.89AM SSL_EncryptData> Asked to write 255 and wrote 293 [03EC:000F-1AF0] 06/01/2015 11:04:34.89AM SSL_Handshake> TLS/SSL Handshake completed successfully Handshake of Client and Server using TLS 1.2 [0150:000F-15E4] 26.03.2015 11:07:30,45 SSLEncodeClientHello> We offered SSL/TLS version TLS1.2 (0x0303) [0150:000F-15E4] 26.03.2015 11:07:30,61 SSLProcessServerHello> Server chose SSL/TLS version TLS1.2 (0x0303) Handshake of Client and Server using SSL 3.0 [0150:000F-15E4] 26.03.2015 11:53:48,46 SSLEncodeClientHello> We offered SSL/TLS version SSLV3.0 (0x0300) [0150:000F-15E4] 26.03.2015 11:53:48,46 SSLProcessServerHello> Server chose SSL/TLS version SSLV3.0 (0x0300) 29
DEBUG and TROUBLESHOOTING SSLCheckCertChain> Invalid certificate chain received Cert Chain Evaluation Status: err: 5950, Certificate is expired or not yet valid Connect Interrogation of Established SSL Session vs. Policy Failed Unable to get NTI SSL configuration or certificate information. Application tries to connect to Domino via SSL "int_mapsslerror> Mapping SSL error -5000 to 4176" [1CD8:0004-040C] 06/03/2015 04:17:32.11 PM LDAP server is unavailable ReturnCode=0x1C79 (Unknown error) (LO85203 / SPR # DWON9X5L53 -Domino reported a problem connecting as an LDAP client after applying IF7 for Domino 8.5.3 FP6) TN1959341 http://www-01.ibm.com/support/docview.wss?uid=swg21959341 Example of an unsuccessful handshake: [1EB8:0004-1BA8] 06/01/2015 11:21:30.89AM SSL_Handshake> Afterhandshake2 state 2 [1EB8:0004-1BA8] 06/01/2015 11:21:30.89AM SSL_Handshake> SSL Error:-6989 [1EB8:0004-1BA8] 06/01/2015 11:21:30.89AM int_mapsslerror> Mapping SSL error -6989 to 4165[SSLConnectionClosedError ] [1EB8:0004-1BA8] 06/01/2015 11:21:30 AM LDAP Server is NOT available. [1EB8:0004-1BA8] 06/01/2015 11:21:30 AM Error attempting to access the Directory *LDAPHostname.COM:636 (noavailable alternatives), error is LDAP Server is NOT available. 30
DEBUG and TROUBLESHOOTING Domino 8.5.x and 9.x with POODLE fix support TLS 1.0 http://www-01.ibm.com/support/docview.wss?uid=swg21687167 http://www-10.lotus.com/ldd/dominowiki.nsf/dx/ibm_domino_tls_1.0 Domino 9.0.1 FP3 IF2 - supports TLS 1.2 and added detailed logging for SSL/TLS connections http://www-01.ibm.com/support/docview.wss?uid=swg21657963 http://www-10.lotus.com/ldd/dominowiki.nsf/dx/tls_1.2 Installing Trusted Root Certificate into Domino SSL Key Ring (SHA-1/Dom 8.5.x) http://www-01.ibm.com/support/docview.wss?uid=swg21449281 SHA-2 Domino Keyring generation (SHA-2/Dom9.x) http://www-10.lotus.com/ldd/dominowiki.nsf/dx/domino_keyring TLS Cipher Configuration http://www-10.lotus.com/ldd/dominowiki.nsf/dx/tls_cipher_configuration Domino Security Wiki http://www-10.lotus.com/ldd/dominowiki.nsf/xpviewcategories.xsp?lookupname=domino security 31
DEBUG and TROUBLESHOOTING Application fails to connect to Domino via SSL http://www-01.ibm.com/support/docview.wss?uid=swg21198731 Security Bulletin: IBM Domino LDAP Server (CVE-2015-0117), SSLv2 (CVE-2015-0134) & Notes System Diagnostics (CVE-2015-0179) vulnerabilities http://www.ibm.com/support/docview.wss?uid=swg21700029 Domino 9.0.1 FP3 IF1 - introduced a new notes.ini parameter: SSL_ENABLE_INSECURE_SSLV2_HELLO=1 http://www.ibm.com/support/docview.wss?uid=swg21697359 http://www-10.lotus.com/ldd/dominowiki.nsf/dx/sslv2 Firefox users unable to connect to Domino-based certificate or self-signed secured Web sites after updating Firefox to version 31 https://www.ibm.com/support/docview.wss?uid=swg21680147 Protector cannot connect to Domino for LDAP over TLS http://www-01.ibm.com/support/docview.wss?uid=swg21697982 32
QUESTIONS? Press *1 on your telephone to ask a question. Visit our Support Technical Exchange page or our Facebook page for details on future events. To help shape the future of IBM software, take this quality survey and share your opinion of IBM software used within your organization: https://ibm.biz/bdxqb2 IBM Collaboration Solutions Support page http://www.facebook.com/ibmlotussupport IBM Collaboration Solutions Support http://twitter.com/ibm_icssupport