BusinessObjects Enterprise XI

Transcription

1 Securing Server Communication with SSL through the Central Configuration Manager Overview Contents This paper will discuss the steps needed to secure BusinessObjects Enterprise XI server communication over the Secure Sockets Layer (SSL). It will review creating a self-signed certificate for use, as well as using a signed certificate by submitting a request to an appropriate Certificate Authority for signing. The paper will discuss any differences in configuration between Windows and UNIX/Linux platforms. INTRODUCTION... 1 DEFAULT CONFIGURATION USING A SELF-SIGNED CERTIFICATE...1 Initial Configuration of SSLC.CNF File...1 Sample SSLC.CNF Configuration File... 2 SSLC Utility...2 Phase 1: Creating Key and Certificate Files for the Certificate Authority (CA) 2 Phase 2: Creating Key and Certificate Files for the Enterprise Servers... 4 CUSTOM CONFIGURATION USING A CORPORATE CERTIFICATE AUTHORITY 6 Overview of Configuration Process...6 FINDING MORE INFORMATION...8 3/30/ :29 AM Copyright 2007 Business Objects. All rights reserved.

2 Introduction Many corporations are faced with the need to secure all incoming and outgoing communications that deal with production data. While client to server communications can be secured via SSL, what provisions are in place for securing server-to-server communications in the same way? Enterprise XI offers the ability to use SSL to secure these transmissions so that all data transfer is secured according to such requirements. This document will discuss the process for configuring the Enterprise XI servers to communicate using SSL. Default Configuration Using a Self-Signed Certificate The Enterprise XI documentation describes the procedure for creating a self-signed certificate, and using that to generate the necessary files for utilizing server to server SSL. These steps involve using the SSLC utility, provided by RSA Security Inc., to configure server to server SSL. Initial Configuration of SSLC.CNF File SSLC.cnf is a configuration file that tells the SSLC tool where to store new certificates, where to find the random number file used for generating new private keys, and what attributes are required on a newly issued certificate. Here are the steps to configure the file: 1. Go to Start > Programs > Accessories > Command Prompt. 2. In the command prompt window type: cd\program Files\Business Objects\BusinessObjects Enterprise 11\win32_x86 3. Open the SSLC configuration file by typing: Notepad sslc.cnf On UNIX, the file will be located in the platform specific path for the Operating System being used. For example, for Sun Solaris the path would be: $INSTALLROOT/bobje/enterprise11/solaris_sparc/sslc.cnf 4. Define the directories that will contain the SSLC certificate files. A sample Windows configuration file is included on the next page. 3/30/ :29 AM Copyright 2007 Business Objects. All rights reserved. Page 1

3 Sample SSLC.CNF Configuration File dir = C:/SSL # Location for everything certs = C:/SSL/certs # Location of issued certs are kept crl_dir = C:/SSL/crl # Location of issued crl are keptx database = C:/SSL/index.txt # database index file. new_certs_dir = C:/SSL/newcerts # default place for new certs. Certificate = C:/SSL/cacert.pem # CA certificate serial = C:/SSL/serial # Current serial number crl = C:/SSL/crl.pem # Current CRL private_key = C:/SSL/private/cakey.pem# Private key RANDFILE = C:/SSL/private/.rand # Private random number file The directory structure in this file should follow the format C:/Directory on Windows, and /directory/subdirectory on UNIX. The Directory Delimiter should be a forward slash / in both cases. SSLC Utility Once the file has been configured for use, as shown above, the SSLC utility can be used to generate certificates for usage in Enterprise XI environment. On Windows: Go to Start > Programs > Accessories > Command Prompt and type: cd\program Files\Business Objects\BusinessObjects Enterprise 11\win32_x86 On Solaris: $INSTALLROOT/bobje/enterprise11/solaris_sparc/sslc The steps for creating the necessary certificate files are included below. Phase 1: Creating Key and Certificate Files for the Certificate Authority (CA) The steps below explain the process for creating a self-signed Certificate Authority (CA) Certificate and associated private key, as well as a Server Certificate signed by this CA for use with the Enterprise XI Servers. 1. Create a CA certificate request called cacert.req and the associated private key file (privkey.pem) sslc req config sslc.cnf new out cacert.req 3/30/ :29 AM Copyright 2007 Business Objects. All rights reserved. Page 2

4 This invokes the SSLC program with the request parameter, directs the application to the config file to use for reference, and specifies a new request with an output file of cacert.req in the same directory. 2. Decrypt the private key file to be used for certificate signing. sslc rsa in privkey.pem out cakey.pem This command passes the encrypted private key file as input, and outputs a file called cakey.pem, which is the decrypted private key file. 3. Sign the CA certificate in order to generate an appropriate server certificate: sslc x509 in cacert.req out cacert.pem req signkey cakey.pem days 365 This invokes the SSLC utility to take the certificate request as input and output a certificate called cacert.pem, signed with the decrypted private key, and valid for 365 days. 365 days is the maximum amount of validity for the certificate. 4. At this point the files created by SSLC need to be moved into the appropriate directories for use in generating the Server certificate used by the Enterprise servers. Refer to the SSLC.CNF file for the following information: certificate = $dir/cacert.pem private_key = $dir/private/cakey.pem Move the cacert.pem and the cakey.pem into the appropriate directories (C:/SSL/cacert.pem and C:/SSL/private/cakey.pem in this example). 5. Create a file with the name specified by the sslc..cnf file s database setting. By default, this file is $dir/index.txt and may be empty. In this example it is C:/SSL/index.txt. 6. Create a file with the name specified by the sslc.cnf file s serial setting and ensure that the file provides an octet-string serial number (in hexadecimal format). Make the number larger to generate more certificates in the future. A number like would provide ample capacity. This number will increment as more certificates are generated. 7. Create the directory specified by the sslc.cnf file s new_certs_dir setting. In this example it is C:/SSL/newcerts. 3/30/ :29 AM Copyright 2007 Business Objects. All rights reserved. Page 3

5 At this point you are ready to generate the certificate files the Enterprise servers will use. Phase 2: Creating Key and Certificate Files for the Enterprise Servers 1. Create a certificate request and a private key by typing the following command: sslc req config sslc.cnf new out servercert.req 2. Make a copy of the private key copy privkey.pem server.key 3. Sign the certificate with the CA certificate by typing the following command: sslc ca config sslc.cnf days 365 out servercert.pem in servercert.req 4. Convert the certificates from PEM to DER encoded certificates in the following fashion: sslc x509 in cacert.pem out cacert.der outform DER sslc x509 in servercert.pem out servercert.der outform DER 5. Now create a text file to contain the plain text passphrase used to decrypt the generated private key: writing new private key to 'privkey.pem' Enter PEM pass phrase: Verifying password - Enter PEM pass phrase: This is the private key the system prompts for during the certificate creation process. 6. Store the following key and certificate files in a secure location (under the same directory) that can be accessed by the machines in your BusinessObjects Enterprise deployment: The trusted certificate file (cacert.der) The generated server certificate file (servercert.der) The server key file (server.key) The passphrase file (passphrase.txt) This location will be used to configure SSL for the CCM and your Web Application Server. Phase 3: Configuring the SSL Protocol 3/30/ :29 AM Copyright 2007 Business Objects. All rights reserved. Page 4

6 Configure the SSL Protocol in the CCM: 1. In the CCM on Windows, right-click a server that is in the Stopped state, click Properties > Protocol. 2. Provide the file path for the directory where you stored the key and certificate files. Make sure you provide the directory for the machine that the server is running on. 3. Repeat steps 1-2 for all servers. Configure the SSL Protocol for the Web Application Server: If using a J2EE web application server, run the Java SDK with the following system properties set: -Dbusinessobjects.orb.oci.protocol=ssl -DcertDir=c:/ssl -DtrustedCert=cacert.der -DsslCert=servercert.der -DsslKey=server.key -Dpassphrase=passphrase.txt For Tomcat on Windows, click Start > Programs > Tomcat > Tomcat configuration > Java and set this under the Java Options section. On UNIX, these properties may be set in the Catalina.sh file. Please consult your vendor specific documentation for information regarding the setup on other supported Java Application Servers. Configure the SSL Protocol for the.net and/or COM SDK: 1. Click Start > Programs > Accessories > Command Prompt. 2. Run the sslconfig utility. By default, this application is located at C:\Program Files\Business Objects\BusinessObjects Enterprise 11\win32_x86. Here is the syntax to use for the utility: sslconfig dir (certdir) mycert (sdkcert) rootcert (rootcert) mykey (privatekey) passphrase (passphrase) protocol (protocol) In the previous example the command would look like: sslconfig dir C:/SSL mycert servercert.der rootcert cacert.der mykey server.key passphrase passphrase.txt protocol ssl 3. To review the settings committed type: sslconfig display This command outputs the following: ********************** COM/.Net SDK 3/30/ :29 AM Copyright 2007 Business Objects. All rights reserved. Page 5

7 ********************** ===== Begin Current SSL Settings ===== CommunicationProtocol = ssl SSLCertDirectory = C:/SSL SSLCertificate = servercert.der SSLTrustCertificate = cacert.der SSLKey = server.key SSLPassphrase = passphrase.txt ===== End Current SSL Settings ===== Your Business Objects Enterprise 11 Deployment is now configured for Server to Server SSL. Custom Configuration Using a Corporate Certificate Authority The configuration listed above may be acceptable in a large number of deployments; however, there may be scenarios in which using a self-signed certificate and CA does not conform to corporate standards. In this scenario we will review the process for creating a certificate signing request, and submitting it to a corporately sponsored CA. This could be an internal CA managed by a security applications team, or a commercial CA such as Thawte or etrust. Overview of Configuration Process The steps for configuration are primarily the same as the method listed above, with a few notable exceptions. To complete this process, we will use the SSLC tool to ensure the Base 64 encoded certificates (CER/DER) generated by common certificate authorities are in line with the DER format required by the CCM. 1. Begin by configuring the SSLC.CNF file as outlined by the table shown above. 2. Execute the command and follow the directions to generate the servercert.req file: sslc req config sslc.cnf new out servercert.req 3. Make a copy of the private key generated: copy privkey.pem serverkey.pem 4. Submit the Certificate Signing Request to your preferred Certificate Authority for signing. Retrieve the certificate in Base 64 encoded format. The extension will typically be CER or DER. 5. Retrieve the CA Certificate that identifies the signing authority. This is also typically stored in CER/DER format. 3/30/ :29 AM Copyright 2007 Business Objects. All rights reserved. Page 6

8 6. Copy the two files to the directory containing the SSLC utility. 7. Convert the two files to ensure they are in the proper format using the following commands: sslc x509 in cacert.cer out cacert.der outform DER sslc x509 in servercert.cer out servercert.der outform DER 8. Now create a text file to contain the plain text passphrase used to decrypt the generated private key: writing new private key to 'privkey.pem' Enter PEM pass phrase: Verifying password - Enter PEM pass phrase: This is the private key the system prompts for during the certificate creation process. 9. Store the following key and certificate files in a secure location (under the same directory) that can be accessed by the machines in your BusinessObjects Enterprise deployment: The trusted certificate file (cacert.der) The generated server certificate file (servercert.der) The server key file (server.key) The passphrase file (passphrase.txt) This location will be used to configure SSL for the CCM and your Web Application Server. 10. Complete the steps outlined in Phase 3: Configuring the SSL Protocol This completes the procedure for configuring Enterprise XI for use with Server to Server SSL. At this time you should be able to start all servers in the Enterprise environment successfully and log on to the system. DISCLAIMER The third party products discussed in this white paper were not fully tested in conjunction with BusinessObjects Enterprise XI prior to its release. Officially supported Enterprise XI platforms are listed in the text file Platforms.txt found in the \docs folder of the Enterprise XI CD as well as the updated supported platforms white papers: boe_xi_supported_platforms_mlb.pdf boe_xi_supported_platforms_aix.pdf boe_xi_supported_platforms_linux.pdf boe_xi_supported_platforms_solaris.pdf These white papers are found at The information in this document is provided as a courtesy to assist our customers with the configuration of our product in conjunction with these third party platforms. In the event issues arise with an unsupported configuration, there is no escalation support; however, they will be considered during the development of the next generation of our product. 3/30/ :29 AM Copyright 2007 Business Objects. All rights reserved. Page 7

9 Finding More Information For more information and resources, refer to the product documentation and visit the support area of the web site at: Installing and Configuring SSL Support SSL Configuration/How-To Apache Tomcat 3/30/ :29 AM Copyright 2007 Business Objects. All rights reserved. Page 8