EJBCA Enterprise Cloud Edition CloudHSM Integration Guide

Transcription

1 EJBCA Enterprise Cloud Edition CloudHSM Integration Guide PRINT DATE:

2 Copyright 2019 PrimeKey Solutions Published by PrimeKey Solutions AB Solna Access, Sundbybergsvägen 1 SE Solna, Sweden To report errors, please send a note to support@primekey.com. Notice of Rights All rights reserved. No part of this guide may be reproduced or transmitted in any form by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. For more information on getting permission for reprints and excerpts, contact support@primekey.com. Notice of Liability The information in this guide is distributed on an As Is basis without warranty. While every precaution has been taken in the preparation of the guide, neither the authors nor PrimeKey shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the instructions contained in the guide or by computer software and hardware products described in it. Trademarks Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this guide, and PrimeKey was aware of a trademark claim, the designations appear as requested by the owner of the trademark. All other product names and services identified throughout this guide are used in editorial fashion only and for the benefit of such companies with no intention of infringement of the trademark. No such use, or the use of any trade name, is intended to convey endorsement or other affiliation with this guide.

3 Table of Contents Introduction... 5 Documentation...5 Multiple Crypto Tokens with AWS CloudHSM Create CloudHSM Cluster Use OpenSSL to Validate the HSM Initialize the CloudHSM Assigning the Security Group to the EJBCA Instance Configure the cloudhsm-client PKCS11 PIN Activate the Cluster Create a CloudHSM Crypto User Create a Keystore in the HSM with clienttoolbox Test with EJBCA ClientToolbox Create a CryptoToken in EJBCA Auto Activating a CloudHSM CryptoToken Appendix A - Restoring an HSM Backup to a New Instance Appendix B - Troubleshooting HSM Issues... 28

4 2019 PRIMEKEY 4 (4)

5 Introduction This guide is intended to show how to get the EJBCA Enterprise Cloud Edition integrated with AWS CloudHSM. The AWS CloudHSM is the newer offering from AWS based on Cavium, not to be confused with the SafeNetbased AWS CloudHSM Classic. For more information, refer to the AWS CloudHSM User Guide. The EJBCA Enterprise Cloud Edition (ECE) and AWS CloudHSM integration includes the following and more steps: 1 - Create CloudHSM Cluster Validate the HSM 3 - Initialize the CloudHSM Assign Security Group Configure the CloudHSM Client 7 - Activate the Cluster Create CloudHSM Crypto User Create Keystore Create CryptoToken in EJBCA If you already have a CloudHSM cluster configured, proceed to the step 4 - Assigning the Security Group to the EJBCA Instance and then 5 - Configure the cloudhsm-client. This will get the EJBCA ECE instance configured to talk to the CloudHSM cluster so that key creation can begin. Documentation EJBCA Enterprise Cloud Edition documentation is available on: EJBCA Enterprise Edition documentation is available on: Additional information on EJBCA Community Edition is available on: AWS Documentation AWS CloudHSM Documentation AWS CloudHSM User Guide 2019 PRIMEKEY 5 (5)

6 Multiple Crypto Tokens with AWS CloudHSM Typically, when using EJBCA and creating CryptoTokens, a CryptoToken will generally look at a slot in an HSM for its key material. When using CloudHSM, the CloudHSM client provided by AWS always looks at a single cluster of HSMs. This cluster is always referenced as slot 1 to the host accessing it. AWS' CloudHSM client does not allow a host to access more than one CloudHSM cluster at a time. EJBCA can have a multitier CAs, but because of the CloudHSM clients ability to only talk to a single HSM cluster, it is advised to have a dedicated offline root CA that can sign your issuing CAs. In this scenario, a Root CA would sign an Issuing CA as an external CA. Please see the EJBCA documentation for signing an external CA for more information on signing an external CA at the following URL: There are two ways to accomplish this. On a single host, or a second off-line Root CA host. Scenario 1 - Two instances: 1. Start two EJBCA Enterprise Cloud instances 2. Initialize the CloudHSM client on each instance to talk to each cluster. Information on this is located in the section "5 - Configure the cloudhsm-client". 3. Create the self signed Root CA on instance Create the issuing CA on instance 2 and set it to be signed by an External CA. 5. Download the CSR from the issuing CA on instance Sign the CSR on instance 1 with the Root CA. 7. Bring the signed certificate back to instance 2 and fulfill the waiting CAs CSR. Scenario 2 - Single instance 1. Start a single EJBCA Enterprise Cloud instance 2. Initialize the CloudHSM client on the instance to talk to the first CloudHSM cluster. Information on this is located in the section "5 - Configure the cloudhsm-client". 3. Create the self signed Root CA. 4. Reconfigure the CloudHSM client to talk to the second CloudHSM cluster. 5. Restart EJBCA/WildFly 6. Create the issuing CA and set it to be signed by an External CA. 7. Download the CSR from the issuing CA 8. Reconfigure the CloudHSM client to talk to the first CloudHSM cluster. 9. Restart EJBCA/WildFly 10. Sign the CSR with the Root CA. 11. Reconfigure the CloudHSM client to talk to the second CloudHSM cluster. 12. Restart EJBCA/WildFly 13. Bring the signed certificate back to the issuing CA and fulfill the waiting CAs CSR PRIMEKEY 6 (6)

7 A single instance can have more than one issuing CA on it. Just create the three keys for each CA in a single Crypto Token. For example in a single CryptoToken, you might have the following keys: IssuingCA1_signKey0001 IssuingCA1_defaultKey0001 IssuingCA1_testKey0001 IssuingCA2_signKey0001 IssuingCA2_defaultKey0001 IssuingCA2_testKey0001 Although you can have all of the keys for a Root CA and an Issuing CA in the same CryptoToken, this is not advisable since the Root CA would always need to be online so the Issuing CA can run PRIMEKEY 7 (7)

8 1 - Create CloudHSM Cluster AWS CloudHSM provides Hardware Security Modules (HSMs) in a cluster, a collection of individual HSMs that AWS CloudHSM keeps in sync. When you create a cluster, you specify an Amazon Virtual Private Cloud (VPC) and a Subnet in the Availability Zone (AZ) of your instance. You then create an HSM in your cluster, selecting an Availability Zone (AZ) for the creation of the HSM. Follow the steps below to create a cluster and an HSM: 1. Navigate to the AWS CloudHSM console on 2. Click Create Cluster. 3. In the Cluster configuration section, select the VPC and Subnet in the Availability Zone (AZ) of your EJBCA ECE Instance: 4. Wait for the cluster creation process to complete: 2019 PRIMEKEY 8 (8)

9 5. Once its completed, the cluster will be in an Uninitialized state. Click Initialize. 6. Choose an Availability Zone (AZ) to create the HSM in: 7. Click Create and wait for the HSM to be created. Click refresh after a few minutes. 8. Once completed, the wizard will show you the HSM created and allow you to continue by clicking Next: 9. Links appear to Download a certificate signing request (CSR) from the HSM Cluster to authenticate it PRIMEKEY 9 (9)

10 The certificates are used in the next step, see 2 - Use OpenSSL to Validate the HSM PRIMEKEY 10 (10)

11 2 - Use OpenSSL to Validate the HSM To start using the HSM, you have to sign a Cluster CSR using a CA. Since EJBCA is setup to be a CA but the keys are not in an HSM, yet we are trying to create an HSM. This seems like a predicament. Per Amazon: The certificate is used by the software libraries for authentication when you access the HSM via cloudhsm_mgmt_util. For example, for the older version i.e. CloudHSM Classic, to login to the HSM, customer's must use SSH authentication to login as manager user [2]. Hence, I believe you can consider the private key/ certificate you used to sign the CSR similar to a private key of a SSH key pair. Even if you even use other HSMs, you have to use soft keystore somewhere. With that in mind we are proceeding as follows to create the CSR and generate they key pair. 1. Download the 4 certificates in the links in the UI: There will be 4 files that look like the following: 2. Copy these certificates to the EJBCA instance. The prefix of the file is the cluster ID. For example, this cluster ID is cluster-ijd53hj4zdv_file-type. In the example below we will show it for the CSR but you will need to copy each of these files. You can also use your preferred SCP client: # scp -i ~/Documents/C2\ Comp/PrimeKey/EC2\ Creds/c2-ssh/c2-ssh.pem ec2- user@ec compute-1.amazonaws.com:/home/ec2-user clusterijd53hj4zdv_clustercsr.csr ~/Downloads/cluster-ijd53hj4zdv_ClusterCsr.csr 3. Use OpenSSL to create a key to validate the HSM. This command asks for a password to protect the key and you may use the instance ID or another password of your choosing. # openssl genrsa -aes256 -out customerca.key Create a self signed issuing cert with the key created. This command asks for values for the cert (country code, location, etc.) and you can populate them or press Enter to leave them blank. You must populate at least one value PRIMEKEY 11 (11)

12 # openssl req -new -x509 -days key customerca.key -out customerca.crt 5. Sign the CSR with the issuing cert and key. Be sure to change the CSR (after the -in value) to be the one that was downloaded from your HSM cluster. This command creates a file named CustomerHsmSignedCertificate.crt. Use this file as the signed certificate when you initialize the cluster. # openssl x509 -req -days in cluster-ijd53hj4zdv_clustercsr.csr \ -CA customerca.crt \ -CAkey customerca.key \ -CAcreateserial \ -out CustomerHsmSignedCertificate.crt 2019 PRIMEKEY 12 (12)

13 3 - Initialize the CloudHSM Follow the steps below to use your signed HSM certificate and your CA's issuing certificate to initialize your AWS CloudHSM cluster. 1. On the AWS CloudHSM consoleupload the certificates page, do the following: a. Next to Cluster certificate, click Upload file and select the HSM certificate that you signed previously. If you completed the steps in the previous 2 - Use OpenSSL to Validate the HSM section, select the file named CustomerSignedHsmCertificate.crt. If you performed these steps on the cloud instance you will need to download them back to a client that has access to the HSM cluster creation wizard. b. Next to Issuing certificate, click Upload file and select your CA's issuing certificate. If you completed the steps in the previous section, select the file named customerca.crt. If you used a CA to issue the cluster certificate, provide a certificate chain that begins with the certificate that issued the cluster certificate and ends with the CA's root certificate. The certificate chain must be in PEM format and can contain a maximum of 5000 characters. c. Click Upload and initialize. 2. The wizard initializes the HSM and informs you that the initialization is now in progress. Click refresh to refresh the status PRIMEKEY 13 (13)

14 2019 PRIMEKEY 14 (14)

15 4 - Assigning the Security Group to the EJBCA Instance The Cloud HSM initialization process creates a new Security Group as part of its install process. This Security Group needs to be added to the EJBCA instance for the two to communicate. To assign the security group to the EJBCA instance, do the following: 1. Select your EJBCA Enterprise Instance (or right click on it) and select: Networking > Change Security Groups. 2. You will see a new security group that was generated from the CloudHSM install process. Select it and click Assign Security Group PRIMEKEY 15 (15)

16 5 - Configure the cloudhsm-client To configure the CloudHSM client cloudhsm-client, do the following: 1. SSH into the EJBCA instance. 2. Copy your issuing certificate (the one that you used to sign the cluster's certificate) to the following location on the client instance: /opt/cloudhsm/etc/customerca.crt You need root permissions on the client instance to copy your certificate to this location. If you used your EJBCA instance to generate this file it is located in /home/ec2-user and you may also move or copy it to this location. 3. Stop the cloudhsm-client # sudo stop cloudhsm-client or # sudo service cloudhsm-client stop 4. Use the following command to update the configuration files for the AWS CloudHSM client and command line tools, specifying the IP address of the HSM in your cluster. If you don't know the HSM's IP address, view your cluster in the AWS CloudHSM console. # sudo /opt/cloudhsm/bin/configure -a <IP address> Updating server config in /opt/cloudhsm/etc/cloudhsm_client.cfg Updating server config in /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg 5. Start the CloudHSM client: # sudo start cloudhsm-client or # sudo service cloudhsm-client start 6. Ensure the cloudhsm_mgmt_util configuration file is up to date. We need to do this to ensure cloudhsm_mgmt_util is aware of all the HSM instances in the cluster. # sudo /opt/cloudhsm/bin/configure -m 7. Connect to the HSM instances, enable end-to-end encryption, and log in to the HSM instances. Enabling end-to-end encryption encrypts the communication between cloudhsm_mgmt_util and the HSM to prevent interception of sensitive information such as passwords: 2019 PRIMEKEY 16 (16)

17 # /opt/cloudhsm/bin/cloudhsm_mgmt_util /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg aws-cloudhsm> enable_e2e 2019 PRIMEKEY 17 (17)

18 6 - PKCS11 PIN The PKCS #11 interface defines a PIN (personal identification number) for users of a cryptographic token. To specify a PKCS #11 PIN in the context of the AWS CloudHSM software library for PKCS#11, use the following format: <HSM_user_name>:<password> For example, the following is the PKCS #11 PIN for an HSM crypto user (CU) with user name CryptoUser and password CUPassword123!: CryptoUser:CUPassword123! 2019 PRIMEKEY 18 (18)

19 7 - Activate the Cluster To activate the cluster, do the following: 1. SSH into the EJBCA instance if not already done so. 2. Use the following command to start the AWS CloudHSM cloudhsm_mgmt_util command line tool: # sudo /opt/cloudhsm/bin/cloudhsm_mgmt_util /opt/cloudhsm/etc/ cloudhsm_mgmt_util.cfg If you get a connection timed out, the security group has not been added to the instance. If you get "E2E failed: unable to establish ssl connection" this is because there is no customerca.crt (or its an incorrect file) in /opt/cloudhsm/etc. 3. Use the enable_e2e command to enable end-to-end encryption: aws-cloudhsm> enable_e2e E2E enabled on server 0(server1) 4. Use the loginhsm command to log in to the HSM as the precrypto officer (PRECO) user: aws-cloudhsm> loginhsm PRECO admin password loginhsm success on server 0(server1) 5. Use the changepswd command to change the precrypto officer (PRECO) user's password: aws-cloudhsm> changepswd PRECO admin <NewPassword> *************************CAUTION******************************** This is a CRITICAL operation, should be done on all nodes in the cluster. Cav server does NOT synchronize these changes with the nodes on which this operation is not executed or failed, please ensure this operation is executed on all nodes in the cluster. **************************************************************** Do you want to continue(y/n)?y Changing password for admin(preco) on 1 nodes 6. Logout as user PRECO: aws-cloudhsm> logouthsm logouthsm success on server PRIMEKEY 19 (19)

20 7. Use the following syntax to login to the HSM going forward: loginhsm <user type> <user name> <password> For example: aws-cloudhsm> loginhsm CO admin <password> 2019 PRIMEKEY 20 (20)

21 8 - Create a CloudHSM Crypto User To create a CloudHSM crypto user, do the following: 1. Login as the CO admin: aws-cloudhsm> login CO admin <password> 2. Create an ejbca crypto user: aws-cloudhsm> createuser CU ejbca <password> 3. Use listusers to see all users: aws-cloudhsm> listusers aws-cloudhsm>listusers Users on server 0( ): Number of users found:2 User Id User Type User Name MofnPubKey LoginFailureCnt 2FA 1 CO admin NO 0 NO 2 CU ejbca NO 0 NO 4. Quit the client utility: aws-cloudhsm> quit For more detailed user management, refer to the AWS CloudHSM documentation Managing HSM Users in AWS CloudHSM PRIMEKEY 21 (21)

22 9 - Create a Keystore in the HSM with clienttoolbox To create a keystore in the HSM using clienttoolbox, do the following: 1. Create a testkey with clienttoolbox. EJBCA will use this key for healthcheck and keepalive to the HSM. It is important to run these commands as the wildfly user. This is due to file system access permissions and maintaining the permissions for wildfly to be able to use these keys. # su - wildfly # /opt/ejbca/dist/clienttoolbox/ejbcaclienttoolbox.sh PKCS11HSMKeyTool generate /opt/primekey/cloudhsm/p11.conf 2048 testkey You will be prompted for a password in the format of <HSM_CryptoUser>:<password> For example, the following is the PKCS #11 PIN for an HSM crypto user (CU) with user name CryptoUser and password CUPassword123!: CryptoUser:CUPassword123! 3. Create a total of three keys for EJBCA: testkey (created in step 1) signkey defaultkey 4. Create two more keys called signkey and defaultkey with the following commands: # /opt/ejbca/dist/clienttoolbox/ejbcaclienttoolbox.sh PKCS11HSMKeyTool generate /opt/primekey/cloudhsm/p11.conf 4096 signkey0001 # /opt/ejbca/dist/clienttoolbox/ejbcaclienttoolbox.sh PKCS11HSMKeyTool generate /opt/primekey/cloudhsm/p11.conf 4096 defaultkey0001 If ECC keys are desired, you can use a named curve. For example, to generate a prime256v1 curve you could use the following command: # /opt/ejbca/dist/clienttoolbox/ejbcaclienttoolbox.sh PKCS11HSMKeyTool generate /opt/ PrimeKey/cloudhsm/p11.conf prime256v1 testkeyecdsa0001 For more information consult the EJBCA User Guide on ECC named curves PRIMEKEY 22 (22)

23 10 - Test with EJBCA ClientToolbox Use the EJBCA clienttoolbox to validate the key was created and is available on the CloudHSM according to the following example: 1. Use EJBCA clienttoolbox to validate key creation: # /opt/ejbca/dist/clienttoolbox/ejbcaclienttoolbox.sh PKCS11HSMKeyTool test / opt/primekey/cloudhsm/lib/libliquidsec_pkcs11.so 1 When prompted for a password you must use the CloudHSM password format of username:password, for example: CryptoUser:CUPassword123!. 2. Ensure the clienttoolbox outputs the results of the keys found on the CloudHSM according to the following example: 2019 PRIMEKEY 23 (23)

24 11 - Create a CryptoToken in EJBCA If you have already performed these steps before creating the keys with the clienttoolbox, restart EJBCA using the command service wildfly restart. If you are creating the crypto token for the first time, proceed with the following steps: 1. Login to EJBCA as Superadmin. 2. Select Crypto Tokens and click Create new: 3. Specify the values as follows: Name: <anything> (For example CloudHSM Root Slot) Type: PKCS#11 Authentication Code: <HSM_CryptoUser>:<password> (ex. CryptoUser:CUPassword123!) AutoActivation: Clear (Do not set this as setting it will prevent the CryptoToken from activating at all). Use Explicit ECC parameters: Clear PKCS#11: Library: AWS CloudHSM PKCS#11: Reference Type: Slot ID PKCS#11: Reference: 1 PKCS#11: Attribute Type: Default 4. Click Save. 5. All keys in the slot created by ClientToolBox display in the Crypto Token: 2019 PRIMEKEY 24 (24)

25 Auto Activating a CloudHSM CryptoToken If you would like to auto activate a crypto token at boot time, we recommend hashing the password with the ejbca.sh command line: [/opt/ejbca/bin]#./ejbca.sh encryptpwd Please note that this encryption does not provide absolute security. If 'password.encryption.key' property haven't been customized it doesn't provide more security than just preventing accidental viewing. Enter word to encrypt: Encrypting pwd (with default key) 939fb2c5ec0094dcb a9d55cb4236ba0c212a83641d9b a76 Once obtaining this hash use the following command and run a script at startup: [/opt/ejbca/bin]#./ejbca.sh cryptotoken activate --token "CloudHSM" 939fb2c5ec0094dcb a9d55cb4236ba0c212a83641d9b a76 CryptoToken activated successfully using supplied PIN PRIMEKEY 25 (25)

26 Appendix A - Restoring an HSM Backup to a New Instance Follow the steps below to restore an HSM from a backup to a new node: 1. Click on CloudHSM in the Services section of the AWS Console and find the existing cluster for the HSM you want to restore. AWS CloudHSM is only billing if there is an HSM in the active cluster. 2. Select the link to the cluster that contains the HSM material to restore. 3. Click Create HSM and step through the wizard to create a new cluster by selecting the Subnet/AZ for the HSM. 4. Once completed it will take some time for AWS to create the HSM. During this time the state will be "Create in progress". 5. Once complete the HSM state will be active. Click the refresh button to refresh the status. 6. Ensure the appropriate security group for the HSM created is assigned to the EJBCA Enterprise Cloud Instance. For more information, see Assigning the Security Group to the EJBCA Instance. 7. SSH into your EJBCA instance with your SSH key using the following command syntax: # ssh -i ~/Documents/my-ssh.pem ec2-user@ec2-my-public-ec2- ip.compute-1.amazonaws.com 8. Ensure that the certificate that is used to authenticate with the CloudHSM resides at: # /opt/cloudhsm/etc/customerca.crt 9. Stop the CloudHSM client using one of the following commands: # sudo stop cloudhsm-client or # sudo service cloudhsm-client stop 10. Configure the CloudHSM client with the following command replacing with the IP address of your HSMCluster: 2019 PRIMEKEY 26 (26)

27 # sudo /opt/cloudhsm/bin/configure -a Copy any public key certificates used with EJBCA in the past to the following directory (EJBCA will not be able to read any keys in CloudHSM without the matching public certificate that was created with it): # /opt/ejbca/.liquidsec/ 12. Start the CloudHSM client with the following command: # sudo start cloudhsm-client or # sudo service cloudhsm-client start 13. Update the cloudhsm_mgmt_util Configuration File # sudo /opt/cloudhsm/bin/configure -m 14. Start cloudhsm_mgmt_util to ensure you can connect to the HSM # /opt/cloudhsm/bin/cloudhsm_mgmt_util /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg 15. Restart Wildfly with the following command: # sudo systemctl restart wildfly or # sudo service wildfly restart 2019 PRIMEKEY 27 (27)

28 Appendix B - Troubleshooting HSM Issues If you are getting errors when generating keys use the following to troubleshoot: 1. As of around EJBCA 6.15 you can get an error that says "Re-enter authentication code and Confirm Save to continue." Re-enter the password and try again. This will generate if the configuration already exists and you are saving an existing entry with an existing HSM slot reference. 2. As the "wildfly" user, attempt to use the test command to read the keys. If clienttoolbox cannot read the keys, EJBCA cannot either: # /opt/ejbca/dist/clienttoolbox/ejbcaclienttoolbox.sh PKCS11HSMKeyTool test / opt/primekey/cloudhsm/lib/libliquidsec_pkcs11.so 1 3. Using AWS keytool are the keys that were previously created visible with the CLI? Some AWS commands that might help: # Load the key management utility /opt/cloudhsm/bin/key_mgmt_util loginhsm -u CU -s CRYPTO_USER_NAMNE -p HSM_PASSWORD # list all keys in HSM findkey # If the clienttoolbox create key command ran and you are sure it did not work (because the cert locally didn't get created (even if you get an error the key can still create on the HSM) you can delete the key with the following command: deletekey -k HANDLE_NUM # To see details about a given key (then exit out of key management utility and read the txt file): getattribute -a 512 -out /tmp/keyattr.txt -o HANDLE_NUM 2019 PRIMEKEY 28 (28)