How to Configure Big Data Management 10.1 for MapR 5.1 Security Features

Transcription

1 How to Configure Big Data Management 10.1 for MapR 5.1 Security Features 2014, 2016 Informatica LLC. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. All other company and product names may be trade names or trademarks of their respective owners and/or copyrighted materials of such owners.

2 Abstract Informatica Big Data Management 10.1 adds support for MapR 5.1 Kerberos and MapR Ticket. This document describes how to use these MapR security features with Big Data Management and MapR 5.1. Supported Versions Informatica Big Data Management 10.1 Table of Contents Overview... 2 Install the RPM Archive and the EBF on MapR Cluster Nodes Installing on an Existing Cluster Configure the Informatica Domain to Communicate with a Kerberos-Enabled MapR 5.1 Cluster Generate a MapR Ticket Enable the TLS Protocol and Kerberos Copy the Truststore File to the Data Integration Service Machine Add Additional Properties for the Native Runtime Engine Add Additional Properties for the Blaze Runtime Engine Configure Hive Connection Properties Edit hive-site.xml to Enable a Mapping to Run with the Hive Run-Time Engine... 6 Configure the Informatica Domain to Communicate with a Cluster that Uses MapR Ticket Authentication... 6 Configure Hive and HDFS Metadata Fetch for MapR Ticket or Kerberos Running Mappings Using the Teradata Connector for Hadoop on a Hive or Blaze Engine Overview Informatica Big Data Management EBF adds support for MapR 5.1 security features. You can implement Big Data Management in a MapR environment that uses Kerberos or MapR Ticket for authentication. Before you follow the steps in this article, perform the following preliminary steps: 1. Install and configure an Informatica domain. 2. Configure Big Data Management to run mappings in a MapR 5.1 cluster. For more information, see the Big Data Management 10.1 Installation and Configuration Guide. After you complete the preliminary steps, perform the following steps to enable Big Data Management to use the security features of MapR 5.1: 1. Download and install EBF Select one of the following authentication methods: Configure the Informatica domain to communicate with the Kerberos-enabled MapR 5.1 cluster. Configure the Informatica domain to communicate with a cluster that uses MapR Ticket authentication. 3. Optionally, configure Hive and HDFS metadata fetch for MapR Ticket. 2

3 Install the RPM Archive and the EBF on MapR Cluster Nodes Install the RPM archive and the EBF on each node of the cluster. Installing on an Existing Cluster If you are installing the EBF on an existing cluster, download and uncompress the EBF archive file, and then perform the following steps: 1. Install the RPM package. 2. Install server and Big Data Management Utility updates. 3. Update clients. 4. Copy.jar files that you need to run mappings on the MapR cluster. Install the RPM Package Install the RPM package to implement support for MapR 5.1 on each cluster node. 1. Edit the input.properties file with the following information: DEST_DIR - destination directory for Informatica Big Data Management. Note: Install the binaries on each node of the cluster. 2. Type installebf.sh to run the installer. 3. Accept the license terms. 4. Select 1 to install the package on a local cluster node. Install Server Updates Install server updates on the machine where the Data Integration Service runs. 1. Edit the input.properties file with the following information: DEST_DIR - destination directory on the on the machine where the Data Integration Service runs. 2. Type installebf.sh to run the installer. The installer updates servers and the Big Data Management Configuration Utility. Update Clients Update clients on the Informatica host machine. 1. Edit the input.properties file with the following information: DEST_DIR - destination directory on the on the machine where clients are installed. 2. Type installebf.sh to run the installer. The installer updates the Developer tool. Configure the Informatica Domain to Communicate with a Kerberos-Enabled MapR 5.1 Cluster To use Kerberos authentication, perform the following tasks: 1. Generate a MapR ticket. 2. Enable the TLS protocol and Kerberos. 3. Copy the Truststore file to the Data Integration Service machine. 3

4 4. Add additional properties for the runtime engine you want to use. 5. Configure Hive connection properties. 6. Edit hive-site.xml to add properties to enable mappings to use the Hive run-time engine. Generate a MapR Ticket To enable mappings to run on a Kerberos-enabled MapR cluster, generate a MapR ticket for the Data Integration Service user. 1. Run the MapR kinit utility on the CLDB node of the cluster to create a Kerberos ticket for the Data Integration Service user. For information about how to generate MapR Tickets, refer to MapR documentation. 2. Run the maprlogin kerberos utility. Type: maprlogin kerberos The utility generates a MapR ticket in the /tmp directory using the following naming convention: maprticket_<userid> where <userid> corresponds to the Data Integration Service user. 3. Copy the ticket file from the cluster node to the following directory on the VM that runs the Data Integration Service: /tmp Enable the TLS Protocol and Kerberos Enable the TLS security protocol in the Administrator tool. 1. In the Administrator tool, browse to the Data Integration Service Process Properties tab. 2. In the Advanced Properties area, add the following line to the JVM Command Line Options: -Dhadoop.login=<MAPR_ECOSYSTEM_LOGIN_OPTS> -Dhttps.protocols=TLSv1.2 where <MAPR_ECOSYSTEM_LOGIN_OPTS> is the value of the MAPR_ECOSYSTEM_LOGIN_OPTS property in the file /opt/mapr/conf/env.sh. 3. Restart the Data Integration Service for the change to take effect. Copy the Truststore File to the Data Integration Service Machine 1. Copy the truststore file from the following location on the cluster: /opt/mapr/conf/ssl_truststore 2. Paste the truststore file to the following location on the Data Integration Service host: <Informatica installation directory>/services/shared/hadoop/mapr_5.1.0/conf/ssl_truststore Add Additional Properties for the Native Runtime Engine To use the native Data Integration Service to run mappings, define custom properties in the Administrator tool. 1. In the Administrator tool, browse to the Data Integration Service Process tab. 4

5 2. In the Custom Properties area, define the following properties and values: Property Value ExecutionContextOptions.JVMOption2 -Dhadoop.login=<MAPR_ECOSYSTEM_LOGIN_OPTS> - Dhttps.protocols=TLSv1.2 where <MAPR_ECOSYSTEM_LOGIN_OPTS> is the value of the MAPR_ECOSYSTEM_LOGIN_OPTS property in the file /opt/mapr/conf/ env.sh. ExecutionContextOptions.JVMOption7 -Dhttps.protocols=TLSv1.2 Add Additional Properties for the Blaze Runtime Engine To use the Blaze engine to run mappings, define custom properties in the Administrator tool. 1. In the Administrator tool, browse to the Data Integration Service Process tab. 2. In the environment variables area, define the Kerberos authentication protocol: Property JAVA_OPTS Value -Dhadoop.login=<MAPR_ECOSYSTEM_LOGIN_OPTS> -Dhttps.protocols=TLSv1.2 where <MAPR_ECOSYSTEM_LOGIN_OPTS> is the value of the MAPR_ECOSYSTEM_LOGIN_OPTS property in the file /opt/mapr/conf/env.sh. Configure Hive Connection Properties To use Hive to run mappings, you configure the Data Access Connection String with the service principal name. 1. In the Administrator tool, browse to the Connections tab and browse to the HiveServer2 Connection Properties area. 2. Configure the following connection properties: Property Metadata Connection String Data Access Connection String Value jdbc:hive2://<domain_host>:<port_number>/ default;principal=<service_principal_name> jdbc:hive2://<domain_host>:<port_number>/ default;principal=<service_principal_name> Note: You can retrieve the service principal name from the MapR Control System browser. 5

6 3. In the environment variables area, configure the following property to define the Kerberos authentication protocol: Property JAVA_OPTS Value -Dhadoop.login=<MAPR_ECOSYSTEM_LOGIN_OPTS> -Dhttps.protocols=TLSv1.2 where <MAPR_ECOSYSTEM_LOGIN_OPTS> is the value of the MAPR_ECOSYSTEM_LOGIN_OPTS property in the file /opt/mapr/conf/env.sh. Edit hive-site.xml to Enable a Mapping to Run with the Hive Run-Time Engine To run mappings using Hive, open the file <Informatica installation directory>/services/shared/hadoop/ mapr_<version>/conf/hive-site.xml for editing and make the following changes: 1. Locate the hive.server2.authentication property and change the value as follows: <property> <name>hive.server2.authentication</name> <value>kerberos</value> <description> </description> </property> 2. Add the following property: <property> <name>hive.metastore.sasl.enabled</name> <value>true</value> <description> </description> </property> 3. Open the file /opt/mapr/conf/hive-site.xml on the cluster and copy the following properties: hive-metastore.principal hive-metastore.keytab Paste both properties to <Informatica installation directory>/services/shared/hadoop/ mapr_<version>/conf/hive-site.xml Configure the Informatica Domain to Communicate with a Cluster that Uses MapR Ticket Authentication You can use the following runtime engines to run mappings on MapR clusters that use the MapR Ticket method of authentication: Native (Data Integration Service) Blaze Hive To use the MapR Ticket method of authentication, perform the following tasks: 1. Retrieve the value of the hive.server2.authentication property. 2. Open the following file for editing: <Informatica Installation Directory>/services/shared/hadoop/mapr_5.1.0/conf/hivesite.xml 3. Change the value of the hive.server2.authentication property from NONE to the hive.server2.authentication property value that you obtained from the cluster. 6

7 4. Add the following property to the hive-site.xml file: <property> <name>hive.metastore.sasl.enabled</name> <value>true</value> </property> Then save and close the hive-site.xml file. 5. Generate a MapR Ticket file for the Data Integration Service user on the cluster. To understand how to generate MapR tickets, refer to MapR documentation. 6. Copy the MapR ticket file to the following directory on the VM that runs the Data Integration Service: /tmp Configure Hive and HDFS Metadata Fetch for MapR Ticket or Kerberos Note: Before performing this task, complete the steps in the topic "Configure the Developer Tool" in the Informatica 10.1 Big Data Management Installation and Configuration Guide. To configure users for MapR Ticket or Kerberos-enabled MapR clusters, establish Linux accounts and configure user permissions for users. 1. Create a Linux user on the node where the HiveServer2 service runs. Use the same username as the Windows user account that runs the Developer tool client. We will refer to this user as the client user. 2. If the cluster is Kerberos-enabled, you can perform the following steps to generate a MapR ticket. Alternatively, follow steps 3 and 4. a. Install maprclient on the Windows machine. b. Generate a Kerberos ticket on the Windows machine. c. Use maprlogin to generate a maprticket at %TEMP%. Skip to step On the same node, log in as the client user and generate a MapR ticket. Refer to MapR documentation for more information. If the cluster is not Kerberos-enabled, follow these steps: a. Type the following command: maprlogin password b. When prompted, provide the password for the client user. If the cluster is Kerberos-enabled, follow these steps: a. Generate a Kerberos ticket using kinit. b. Type the following command to generate a maprticket: maprlogin kerberos The cluster generates a MapR ticket associated with the client user. By default, tickets on Linux systems are generated in the /tmp directory and have a name like maprticket_<username>. 4. Copy the MapR ticket file and paste it to the %TEMP% directory on the Windows machine. 5. Rename the file like this: maprticket_<username> where <username> is the username of the client user. 6. On the MapR Control System browser, get the value of the property hive.server2.authentication. 7

8 7. Open the file <Informatica_client_installation>\clients\DeveloperClient\hadoop \mapr_<version_number>\conf\hive-site.xml for editing. 8. Change the value of the property hive.server2.authentication from NONE to the value you got in Step 5. Note: If Kerberos is enabled on the cluster, comment out the hive.server2.authentication property in hivesite.xml. 9. Add the following lines to the hive-site.xml file: <property> <name>hive.metastore.sasl.enabled</name> <value>true</value> </property> 10. Save and close the hive-site.xml file. To test the Hive connection, or perform a metadata fetch task, use the following format for the connection string if the cluster is Kerberos-enabled: Example: jdbc:hive2://<hostname>:10000/default;principal=<spn> jdbc:hive2://myserver2:10000/default;principal=mapr/myserver2@clustername If custom authentication is enabled, specify the user name and password in the Database Connection tab of the Hive connection. Running Mappings Using the Teradata Connector for Hadoop on a Hive or Blaze Engine When you use Teradata Connector for Hadoop to run Teradata mappings on a Hive or Blaze engine, the Data Integration Service user must have the MapR Ticket available on all the nodes of the Hadoop cluster. Author Big Data Management Team 8