Talend Open Studio for Big Data. Installation and Upgrade Guide 5.6.1

Transcription

1 Talend Open Studio for Big Data Installation and Upgrade Guide 5.6.1

2 Talend Open Studio for Big Data Adapted for v Supersedes previous releases. Publication date: December 11, 2014 Copyleft This documentation is provided under the terms of the Creative Commons Public License (CCPL). For more information about what you can and cannot do with this documentation in accordance with the CCPL, please read: License Agreement The software described in this documentation is licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License. You may obtain a copy of the License at Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. This product includes software developed at AOP Alliance (Java/J2EE AOP standards), ASM, Amazon, AntlR, Apache ActiveMQ, Apache Ant, Apache Avro, Apache Axiom, Apache Axis, Apache Axis 2, Apache Batik, Apache CXF, Apache Cassandra, Apache Chemistry, Apache Common Http Client, Apache Common Http Core, Apache Commons, Apache Commons Bcel, Apache Commons JxPath, Apache Commons Lang, Apache Datafu, Apache Derby Database Engine and Embedded JDBC Driver, Apache Geronimo, Apache HCatalog, Apache Hadoop, Apache Hbase, Apache Hive, Apache HttpClient, Apache HttpComponents Client, Apache JAMES, Apache Log4j, Apache Lucene Core, Apache Neethi, Apache Oozie, Apache POI, Apache Parquet, Apache Pig, Apache PiggyBank, Apache ServiceMix, Apache Sqoop, Apache Thrift, Apache Tomcat, Apache Velocity, Apache WSS4J, Apache WebServices Common Utilities, Apache Xml-RPC, Apache Zookeeper, Box Java SDK (V2), CSV Tools, Cloudera HTrace, ConcurrentLinkedHashMap for Java, Couchbase Client, DataNucleus, DataStax Java Driver for Apache Cassandra, Ehcache, Ezmorph, Ganymed SSH-2 for Java, Google APIs Client Library for Java, Google Gson, Groovy, Guava: Google Core Libraries for Java, H2 Embedded Database and JDBC Driver, Hector: A high level Java client for Apache Cassandra, Hibernate BeanValidation API, Hibernate Validator, HighScale Lib, HsqlDB, Ini4j, JClouds, JDO-API, JLine, JSON, JSR 305: Annotations for Software Defect Detection in Java, JUnit, Jackson Java JSON-processor, Java API for RESTful Services, Java Agent for Memory Measurements, Jaxb, Jaxen, JetS3T, Jettison, Jetty, Joda-Time, Json Simple, LZ4: Extremely Fast Compression algorithm, LightCouch, MetaStuff, Metrics API, Metrics Reporter Config, Microsoft Azure SDK for Java, Mondrian, MongoDB Java Driver, Netty, Ning Compression codec for LZF encoding, OpenSAML, Paraccel JDBC Driver, Parboiled, PostgreSQL JDBC Driver, Protocol Buffers - Google's data interchange format, Resty: A simple HTTP REST client for Java, Riak Client, Rocoto, SDSU Java Library, SL4J: Simple Logging Facade for Java, SQLite JDBC Driver, Scala Lang, Simple API for CSS, Snappy for Java a fast compressor/ decompresser, SpyMemCached, SshJ, StAX API, StAXON - JSON via StAX, Super SCV, The Castor Project, The Legion of the Bouncy Castle, Twitter4J, Uuid, W3C, Windows Azure Storage libraries for Java, Woden, Woodstox: High-performance XML processor, Xalan-J, Xerces2, XmlBeans, XmlSchema Core, Xmlsec - Apache Santuario, YAML parser and emitter for Java, Zip4J, atinject, dropbox-sdk-java: Java library for the Dropbox Core API, google-guice. Licensed under their respective license.

3 Table of Contents Preface... v 1. General information... v 1.1. Purpose... v 1.2. Audience... v 1.3. Typographical conventions... v Chapter 1. Prior to installing the Talend products Installation requirements Studio specific prerequisites Installing database client software (for bulk mode) Installing the XULRunner package (for Linux users) Compatible Platforms and Java environments... 3 Chapter 2. Installing Talend Studio for the first time Downloading and installing Talend Studio Launching Talend Studio Launching the Studio Configuring Talend Studio Identify required external modules Install external modules Chapter 3. Upgrading your Talend products Backing up the environment Upgrading the Talend projects in the Studio Appendix A. Supported Third-Party System/Database/Business Application Versions A.1. Supported systems, databases and business applications A.2. Supported Hadoop distribution versions Talend Installation and Upgrade Guide

4 Talend Installation and Upgrade Guide

5 Preface 1. General information 1.1. Purpose This Installation Guide explains how to install, configure and upgrade the Talend modules and related applications. For detailed explanation on how to use and fine-tune the Talend applications, please refer to the appropriate Administrator or User Guides of the Talend solutions. Information presented in this document applies to Talend products Audience This guide is for administrators and users of the Talend products. The layout of GUI screens provided in this document may vary slightly from your actual GUI Typographical conventions This guide uses the following typographical conventions: text in bold: window and dialog box buttons and fields, keyboard keys, menus, and menu and options, text in [bold]: window, wizard, and dialog box titles, text in courier: system parameters typed in by the user, text in italics: file, schema, column, row, and variable names, The icon indicates an item that provides additional information about an important point. It is also used to add comments related to a table or a figure, The icon indicates a message that gives information about the execution requirements or recommendation type. It is also used to refer to situations or information the end-user needs to be aware of or pay special attention to. Any command is highlighted with a grey background or code typeface. Talend Installation and Upgrade Guide

6 Talend Installation and Upgrade Guide

7 Chapter 1. Prior to installing the Talend products This chapter provides useful information on software and hardware prerequisites you should be aware of, prior to starting the installation of the Talend modules. In the following documentation: recommended: designates an environment recommended by Talend based on our experiences and customer usage; supported: designates a supported environment for use with the listed component or service; supported with limitations: designates an environment that is supported by Talend but with certain conditions explained in notes. Talend Installation and Upgrade Guide

8 Installation requirements 1.1. Installation requirements To make the most out of the Talend products, please consider the following hardware and software requirements. Memory usage heavily depends on the size and nature of your Talend projects. However, in summary, if your Jobs include many transformation components, you should consider upgrading the total amount of memory allocated to your servers, based on the following recommendations. Table 1.1. Memory usage Product Client/Server Recommended alloc. memory Studio Client 3GB minimum, 4 GB recommended The same requirements also apply for disk usage. It also depends on your projects but can be summarized as: Table 1.2. Disk usage Product Client/Server Required disk space for installation Studio Client 3GB 3+ GB Ulimit settings on Unix systems Required disk space for use To make the most out of the Talend server modules and improve performance on Unix systems, you should set the limit of system resources (ulimit) to unlimited. Environment variable configuration: on Windows Prior to installing your Talend solutions, you have to set the JAVA_HOME Environment variable: Define your JAVA_HOME environment variable so that it points to the JDK directory. Example: if the JDK path is C:\Java\JDKx.x.x\bin, you must set the JAVA_HOME environment variable to point to: C:\Java\JDKx.x.x. It is highly recommended that the full path to the server installation directory is as short as possible and does not contain any space character. If you already have a suitable JDK installed in a path with a space, you simply need to put quotes around the path when setting the values for the environment variable. Environment variable configuration: on Linux Prior to installing your Talend solutions, you have to set the JAVA_HOME and Path environment variables: 1. Find the JDK home dir and export it in the JAVA_HOME environment variable. Example: export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_40/ export PATH=$JAVA_HOME/bin:$PATH 2. Add these two lines at the end of the global profiles in the /etc/profile file or in the user profiles in the ~/.profile file. Note that after changing one of these files you have to log on again. For more information on how to set the JAVA_HOME variable on Unix and Windows systems, see the online Oracle documentation. 2 Talend Installation and Upgrade Guide

9 Studio specific prerequisites 1.2. Studio specific prerequisites To use the Studio properly, you first need to install external programs specific to bulk components (if you want to use Oracle, Sybase, Informix or Ingres bulk functionality) Installing database client software (for bulk mode) Some bulk components, like Oracle, Sybase, Informix or Ingres, require database client software to run properly: OracleBulkExec uses the sqlldr external utility. This utility is available in Oracle clients that must be installed on the computer. Informix uses the dbload external utility. Ingres uses the sql external utility. Sybase uses the bcp.exe external utility. This utility is asked for in the Sybase bulk components' Basic Settings view. For more information, see tsybasebulkexec, tsybaseoutputbulk and tsybaseoutputbulkexec components on the appropriate Talend Components Reference Guide Installing the XULRunner package (for Linux users) On Linux, the XULRunner package is required to run the Studio. The XULRunner package version that is recommended is XULRunner v The XULRunner packages versions that are supported are v1.8.x x and v3.6.x. 1. Download XULRunner version from this location. 2. Unpack the archive file in the same directory where you unpacked the studio archive, but do not unpack it within the Studio folder. 3. Add the following line at the end of the Studio.ini file that corresponds to your Linux architecture: -Dorg.eclipse.swt.browser.XULRunnerPath=</usr/lib/xulrunner> where </usr/lib/xulrunner> is the xulrunner installation path. For example, if you have unpacked the Studio in a directory under your user home directory /home/<user>/ Talend/, you need to add the following to the.ini file: -Dorg.eclipse.swt.browser.XULRunnerPath=/home/<user>/Talend/xulrunner/ 1.3. Compatible Platforms and Java environments Please refer to the following grids for a summary of supported OS and Java Runtime environments. Talend Installation and Upgrade Guide 3

10 Compatible Platforms and Java environments Table 1.3. Java environments Support type Vendor Version Recommended Oracle Java/JDK 7 Supported with limitations 1 IBM Java/JDK 6 Deprecated Oracle Java/JDK 6 1. Only supported for CommandLine, JobServer and Talend ESB, and only on Operating Systems Suse SLES and AIX. For more information on Java specificities (version, OS compatibility), please see the Knowledge Base articles on Talend Help Center. Table 1.4. Talend Studio Support type OS Version Processor Recommended Supported Deprecated Unix Linux Ubuntu bit Windows Microsoft Windows Professional 7 64-bit Unix Windows Mac OS Unix Linux Ubuntu bit Linux Ubuntu /64-bit Redhat Linux Enterprise Server Edition/ CentOS Redhat Linux Enterprise Server Edition/ CentOS bit 6.X (>=6.1) 64-bit SUSE SLES /64-bit Microsoft Windows bit Microsoft Windows 8 64-bit Microsoft Windows Professional 7 32-bit OS X Mavericks/ bit OS X Mountain Lion/ bit OS X Lion/ bit Linux Ubuntu 13.04/ /64-bit Redhat Linux Enterprise Server Edition/ CentOS 5.3 to /64-bit SUSE SLES /64-bit Windows Microsoft Windows Vista, SP1 32-/64-bit 4 Talend Installation and Upgrade Guide

11 Chapter 2. Installing Talend Studio for the first time We strongly encourage you to read the Prior to installing the Talend products before starting this chapter. This chapter details the procedures required to install Talend Studio. Talend Installation and Upgrade Guide

12 Downloading and installing Talend Studio 2.1. Downloading and installing Talend Studio Download 1. Get the archive file from the download section of the Talend website. Note that the.zip file contains binaries for ALL platforms (Linux/Unix, Windows and MacOS). 2. Once the download is complete, extract the archive file on your hard drive. It is recommended to avoid spaces and long names in the target installation directory path. Configure the memory settings If you want to tune the memory allocation for your JVM, you only need to edit the.ini file corresponding to your executable file. For example: For Talend Studio on 32bit-Windows, edit the file: TOS_BD-win32-x86.ini; For Talend Studio on Linux, edit the file: TOS_BD-linux-gtk-x86.ini. The default values are: -vmargs -Xms40m -Xmx500m -XX:MaxPermSize=128m If you only have 512Mo of memory on your computer, you can specify the memory allocation as following, for example: -vmargs -Xms40m -Xmx256m -XX:MaxPermSize=64m Learn more on Launching Talend Studio The following procedures describe how to launch the Studio, how to log in the Studio and how to create your first project Launching the Studio Launch the Studio On Windows, double-click the executable file to launch Talend Studio. On Unix-like systems, add execution rights on the desired TOS_BD-* binary before launching it. On a standard Linux box, the command is: $ chmod +x TOS_BD-linux-gtk-x86.sh $./TOS_BD-linux-gtk-x86.sh On Mac OS X, launch the following file: 6 Talend Installation and Upgrade Guide

13 Launching the Studio TOS_BD-macosx-cocoa.app/Contents/MacOS/TOS_BD-macosx-cocoa Public license First screen is a license screen. In the [License] window that appears, read and accept the terms of the license agreement to proceed to the next step. Login and first project 1. As first time user, you need to set up a new project or you can also import a Demo project which gathers numerous job samples. To select a demo project, select TALENDDEMOSJAVA and click Import... To create a new project, enter the name of your project in the corresponding field and click Create... to complete the description of your project. 2. In the Project name field, type in the name of the project. In the Project description field, type in a description for this project. Click Finish when complete, and the newly created project is displayed in the Login window. 3. In the Login window, open the project you just created. A registration window opens. Talend Installation and Upgrade Guide 7

14 Configuring Talend Studio If required, follow the instructions provided to join the Talend community or click Skip to open a welcome window and launch the Studio Configuring Talend Studio Talend Studio requires specific third-party Java libraries or database drivers (.jar files) to be installed to connect to sources and targets. Those libraries or drivers, known as external modules, can be required by some of Talend components. Due to license restrictions, Talend may not be able to ship certain external modules within Talend Studio Identify required external modules When you launch Talend Studio or select Help > Install Additional Packages in the Talend Studio menu, if any external modules are found missing for any features, the [Additional Talend packages] wizard opens, showing the Optional and Required third-party libraries check boxes. Make sure these check boxes are selected and click Finish to open the [Download external modules] dialog box, which lists all the available external modules, displays the license terms under which the external modules are provided, and lets you install all the modules at a single click. For more information, see Install external modules. On your design workspace, if a component requires the installation of external modules before it can work properly, a red error indicator appears on the component. With your mouse pointer over the error indicator, you can see a tooltip message showing which external modules are required for that component to work. When you open the Basic settings or Advanced settings view of a component for which one or more external modules are required, you will see a piece of highlighted information about external modules, followed by an Install button. Clicking the Install button opens a wizard that will show you the external modules to be installed. The Modules view lists all the modules required to use the components embedded in the Studio, including those Java libraries and drivers that you must install to get the relevant components working. If the Modules view is not shown under your design workspace, go to Window > Show View > Talend and then select Modules from the list. The table below describes the information presented in the Modules view. Column Status Description points out if a module is installed or not installed on your system. 8 Talend Installation and Upgrade Guide

15 Identify required external modules Column Description The icon indicates that the module is not necessarily required for the corresponding component listed in the Context column. Context Module Description Required The icon indicates that the module is absolutely required for the corresponding component. lists the name of Talend component using the module. If this column is empty, the module is then required for the general use of Talend Studio. This column lists any external libraries added to the routines you create and save in the Studio library folder. For more information, see the Talend Studio User Guide. lists the module exact name. explains why the module/library is required. the selected check box indicates that the module is required. In addition to the Modules view, the Studio provides a mechanism that enables you to easily identify, download and install most of the required third-party modules from the Talend website and directs you to valid websites for the rest. A Jar installation wizard appears whenever any required external module is found missing for any feature in the Studio, including when you: drop a component from the Palette if one or more external modules required for that component to work are missing in the Studio, or click the Guess schema button in the Component view of a component if one or more external modules required for that component to work are missing in the Studio, click Install on the top of the Basic settings or Advanced settings view of a component for which one or more required external modules are missing, run a Job that involves components for which one or more required external modules are missing, or click the button in the Modules view. When you click this button, the wizard that appears will list all the required external modules that are not integrated in the Studio. The table below describes the information presented in the wizard. Talend Installation and Upgrade Guide 9

16 Install external modules Item Jar Module Required by component Required License More information Action Description The file name of the external module. A short description about the nature of the module. Lists the components that require the external module. The selected check box indicates that the module is required. The license under which the module is provided. Provides the URL of the valid website where you can find more information about this module and download the module manually. : Click to open the [Download external modules] dialog box to download and install the module, which is available on the Talend website; : Click the link to open the valid website to download the module, which is not available on the Talend website, and then click the jar button to import the downloaded module into your studio; Do not show again : You need to find and download the module yourself and click the jar button to import it into the your studio. Click to open the [Download external modules] dialog box to download and install all the required modules that are available on the Talend website. Select to prevent the wizard from appearing again unless you click the Modules tab view. button in the This check box shows only when you drop a component, or guess the schema of a database, that requires an external module, or click the Install button on the Component tab of a component that requires an external module. Click here to obtain more Click to go to Talend online documentation on installing third-party modules. information about external modules This wizard lists the external modules to be installed, the licenses under which they are provided, and the URLs of the valid websites where they are downloadable, and allows you to download and install automatically all the modules available on the Talend website and download those not available on the Talend website by following the links provided in the Action column and then install them into your Studio manually. When you drop a component, or guess the schema of a database, that requires an external module for which neither the Jar file nor its download URL information is available on the Talend website, the Jar installation wizard does not appear, but the Error Log view will present an error message informing you that the download URL for that module is not available. You can try to find and download it by yourself, and then install it manually into the Studio. To show the Error Log view on the tab system, go to Window > Show views, then expand the General node and select Error Log Install external modules To download and install modules in the Studio To download and install external modules automatically, do the following: If you are working behind a network proxy, make sure you have correctly set up your proxy before you can download and install external modules in your Studio. To access the proxy settings, select Window > Preferences from the menu to open the [Preferences] window, then expand the General node and click Network Settings. 1. In the Jar installation wizard, click the Download and Install button to install a particular module, or click the Download and install all modules available button to install all the available modules, or select Help > Install Additional Packages from the menu to open the [Additional Talend Packages] wizard. From this wizard, make sure the Optional and Required third-party libraries check boxes are selected and click Finish. The [Download external modules] dialog box opens. 10 Talend Installation and Upgrade Guide

17 Install external modules This [Additional Talend Packages] wizard appears automatically when you launch Talend Studio if any additional packages, including external modules, need to be installed for any features to function in the Studio. 2. To download and install the external module(s) provided under a particular license, select that license from the Licenses pane, review the license terms, select the I accept the terms of the license agreement option, and click Finish to start the download and installation process. To download and install all external modules provided under all the listed licenses, click the Accept all button to start the download and installation process. Upon installation of the chosen external module or modules, a dialog box appears to notify you about the number of modules successfully installed and/or about the modules failed to install, if any. To install manually an external module you already have in your local file system, do the following: Talend Open Studio for Big Data does not come with the JDBC drivers for Oracle databases due to Apache license restrictions. For Oracle9i, the required JDBC driver downloadable from Oracle website is named ojdbc14.jar, the same as that for Oracle 10g. To enable the JDBC driver for Oracle9i you have downloaded to work in Talend Open Studio for Big Data, you have to change the file name to ojdbc14-9i.jar before installing it into the Studio. 1. Click the button in the upper right corner of the Modules view or in Jar installation wizard to browse your local file system. If the Modules view is not shown under your design workspace, go to Window > Show View > Talend and then select Modules from the list. 2. In the [Open] dialog box of your file system, browse to the module you want to install, double-click the.jar file, or select it and then click Open to install it. The dialog box closes and the selected module is installed in the library folder of the current Studio. Talend Installation and Upgrade Guide 11

18 Install external modules You can now use the component dependent on this module in any of your Job designs. To install modules in CommandLine If you use the Studio and CommandLine on different machines, you need to retrieve the downloaded.jar files and add them in CommandLine. 1. Make sure CommandLine is not started, then download the external modules from the Modules view as explained in the previous procedure. 2. Copy the downloaded.jar files from <StudioPath>/lib/java and paste them into <CommandLinePath>/ lib/java, where <StudioPath> and <CommandLinePath> are the installation directories of the Studio and CommandLine respectively. Note that the <CommandLinePath>/lib/java folder is not created by default, it is created the first time you start the CommandLine application. 3. Restart CommandLine. You can now use the component dependent on these modules. To install modules downloaded from external websites Some modules are not available on the Talend website but can be downloaded directly from external websites. Once downloaded, these modules must be placed in specific folders. For the studio, the downloaded modules must be placed in the following folder: <StudioPath>/lib/java 12 Talend Installation and Upgrade Guide

19 Chapter 3. Upgrading your Talend products This chapter describes the various operations required to migrate version of the Talend solutions. We assume that you have installed and configured these solutions as described in the Installing Talend Studio for the first time. The migration and upgrade process includes the following mandatory steps: These steps usually need to be completed in the following order. 1. Backing up the environment, see the Backing up the environment. 2. Upgrading the Talend projects in the Studio, see the Upgrading the Talend projects in the Studio. Talend Installation and Upgrade Guide

20 Backing up the environment 3.1. Backing up the environment Before you start migrating your Talend solutions, make sure your environment is correctly backed up. The environment backup process includes the following mandatory steps: These steps usually need to be completed in the following order. 1. Saving the local projects, see Saving the local projects. Saving the local projects 1. Launch the Studio. 2. Click the icon and export your local projects to an archive file Upgrading the Talend projects in the Studio Importing your local projects 1. Launch the new Studio you have just installed. 2. In the login window, select Import, then import the archive file containing your local projects. The local projects are displayed in the Project list and appear on the Studio Repository view. For more information on how to export local projects to an archive file, see Saving the local projects. 14 Talend Installation and Upgrade Guide

21 Appendix A. Supported Third-Party System/ Database/Business Application Versions This document provides the information about the versions of the systems or databases or business applications supported by Talend Studio. Talend Installation and Upgrade Guide

22 Supported systems, databases and business applications A.1. Supported systems, databases and business applications The access to these systems, databases and business applications varies depending on the Studio you are using. Systems/Databases Versions OS Alfresco 2.1 N/A 1 Amazon Redshift Initial release of Amazon Redshift N/A 1 AS/400 V5R2 to V7R2 N/A 1 Access 2003 Windows Access 2007 Windows Bonita Cassandra 1.1.2/1.2.2/2.0.X/2.1.X Windows + Linux DB Generic ODBC Windows DB2 9.5/9.7 Windows + Linux EXASolution 4 Windows FireBird 2.1 Windows + Linux Greenplum Windows (client only) + Linux HSQLDb N/A 1 Informix Windows + Linux Ingres 9.2 Windows + Linux Interbase 7 and above N/A 1 JavaDB 6 Windows + Linux LDAP No version limitation Windows + Linux Microsoft AX Dynamics AX 4.0 N/A 1 N/A 1 Microsoft CRM 2011 Dynamics AX 2012 N/A MS SQL Server 2000/2003/2005/2008/2012 Windows + Linux MaxDB 7.6 N/A 1 MongoDB 2.1.X/2.2.X/2.4.X/2.5.X/2.6.X Windows + Linux MySQL Netezza Mysql4 Mysql5 Version 6 and earlier have been tested. No issues have been found with other versions until now. Neo4j 1.X.X/2.X.X Linux OleDb 2000/2003/2005/2007/2010 N/A 1 Windows + Linux Windows + Linux Windows + Linux Oracle Oracle 8i/9i/10g/11g/11g (11.6)/12c Windows + Linux ParAccel 3.1/3.5 N/A 1 PostgreSQL 8.3 Windows + Linux PostgresPlus 8.3 Windows + Linux 16 Talend Installation and Upgrade Guide

23 Supported Hadoop distribution versions Systems/Databases Versions OS Salesforce until V26 Windows + Linux SAP 4.6 Windows SAP Hana 1.0 Windows SQLite Windows + Linux Sybase 12.5/12.7/15.2/15.5/15.7 Windows + Linux SybaseIQ 12.5/12.7/15.2 Windows + Linux Teradata 12/13/14/15 Windows + Linux VectorWise 2 Windows + Linux Vertica 3/3.5/4/4.1/5.0/5.1/6.0/6.1.X/7.0.X Windows + Linux VtigerCRM Vtiger 5.0 Vtiger 5.1 exist 1.4 (deprecated) Windows 32bit + Linux 32bit 1. The test information is not available yet. N/A 1 A.2. Supported Hadoop distribution versions Modules Hadoop platforms Versions Kerberos HBase Hortonworks Apache Cloudera MapR Amazon EMR Pivotal HD V1.0.0 (deprecated) V1.2.0 (Bimota) V1.3.0 (Condor) V2.0.0 (BigWheel) V2.1.0 (Baikal) (HBase ) No kinit only kinit only kinit only (HBase ) CDH3 (deprecated) CDH4 (from 4.0 to 4.2) No CDH4.3 + (YARN mode) CDH 5.0 (YARN mode) CDH 5.1 (YARN mode and MR1 mode) No No No No No (Yarn mode) No Apache Apache No No 2.0 HCatalog Hortonworks V1.0.0 (deprecated) Custom 1 V1.2.0 (Bimota) V1.3.0 (Condor) Talend Installation and Upgrade Guide 17

24 Supported Hadoop distribution versions Modules Hadoop platforms Versions Kerberos Cloudera MapR V2.0.0 (BigWheel) V2.1.0 (Baikal) CDH5.0 (YARN mode) CDH5.1 (YARN mode and MR1 mode) HDFS Hortonworks V1.0.0 (deprecated) Custom 1 Apache Cloudera MapR Amazon EMR Pivotal HD V1.2.0 (Bimota) V1.3.0 (Condor) V2.0.0 (BigWheel) V2.1.0 (Baikal) No No CDH3 (deprecated) CDH4 (from 4.0 to 4.2) No CDH4.3 + (YARN mode) CDH5.0 (YARN mode) CDH5.1 (YARN mode and MR1 mode) (deprecated) No No No No No No (YARN mode) No MapR (deprecated) No Apache Apache No No 2.0 Hive Hive 1 (HiveServer) The security information is not available to standalone servers. Hortonworks V1.0.0 (deprecated) Custom 1 Apache Cloudera V1.2.0 (Bimota) V1.3.0 (Condor) V2.0.0 (BigWheel) V2.1.0 (Baikal) (Hive 0.7.1) No (Hive 0.9.0) No CDH3 (deprecated) CDH4 (from 4.0 to 4.2) No CDH4.3 + (YARN mode) CDH5.0 (YARN mode) 18 Talend Installation and Upgrade Guide

25 Supported Hadoop distribution versions Modules Hadoop platforms Versions Kerberos MapR Amazon EMR CDH5.1 (YARN mode and MR1 mode) (deprecated) No No No No No No (YARN mode) No MapR (deprecated) No Apache Apache Microsoft HD Insight 3.1 No Pivotal HD No No 2.0 Hive 2 (HiveServer2) Standalone servers only support the Kerberos authentication without a specific keytab. Hortonworks V1.2.0 (Bimota) Custom 1 Cloudera CDH4 MapR V1.3.0 (Condor) V2.0.0 (BigWheel) V2.1.0 (Baikal) CDH4 (from 4.0 to 4.2) CDH4.3 + (YARN mode) CDH5.0 (YARN mode) CDH5.1 (YARN mode and MR1 mode) No No No (YARN mode) No Pivotal HD 2.0 Microsoft HD Insight 3.1 No Oozie Hortonworks V1.0.0 (deprecated) Custom 1 Cloudera MapR V1.2.0 (Bimota) V1.3.0 (Condor) V2.0.0 (BigWheel) V2.1.0 (Baikal) CDH3 (deprecated) CDH4 (from 4.0 to 4.2) No CDH4.3 + (YARN mode) CDH5.0 (YARN mode) CDH5.1 (YARN mode and MR1 mode) (deprecated) No No No No Talend Installation and Upgrade Guide 19

26 Supported Hadoop distribution versions Modules Hadoop platforms Versions Kerberos No No (YARN mode) No Microsoft HD Insight 3.1 No Pig Spark Standalone YARN Client Hortonworks V1.0.0 (deprecated) Custom 1 Apache Cloudera MapR Amazon EMR Pivotal HD V1.2.0 (Bimota) V1.3.0 (Condor) V2.0.0 (BigWheel) V2.1.0 (Baikal) (Pig 0.9.1) No (Pig 0.9.2) CDH3 (deprecated) CDH4 (from 4.0 to 4.2) No CDH4.3 + (YARN mode) CDH5.0 (YARN mode) CDH5.1 (YARN mode and MR1 mode) (deprecated) No No No No No No (YARN mode) No MapR (deprecated) No Apache Apache No No 2.0 Microsoft HD Insight 3.1 No Custom 1 Cloudera CDH5.1 No MapR No Custom 1 Hortonworks V2.1.0 No MapR No Sqoop Hortonworks V1.0.0 (deprecated) Custom 1 V1.2.0 (Bimota) V1.3.0 (Condor) V2.0.0 (BigWheel) V2.1.0 (Baikal) Cloudera CDH3 (deprecated) No CDH4 (from 4.0 to 4.2) CDH4.3 + (YARN mode) 20 Talend Installation and Upgrade Guide

27 Supported Hadoop distribution versions Modules Hadoop platforms Versions Kerberos MapR Pivotal HD CDH5.0 (YARN mode) CDH5.1 (YARN mode and MR1 mode) (deprecated) No No No No No No (YARN mode) No No 2.0 Talend MapReduce Job Hortonworks V1.0.0 (deprecated) Custom 1 Cloudera MapR Amazon EMR V1.2.0 (Bimota) V1.3.0 (Condor) V2.0.0 (BigWheel) V2.1.0 (Baikal) CDH3 (deprecated) CDH4 (from 4.0 to 4.2) No CDH4.3 + (YARN mode) CDH5.0 (YARN mode) CDH5.1 (YARN mode and MR1 mode) (deprecated) No No No No No No (YARN mode) No Apache Apache Apache Pivotal HD Microsoft HD Insight 3.1 Custom 1 No No This enables the connection between the Studio and a custom Hadoop distribution not yet officially supported in the Studio. For further information, see the sections describing how to connect to a custom Hadoop distribution of the Talend Big Data Getting Started Guide or the documentation of any related component that creates the connection to a Hadoop distribution, such as thdfsconnection. For further information about the versions of all the supported third-party systems/databases, see Supported systems, databases and business applications. Talend Installation and Upgrade Guide 21

28 Talend Installation and Upgrade Guide