Talend Open Studio for MDM. Installation and Upgrade Guide 5.5.2

Transcription

1 Talend Open Studio for MDM Installation and Upgrade Guide 5.5.2

2 Talend Open Studio for MDM Adapted for v Supersedes previous releases. Publication date: January 29, 2015 Copyleft This documentation is provided under the terms of the Creative Commons Public License (CCPL). For more information about what you can and cannot do with this documentation in accordance with the CCPL, please read: Notices All brands, product names, company names, trademarks and service marks are the properties of their respective owners.

3 Table of Contents Preface... v 1. General information... v 1.1. Purpose... v 1.2. Audience... v 1.3. Typographical conventions... v Chapter 1. Prior to installing the Talend products Installation requirements Studio specific prerequisites Installing database client software (for bulk mode) Installing the XULRunner package (for Linux users) Compatible Platforms Compatible Databases Compatible Runtime Containers Compatible Web browsers Port information... 6 Chapter 2. Installing Talend Studio for the first time Downloading and installing Talend Studio Launching Talend Studio Launching the MDM Server Launching the Studio Launching the Talend MDM Web User Interface Configuring Talend Studio Identify required external modules Install external modules Chapter 3. Upgrading your Talend products Backing up the environment Upgrading the Talend projects in the Studio Migrating MDM projects Manually moving system objects from an XML database to a relational database Manually moving repository items from an XML database to a relational database Reimporting and redeploying your Jobs Moving the pictures and web resources Appendix A. Supported Third-Party System/Database/Business Application Versions A.1. Supported systems and databases A.2. Supported databases for profiling data A.3. Supported Hive distributions for profiling data A.4. Supported business applications Talend Installation and Upgrade Guide

4 Talend Installation and Upgrade Guide

5 Preface 1. General information 1.1. Purpose This Installation Guide explains how to install, configure and upgrade the Talend modules and related applications. For detailed explanation on how to use and fine-tune the Talend applications, please refer to the appropriate Administrator or User Guides of the Talend solutions. Information presented in this document applies to Talend products Audience This guide is devoted for administrators of the Talend products. The layout of GUI screens provided in this document may vary slightly from your actual GUI Typographical conventions This guide uses the following typographical conventions: text in bold: window and dialog box buttons and fields, keyboard keys, menus, and menu and options, text in [bold]: window, wizard, and dialog box titles, text in courier: system parameters typed in by the user, text in italics: file, schema, column, row, and variable names, The icon indicates an item that provides additional information about an important point. It is also used to add comments related to a table or a figure, The icon indicates a message that gives information about the execution requirements or recommendation type. It is also used to refer to situations or information the end-user needs to be aware of or pay special attention to. Any command is highlighted with a grey background or code typeface. Talend Installation and Upgrade Guide

6 Talend Installation and Upgrade Guide

7 Chapter 1. Prior to installing the Talend products This chapter provides useful information on software and hardware prerequisites you should be aware of, prior to starting the installation of the Talend modules. In the following documentation: recommended: designates an environment recommended by Talend based on our experiences and customer usage; supported: designates a supported environment for use with the listed component or service; supported with limitations: designates an environment that is supported by Talend but with certain conditions explained in notes. Talend Installation and Upgrade Guide

8 Installation requirements 1.1. Installation requirements To make the most out of the Talend products, please consider the following hardware and software requirements. Memory usage heavily depends on the size and nature of your Talend projects. However, in summary, if your Jobs include many transformation components, you should consider upgrading the total amount of memory allocated to your servers, based on the following recommendations. Table 1.1. Memory usage Product Client/Server Recommended alloc. memory Talend MDM Web User Interface Server 1GB minimum (default configuration), 4 GB recommended Studio Client 3GB minimum, 4 GB recommended The same requirements also apply for disk usage. It also depends on your projects but can be summarized as: Table 1.2. Disk usage Product Client/Server Required disk space for installation Required disk space for use Talend MDM Web User Interface Server 700 MB -(server) 1 GB+ Studio Client 3GB 3+ GB Ulimit settings on Unix systems -(MDM database) 2 x # records number in KB. For example: 5 million records = 10 GB. This represents the size that will be needed on the disk. However, we recommend to multiply the size really needed on the disk by 2 in order to avoid problems during high transactions. To make the most out of the Talend server modules and improve performance on Unix systems, you should set the limit of system resources (ulimit) to unlimited. Environment variable configuration: on Windows Prior to installing your Talend solutions, you have to set the JAVA_HOME Environment variable: Define your JAVA_HOME environment variable so that it points to the JDK directory. Example: if the JDK path is C:\Java\JDKx.x.x\bin, you must set the JAVA_HOME environment variable to point to: C:\Java\JDKx.x.x. It is highly recommended that the full path to the server installation directory is as short as possible and does not contain any space character. If you already have a suitable JDK installed in a path with a space, you simply need to put quotes around the path when setting the values for the environment variable. Environment variable configuration: on Linux Prior to installing your Talend solutions, you have to set the JAVA_HOME and Path environment variables: 1. Find the JDK home dir and export it in the JAVA_HOME environment variable. Example: export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_40/ 2 Talend Installation and Upgrade Guide

9 Studio specific prerequisites export PATH=$JAVA_HOME/bin:$PATH 2. Add these two lines at the end of the global profiles in the /etc/profile file or in the user profiles in the ~/.profile file. Note that after changing one of these files you have to log on again. For more information on how to set the JAVA_HOME variable on Unix and Windows systems, see the online Oracle documentation Studio specific prerequisites To use the Studio properly, you first need to install external programs specific to bulk components (if you want to use Oracle, Sybase, Informix or Ingres bulk functionality) Installing database client software (for bulk mode) Some bulk components, like Oracle, Sybase, Informix or Ingres, require database client software to run properly: OracleBulkExec uses the sqlldr external utility. This utility is available in Oracle clients that must be installed on the computer. Informix uses the dbload external utility. Ingres uses the sql external utility. Sybase uses the bcp.exe external utility. This utility is asked for in the Sybase bulk components' Basic Settings view. For more information, see tsybasebulkexec, tsybaseoutputbulk and tsybaseoutputbulkexec components on the appropriate Talend Components Reference Guide Installing the XULRunner package (for Linux users) On Linux, the XULRunner package is required to run the Studio. The XULRunner package version that is recommended is XULRunner v The XULRunner packages versions that are supported are v1.8.x x and v3.6.x. 1. Download XULRunner version from this location. 2. Unpack the archive file in the same directory where you unpacked the studio archive, but do not unpack it within the Studio folder. 3. Add the following line at the end of the Studio.ini file that corresponds to your Linux architecture: -Dorg.eclipse.swt.browser.XULRunnerPath=</usr/lib/xulrunner> where </usr/lib/xulrunner> is the xulrunner installation path. For example, if you have unpacked the Studio in a directory under your user home directory /home/<user>/ Talend/, you need to add the following to the.ini file: Talend Installation and Upgrade Guide 3

10 Compatible Platforms -Dorg.eclipse.swt.browser.XULRunnerPath=/home/<user>/Talend/xulrunner/ 1.3. Compatible Platforms Despite our intensive tests, you might encounter some issues when installing our products on some Operating Systems. Please refer to the following grid for a summary of supported OS and Java Runtime environments. Table 1.3. Talend Studio OS Version Processor Java JDK/JRE 1 Support type Linux Ubuntu bit Oracle Java 7 recommended Linux Ubuntu /64-bit Oracle Java 6 supported Linux Ubuntu 10.04/ /64-bit Oracle Java 6/7 supported Redhat Linux Enterprise Server Edition/ CentOS Redhat Linux Enterprise Server Edition/ CentOS 5.3 to /64-bit Oracle Java 6 supported 6.X (>=6.1) 64-bit Oracle Java 6/7 supported SUSE SLES 10/11 32-/64-bit Oracle Java 6/7 supported Microsoft Windows 8 64-bit Oracle Java 7 recommended Microsoft Windows 7 64-bit Oracle Java 7 recommended Microsoft Windows bit Oracle Java 7 supported Microsoft Windows Vista SP1 32-/64-bit Oracle Java 6/7 supported Microsoft Windows 7 32-bit Oracle Java 6/7 supported Microsoft Windows XP SP3 32-/64-bit Oracle Java 6 deprecated OS X Mountain Lion/ bit Oracle Java 6 supported 2 OS X Lion/ bit Oracle Java 6/7 supported OS X Mountain Lion/ bit Oracle Java 7 supported 1. Note that Java v.6 is no longer supported by Oracle and that it is recommended to use a recent update of JDK 1.6 (Update 11 or higher). 2. Need to set security settings to accept non MAC-registered applications. Table 1.4. Talend server modules OS Version Processor Java JDK/JRE 1 Support type Linux Ubuntu bit Oracle Java 7 recommended Red Hat Linux Enterprise Server Edition bit Oracle Java 7 recommended SUSE SLES 10/11 32-/64-bit Oracle Java 6/7 supported Linux Ubuntu /64-bit Oracle Java 6 supported Linux Ubuntu 10.04/ /64-bit Oracle Java 6/7 supported Red Hat Linux Enterprise Server Edition/CentOS Red Hat Linux Enterprise Server Edition/CentOS 5.5 to /64-bit Oracle Java 6 supported 6.X (>=6.1) 64-bit Oracle Java 6/7 supported SUSE SLES 10/11 32-/64-bit IBM Java 6 supported with limitations 2 4 Talend Installation and Upgrade Guide

11 Compatible Databases OS Version Processor Java JDK/JRE 1 Support type Solaris (SunOS) 10/11 64-bit Oracle Java 6/7 supported AIX bit IBM Java 6 supported with limitations 2 Microsoft Windows Server 2008 R2 64-bit Oracle Java 7 recommended Microsoft Windows Server 2008 R2 64-bit Oracle Java 6 supported Microsoft Windows Server bit Oracle Java 7 supported Microsoft Windows Server /64-bit Oracle Java 6 deprecated The server modules include Talend MDM Server. 1. It is recommended to use a recent update of JDK 1.6 (Update 11 or higher). 2. Only supported for Commandline, Jobserver and Talend ESB Compatible Databases Please refer to the following grid for a summary of supported databases. Table 1.5. MDM Server Databases Version Support type H2 1.3 supported XML Database N/A deprecated Talend MDM system objects and master data records are stored in two databases. The following table shows which combinations are supported. System Objects Master Data Records Supported Relational database Relational database yes (required for new features) XML database Relational database deprecated XML database XML database deprecated Relational database XML database no 1.5. Compatible Runtime Containers Please refer to the following grid for a summary of supported Web application servers. Table 1.6. Talend MDM Server Runtime Containers Version Support type JBoss Application Server (embedded) recommended (prepackaged with Talend MDM) 1.6. Compatible Web browsers Please refer to the table below for a summary of supported Web browsers. Talend Installation and Upgrade Guide 5

12 Port information Web browser Talend MDM Web User Interface, Drools Microsoft Internet Explorer 11 recommended except for Drools 1 Mozilla Firefox 13 to 28 recommended 2 Microsoft Internet Explorer 10 supported Apple Safari 5 to 7 supported Google Chrome 22 to 34 supported with limitations Microsoft Internet Explorer 11 is recommended for all modules except for Drools Guvnor which supports version Talend maintenance releases will support the most recent browser version at the time of the release. 3. Graphical restrictions Port information This section describes the most important TCP/IP ports the Talend products use. Please make sure that your firewall configuration is compatible with these ports or change the default ports where needed. Table Information : Port: a TCP/IP port or a range of ports. Direction: In (Inbound); Out (Outbound) - related to the communication direction (for example a HTTP Port for a Service we listen on request) will be an Inbound' port. For example, a browser which sends a request to port 7080 will have this port as Outbound' port in this list. Usage: which part of the Product component uses this port (for example 1099 is used by the JMX Monitoring component of Talend Runtime). Config: the file or location where the value can be changed. Remark: anything which is important to mention additionally. Table 1.7. Talend MDM Web User Interface Port Direction Usage Config Remark 8180 IN OUT MDM Server - JBoss HTTP port /server/default/conf/mdm.conf /deploy/jboss-web.deployer/ server.xml /deploy/http-invoker.sar/meta- INF/jboss-service.xml /deploy/jbossws.sar/ jbossws.beans/meta-inf/jbossbeans.xml 8443 IN OUT MDM Server - JBoss HTTPS port /deploy/jboss-web.deployer/ server.xml /deploy/jbossws.sar/ jbossws.beans/meta-inf/jbossbeans.xml 8009 IN OUT MDM Server - JBoss AJP Port /deploy/jboss-web.deployer/ server.xml 3873 IN OUT MDM Server - JBoss Invoker Locator port /deploy/ejb3.deployer/meta-inf/ jboss-service.xml *If this port is in conflict with another application port, you have the possibility to choose between the following ports: 8080, 8280 and Talend Installation and Upgrade Guide

13 Port information Port Direction Usage Config Remark 8093 IN OUT MDM Server - UIL for JMS /deploy/jms/uil2-service.xml 8083 IN OUT MDM Server - JBoss RMI dynamic class loader port 1099 IN OUT MDM Server - JBoss RMI NamingService port 1098 IN OUT MDM Server - JBoss JNP server port 4444 IN OUT MDM server - JBoss RMI/JRMP invoker port /conf/jboss-service.xml /conf/jboss-minimal.xml /conf/jboss-minimal.xml /conf/jboss-service.xml 4445 IN OUT MDM server - JBoss Pooled server /conf/jboss-service.xml Note that the template binding which describes all the ports used by MDM is located in the <JBossPath>/docs/examples/ binding-manager/sample-bindings.xml folder. Talend Installation and Upgrade Guide 7

14 Talend Installation and Upgrade Guide

15 Chapter 2. Installing Talend Studio for the first time We strongly encourage you to read the chapter Prior to installing the Talend products before starting this chapter. This chapter details the procedures required to install Talend Studio. Talend Installation and Upgrade Guide

16 Downloading and installing Talend Studio 2.1. Downloading and installing Talend Studio Download 1. Get the archive file from the download section of the Talend website. Note that the.zip file contains binaries for ALL platforms (Linux/Unix, Windows and MacOS). 2. Once the download is complete, extract the archive file on your hard drive. It is recommended to avoid spaces and long names in the target installation directory path. For Talend Open Studio for MDM, both Talend Studio and the MDM Server are bundled together. There is one compressed file, of the format: TOS_MDM-All-rYYYYY-VA.B.C.zip When you extract it to a directory of your choice, you get two files: - TOS_MDM-Studio-rYYYYY-VA.B.C.zip that contains the Talend Studio. - TOS_MDM-Server-rYYYYY-VA.B.C.jar that contains a cross-platform MDM-dedicated graphical installer to install JBoss and deploy the MDM Server. Configure the memory settings If you want to tune the memory allocation for your JVM, you only need to edit the.ini file corresponding to your executable file. For example: For Talend Studio on 32bit-Windows, edit the file: TOS_MDM-win32-x86.ini; For Talend Studio on Linux, edit the file: TOS_MDM-linux-gtk-x86.ini. The default values are: -vmargs -Xms40m -Xmx500m -XX:MaxPermSize=128m If you only have 512MB of memory on your computer, you can specify the memory allocation as following, for example: -vmargs -Xms40m -Xmx256m -XX:MaxPermSize=64m Learn more on Launching Talend Studio The following procedures describe how to launch the Studio and its related modules, how to log in the Studio and how to create your first project. 10 Talend Installation and Upgrade Guide

17 Launching the MDM Server Launching the MDM Server Installing in GUI mode 1. On Windows, double-click on the.jar file included in the product archive file and follow the instructions. On other platforms, execute the.jar file by right-clicking it and selecting the OpenJDK JRE or JRE. If, at a later time, you want to install the MDM server in Silent mode (see the next procedure), you must generate an installation script during the final step of the wizard. 2. Once you have installed the MDM server, go into the directory where you installed the server. 3. Execute the file: run.bat (Windows) or run.sh (Linux) Installing in Console mode The Console mode is a full headless mode, that is, without a graphical user interface. This may be useful if the server is accessed remotely. 1. Open a console and navigate to the folder containing the.jar file. 2. Execute the following command: java -jar <jar name> -console then follow the instructions to complete the installation. Installing in Unattended mode The Unattended mode is used to automate the installation process. This mode performs an unattended installation that does not prompt the user for any information. Prerequisite: When installing the MDM server using the graphical installer (see the previous procedure), you have generated an installation script during the final step of the wizard. This script is an.xml file that contains the installation configuration settings: in this example, the script has been named auto_install.xml. 1. Open a console and navigate to the folder containing the.jar file. 2. Execute the following command: java -jar <jar name> <script name> The installation script is passed as a parameter to the installer and the installation is performed in Unattended mode. Talend Installation and Upgrade Guide 11

18 Launching the Studio Launching the Studio Launch the Studio On Windows, double-click the executable file to launch Talend Studio. On Unix-like systems, add execution rights on the desired TOS_MDM-* binary before launching it. On a standard Linux box, the command is: $ chmod +x TOS_MDM-linux-gtk-x86.sh $./TOS_MDM-linux-gtk-x86.sh On Mac OS X, launch the following file: TOS_MDM-macosx-cocoa.app/Contents/MacOS/TOS_MDM-macosx-cocoa Public license First screen is a license screen. In the [License] window that appears, read and accept the terms of the license agreement to proceed to the next step. Login and first project 1. As first time user, you need to set up a new project or you can also import a Demo project which gathers numerous job samples. To select a demo project, select MDMPRODUCTDEMO or TALENDDEMOSJAVA and click Import... To create a new project, enter the name of your project in the corresponding field and click Create... to complete the description of your project. 2. In the Project name field, type in the name of the project. In the Project description field, type in a description for this project. Click Finish when complete, and the newly created project is displayed in the Login window. 12 Talend Installation and Upgrade Guide

19 Launching the Talend MDM Web User Interface 3. In the Login window, open the project you just created. A registration window opens. If required, follow the instructions provided to join the Talend community or click Skip to open a welcome window and launch the Studio Launching the Talend MDM Web User Interface You need to define at least one data model and one data container in the Studio for the Web Interface to work. 1. Use the URL: (replace localhost with the actual machine name if it is not running locally) 2. You may use the user/password: user / user. Other valid users are in the login-config.xml file. 3. You may change the current data model and container by sliding open the collapsible panel on the right. For more information about how to log in the Talend MDM Web User Interface, please download the Web User Interface User Guide from the User Manuals part of the Download page on the Talend Website Configuring Talend Studio Talend Studio requires specific third-party Java libraries or database drivers (.jar files) to be installed to connect to sources and targets. Those libraries or drivers, known as external modules, can be required by some of Talend components or by some connection wizards or by both. Due to license restrictions, Talend may not be able to ship certain external modules within Talend Studio Identify required external modules On your design workspace, if a component requires the installation of external modules before it can work properly, a red error indicator appears on the component. With your mouse pointer over the error indicator, you can see a tooltip message showing which external modules are required for that component to work. Talend Installation and Upgrade Guide 13

20 Identify required external modules When you open the Basic settings or Advanced settings view of a component for which one or more external modules are required, you will see a piece of highlighted information about external modules, followed by an Install button. Clicking the Install button opens a wizard that will show you the external modules to be installed. The Modules view lists all the modules required to use the components embedded in the Studio, including those Java libraries and drivers that you must install to get the relevant components or Metadata connection working. If the Modules view is not shown under your design workspace, go to Window > Show View > Talend and then select Modules from the list. The table below describes the information presented in the Modules view. Column Status Description points out if a module is installed or not installed on your system. The icon indicates that the module is not necessarily required for the corresponding component or Metadata connection listed in the Context column. Context Module Description Required The icon indicates that the module is absolutely required for the corresponding componentor Metadata connection. lists the name of Talend componentor Metadata connection using the module. If this column is empty, the module is then required for the general use of Talend Studio. This column lists any external libraries added to the routines you create and save in the Studio library folder. For more information, see the Talend Studio User Guide. lists the module exact name. explains why the module/library is required. the selected check box indicates that the module is required. In addition to the Modules view, the Studio provides a mechanism that enables you to easily identify, download and install most of the required third-party modules from the Talend website and directs you to valid websites for the rest. A Jar installation wizard appears whenever any required external module is found missing for any feature in the Studio, including when you: drop a component from the Palette if one or more external modules required for that component to work are missing in the Studio, or click the Check button in a Metadata connection setup wizard in Talend Studio if one or more external modules required for the connection are missing in the Studio, or click the Guess schema button in the Component view of a component if one or more external modules required for that component to work are missing in the Studio, click Install on the top of the Basic settings or Advanced settings view of a component for which one or more required external modules are missing, 14 Talend Installation and Upgrade Guide

21 Identify required external modules run a Job that involves components or Metadata connections for which one or more required external modules are missing, or click the button in the Modules view. When you click this button, the wizard that appears will list all the required external modules that are not integrated in the Studio. The table below describes the information presented in the wizard. Item Jar Module Required by component Required License More information Action Description The file name of the external module. A short description about the nature of the module. Lists the components that require the external module. The selected check box indicates that the module is required. The license under which the module is provided. Provides the URL of the valid website where you can find more information about this module and download the module manually. : Click to open the [Download external modules] dialog box to download and install the module, which is available on the Talend website; : Click the link to open the valid website to download the module, which is not available on the Talend website, and then click the jar button to import the downloaded module into your studio; Do not show again : You need to find and download the module yourself and click the jar button to import it into the your studio. Click to open the [Download external modules] dialog box to download and install all the required modules that are available on the Talend website. Select to prevent the wizard from appearing again unless you click the Modules tab view. button in the This check box shows only when you drop a component, set up a connection, or guess the schema of a database, that requires an external module, or click the Install button on the Component tab of a component that requires an external module. Talend Installation and Upgrade Guide 15

22 Install external modules Item Description Click here to obtain more Click to go to Talend online documentation on installing third-party modules. information about external modules This wizard lists the external modules to be installed, the licenses under which they are provided, and the URLs of the valid websites where they are downloadable, and allows you to download and install automatically all the modules available on the Talend website and download those not available on the Talend website by following the links provided in the Action column and then install them into your Studio manually. When you drop a component, set up a connection, or guess the schema of a database, that requires an external module for which neither the Jar file nor its download URL information is available on the Talend website, the Jar installation wizard does not appear, but the Error Log view will present an error message informing you that the download URL for that module is not available. You can try to find and download it by yourself, and then install it manually into the Studio. To show the Error Log view on the tab system, go to Window > Show views, then expand the General node and select Error Log Install external modules To download and install modules in the Studio To download and install external modules automatically, do the following: If you are working behind a network proxy, make sure you have correctly set up your proxy before you can download and install external modules in your Studio. To access the proxy settings, select Window > Preferences from the menu to open the Preferences window, then expand the General node and click Network Settings. 1. In the Jar installation wizard, click the Download and Install button to install a particular module, or click the Download and install all modules available button to install all the available modules. The [Download external modules] dialog box opens. 16 Talend Installation and Upgrade Guide

23 Install external modules 2. To download and install the external module(s) provided under a particular license, select that license from the Licenses pane, review the license terms, select the I accept the terms of the license agreement option, and click Finish to start the download and installation process. To download and install all external modules provided under all the listed licenses, click the Accept all button to start the download and installation process. Upon installation of the chosen external module or modules, a dialog box appears to notify you about the number of modules successfully installed and/or about the modules failed to install, if any. To install manually an external module you already have in your local file system, do the following: 1. Click the button in the upper right corner of the Modules view or in Jar installation wizard to browse your local file system. If the Modules view is not shown under your design workspace, go to Window > Show View > Talend and then select Modules from the list. 2. In the [Open] dialog box of your file system, browse to the module you want to install, double-click the.jar file, or select it and then click Open to install it. The dialog box closes and the selected module is installed in the library folder of the current Studio. You can now use the component or Metadata connection dependent on this module in any of your Job designs. To install modules in CommandLine If you use the Studio and CommandLine on different machines, you need to retrieve the downloaded.jar files and add them in CommandLine. Talend Installation and Upgrade Guide 17

24 Install external modules 1. Make sure CommandLine is not started, then download the external modules from the Modules view as explained in the previous procedure. 2. Copy the downloaded.jar files from <StudioPath>/lib/java and paste them into <CommandLinePath>/ lib/java, where <StudioPath> and <CommandLinePath> are the installation directories of the Studio and CommandLine respectively. Note that the <CommandLinePath>/lib/java folder is not created by default, it is created the first time you start the CommandLine application. 3. Restart CommandLine. You can now use the component or Metadata connection dependent on these modules. To install modules downloaded from external websites Some modules are not available on the Talend website but can be downloaded directly from external websites. Once downloaded, these modules must be placed in specific folders. For the studio, the downloaded modules must be placed in the following folder: <StudioPath>/lib/java For the MDM Server, the downloaded JDBC drivers for the Oracle and MySQL databases must be placed in the following folder: <JbossPath>/server/default/lib 18 Talend Installation and Upgrade Guide

25 Chapter 3. Upgrading your Talend products This chapter describes the various operations required to migrate version of the Talend solutions. We assume that you have installed and configured these solutions as described in the chapter Installing Talend Studio for the first time. The migration and upgrade process includes the following mandatory steps: These steps usually need to be completed in the following order. 1. Backing up the environment, see the section Backing up the environment. 2. Upgrading the Talend projects in the Studio, see the section Upgrading the Talend projects in the Studio. 3. Finally, for more information about the migration of MDM projects, see the section Migrating MDM projects. Talend Installation and Upgrade Guide

26 Backing up the environment 3.1. Backing up the environment Before you start migrating your Talend solutions, make sure your environment is correctly backed up. The environment backup process includes the following mandatory steps: These steps usually need to be completed in the following order. 1. Saving the local projects, see Saving the local projects. Saving the local projects 1. Launch the Studio. 2. Click the icon and export your local projects to an archive file Upgrading the Talend projects in the Studio Importing your local projects 1. Launch the new Studio you have just installed. 2. In the login window, select Import, then import the archive file containing your local projects. The local projects are displayed in the Project list and appear on the Studio Repository view. For more information on how to export local projects to an archive file, see the Saving the local projects Migrating MDM projects MDM system objects and master-records are stored in two databases. Depending on your installation, these may be relational or XML databases, or a combination of the two (although XML database support is now deprecated). You can export your data from an XML database and reimport it in a relational database. Since not everything is stored in a database, you must also manually import and redeploy the following items: Jobs, pictures, web resources. Note that Talend MDM data models only support certain types, as shown in the following table. When you migrate a project containing unsupported types, errors will occur. Supported types xsd:anyuri xsd:base64binary xsd:boolean Unsupported types xsd:entities xsd:gday xsd:gmonth 20 Talend Installation and Upgrade Guide

27 Manually moving system objects from an XML database to a relational database Supported types xsd:byte xsd:date xsd:datetime xsd:decimal xsd:double xsd:duration xsd:float xsd:hexbinary xsd:int xsd:integer xsd:language xsd:long xsd:negativeinteger - xsd:nonnegativeinteger - xsd:nonpositiveinteger - xsd:normalizedstring - xsd:positiveinteger - xsd:short - xsd:string - xsd:time - xsd:token - xsd:unsignedbyte - xsd:unsignedint - xsd:unsignedlong - xsd:unsignedshort - Unsupported types xsd:gmonthday xsd:gyear xsd:gyearmonth xsd:id xsd:idref xsd:idrefs xsd:language xsd:name xsd:nmtoken xsd:normalizedstring xsd:qname xsd:token You must delete your web browser cache and cookies whenever you change the version, or the Studio. Unpredictable behavior or display errors will occur if you do not. The following sections explain all the tasks you must carry out to perform a complete migration of all the system objects and all the data objects you have on the MDM server, including master-records, Jobs, pictures and web resources Manually moving system objects from an XML database to a relational database Prerequisite(s): You can carry out this operation with both MDM servers up and running provided you have a connection to each of them in the Server Explorer panel in Talend Studio, or you can perform the export operation from the server with the XML database, stop this server, start up the server with the relational database and complete the import operation. XML database support is now deprecated. If you need to migrate between two databases on a single machine, you must run the two MDM servers side by side. You can do this by setting a different port binding on the target server. To manually move your system objects, including the definitions of your Views, Triggers, Processes, the AUTO_INC counter and the MDM users, from an existing XML database to a relational database, you need to export the following system containers from your existing installation and reimport them in your new installation: Talend Installation and Upgrade Guide 21

28 Manually moving repository items from an XML database to a relational database System/CONF: AUTO_INCREMENT counters System/PROVISIONING: MDM users System/SearchTemplate: saved searches and hierarchies. Before performing this procedure, make sure you process the ACTIVE and FAILED queue, since the migration procedure will cause ACTIVE, FAILED and COMPLETED queues to be lost. To export these system containers, do the following: 1. In the MDM Repository tree view of Talend Studio that is connected to the server with the XML database, click the Import Server Objects from MDM button in the repository icon bar. 2. In the Import Server Objects from MDM Server window that opens, click the [...] button to specify the server from which you want to import the system containers. 3. Click the Deselect all button, expand Data Container > System, and select the CONF, PROVISIONING and SearchTemplate containers, and then click Finish to import these containers into your repository. 4. In the MDM Repository tree view of Talend Studio, expand Data Container and System, right-click the CONF container, and then click Export content from MDM Server. 5. Specify the server from which you want to perform the export action. 6. Browse to the location where you want to export the container, provide a file name for your.zip file, and then click Save. 7. Repeat the above steps for the two other containers, PROVISIONING and SearchTemplate. 8. In the MDM Repository tree view of Talend Studio that is connected to the server with the relational database, expand Data Container and System, right-click the CONF container, and then click Import content to MDM Server. 9. Browse to the location where you saved the.zip file containing your system containers, select the file you want to import, and then click Open. 10. Repeat the above steps for the two other containers, PROVISIONING and SearchTemplate Manually moving repository items from an XML database to a relational database Prerequisite(s): You can carry out this operation with both MDM servers up and running provided you have a connection to each of them in the Server Explorer panel in Talend Studio, or you can perform the export operation from the server with the XML database, stop this server, start up the server with the relational database and complete the import operation. XML database support is now deprecated. If you need to migrate between two databases on a single machine, you must run the two MDM servers side by side. You can do this by setting a different port binding on the target server. To manually move your repository items from an existing XML database to a relational database, do following: 1. In the MDM Repository tree view of Talend Studio that is connected to the server with the XML database, click the Export Items button in the repository icon bar. 22 Talend Installation and Upgrade Guide

29 Reimporting and redeploying your Jobs 2. In the Export Repository items window that opens, specify the location to which you want to export your repository items, select the items you want to export, and then click Finish. 3. In the MDM Repository tree view of Talend Studio that is connected to the server with the relational database, click the Import Items button in the repository icon bar. 4. In the Import Repository items window that opens, specify the location where the repository items you want to import are stored, select the items you want to import, and then click Finish Reimporting and redeploying your Jobs Prerequisite(s): Make sure that the MDM server is up and running. If you have Talend Jobs in your old MDM application, do the following to migrate these Jobs: 1. From the Integration perspective, import your local and remote Jobs as described in the section Upgrading the Talend projects in the Studio. 2. Deploy the Jobs to the new MDM server one by one. For further information, see the Talend Studio User Guide. You can also copy/paste the job scripts (.war or.zip) from their corresponding folder in the old application to the same folder in the new application: jboss ga/server/default/deploy for wars and jboss ga/jobox/deploy for zips. But this will not import the job design that you may need at some point. Another limitation with this copy/paste mode is that it is recommended only between two MDM servers that have the same major version (first number of the unique identifier of the version). If the major versions differ, it is very likely that the MDM components will not work with the new MDM Server. If you are migrating between 2 identical versions or 2 versions where only the minor version differs, however, copying the wars or zips will be a lot faster than redeploying the Jobs Moving the pictures and web resources Prerequisite(s): Make sure that both MDM servers are not running. If you use pictures in your data-model, do the following to migrate them to the new MDM server: 1. In the Jboss folder of the old MDM version, browse to: jboss ga/server/default/deploy/zz.50.ext.imageserver.war/upload 2. Copy the upload folder of the old MDM version and paste it in jboss ga/server/default/data/ mdm_resources/upload. 3. Launch the MDM server and then Talend Studio of the new MDM version as usual and you should have access to the migrated data objects. If you use web resources (images, css, js, etc.) in your smart views, do the following to migrate them to the new MDM server: 4. In the Jboss folder of the old MDM version, browse to: jboss ga/server/default/deploy/jboss-web.deployer/root.war 5. Copy the web resources from the old MDM version and paste them in the same path in the new MDM version. 6. Launch the MDM server and then Talend Studio of the new MDM version as usual and you should have access to the migrated data objects. Talend Installation and Upgrade Guide 23

30 Talend Installation and Upgrade Guide

31 Appendix A. Supported Third-Party System/ Database/Business Application Versions This document provides the information about the versions of the systems or databases or business applications supported by Talend Studio. Talend Installation and Upgrade Guide

32 Supported systems and databases A.1. Supported systems and databases The access to these systems and databases varies depending on the Studio you are using. Systems/Databases Versions OS Amazon Redshift Initial release of Amazon Redshift N/A 1 AS/400 V5R2 to V5R4 N/A 1 AS/400 V5R3 to V6R1 N/A 1 Access 2003 Windows Access 2007 Windows DB Generic ODBC Windows DB2 9.5/9.7 Windows + Linux EXASolution 4 Windows FireBird 2.1 Windows + Linux Greenplum Windows (client only) + Linux HSQLDb N/A 1 Hive Hive 1 (HiveServer) 3 The security information is not available to standalone servers. Hortonworks Data Platform V1.0.0 Kerberos (kinit and keytab) (deprecated) Hortonworks Data Platform V1.2.0 (Bimota) Kerberos (kinit and keytab) Hortonworks Data Platform V1.3.0 (Condor) Kerberos (kinit and keytab) Hortonworks Data Platform V2.0.0 Kerberos (kinit and keytab) (BigWheel) Hortonworks Data Platform V2.1.0 (Baikal) Kerberos (kinit and keytab) Windows + Linux Apache (0.9.0) Apache (0.7.1) Cloudera CDH3 (deprecated) Windows + Linux Windows + Linux Kerberos (kinit and Cloudera CDH4 (from 4.0 to 4.2) keytab) Cloudera CDH4.3 + (YARN mode) Cloudera CDH5.0 (YARN mode) MapR 1.2 (deprecated) Windows + Linux MapR 2.0 MapR MapR MapR MapR EMR MapR (deprecated) EMR Apache Pivotal HD Linux Linux Linux Pivotal HD 2.0 Custom 2 26 Talend Installation and Upgrade Guide

33 Supported systems and databases Systems/Databases Versions OS Hive2 (HiveServer) 3 Standalone servers only support the Kerberos authentication without a specific keytab. Hortonworks Data Platform V1.2.0 (Bimota) Kerberos (kinit and keytab) Hortonworks Data Platform V1.3.0 (Condor) Kerberos (kinit and keytab) Hortonworks Data Platform V2.0.0 Kerberos (kinit and keytab) (BigWheel) Hortonworks Data Platform V2.1.0 (Baikal) Kerberos (kinit and keytab) Kerberos (kinit and Cloudera CDH4 (from 4.0 to 4.2) keytab) Kerberos (kinit and Cloudera CDH4.3 + (YARN mode) keytab) Kerberos (kinit and Cloudera CDH5.0 (YARN mode) keytab) MapR MapR MapR Kerberos (kinit and keytab) Pivotal HD 2.0 Custom 2 Informix Windows + Linux Ingres 9.2 Windows + Linux Interbase 7 and above N/A 1 JavaDB 6 Windows + Linux LDAP No version limitation Windows + Linux MS SQL Server 2000/2003/2005/2008/2012 Windows + Linux MaxDB 7.6 N/A 1 MySQL Netezza Mysql4 Mysql5 Version 6 and earlier have been tested. No issues have been found with other versions until now. OleDb 2000/2003/2005/2007/2010 N/A 1 N/A 1 Windows + Linux Windows + Linux Windows + Linux Oracle Oracle 8i/9i/10g/11g/11g (11.6) Windows + Linux ParAccel 3.1/3.5 N/A 1 PostgreSQL 8.3 Windows + Linux PostgresPlus 8.3 Windows + Linux Salesforce until V26 Windows + Linux SAP 4.6 Windows SAP Hana 1.0 Windows SQLite Windows + Linux Sybase 12.5/12.7/15.2/15.5/15.7 Windows + Linux SybaseIQ 12.5/12.7/15.2 Windows + Linux Teradata 12/13/14/15 Windows + Linux VectorWise 2 Windows + Linux Vertica 3/3.5/4/4.1/5.0/5.1/6.0/6.1.X/7.0.X Windows + Linux exist 1.4 Windows 32bit + Linux 32bit Talend Installation and Upgrade Guide 27

34 Supported databases for profiling data Kerberos (kinit and keytab): The Kerberos authentication with a specific keytab is supported. Kerberos (kinit only): The Kerberos authentication without a specific keytab is supported. 1. The test information is not available yet. 3. The Profiling perspective does not support the Embedded connection mode on Hive distributions. This mode is available mainly for test purposes done by Hadoop developers. The studio may not be able to run correctly with the embedded mode. For further information about what Hadoop distributions are supported from the Profiling perspective, see section Supported Hive distributions for profiling data. A.2. Supported databases for profiling data The table below lists the databases supported from the Profiling perspective of the studio. For a complete list about supported third-party systems, see section Supported systems and databases. Database name AS/400 Hive IBM DB2 9.5/9.7 Database version V5R2 to V5R4 IBM DB2 ZOS 2000/2005/2008 Informix Ingres 9.2 Microsoft SQL Server 2000/2003/2005/2008/2012 MySQL 5.1/5.5/5.6 Netezza 6 Oracle with SID Oracle with service name For a complete list of the supported Hive distributions, see section Supported Hive distributions for profiling data. 9i to 11g 9i to 11g PostgreSQL 8.3/9.1+ SQLite Sybase (ASE and IQ)QLite 12.5/12.7/15.2 Teradata 12/13/14/15 Vertica 6.0/7.0 If you want to profile another version of Vertica, you must first change the driver version from the Preference page. Go to Talend > Data Explorer > JDBC Driver and modify the class path of the driver. Then restart the studio. A.3. Supported Hive distributions for profiling data Below is a list of the supported distributions for Hive profiling. For the time being, the embedded mode on Hive distributions is available mainly for test purposes done by Hadoop developers. The studio may not be able to run correctly with the embedded mode. 28 Talend Installation and Upgrade Guide

35 Supported business applications HortonWorks Cloudera MapR Apache Pivotal HD Hive distribution version Hive 1 Hive 2 HDP (deprecated) Embedded and standalone HDP 1.2 Embedded and standalone Embedded and standalone HDP 1.3 Embedded and standalone Embedded and standalone HDP 2.0 HDP 2.1 Embedded (only Linux) and standalone Embedded (only Linux) and standalone CDH4 (Kerberos Embedded and Standalone authentication is supported) CDH5 (Kerberos Embedded and Standalone authentication is supported) MapR 1.2 Standalone MapR 2.0 Embedded and Standalone MapR Embedded and Standalone Embedded (only Linux) and Standalone (Linux and Windows) Embedded (only Linux) and Standalone (Linux and Windows) Embedded and Standalone Embedded and Standalone MapR Embedded and Standalone Embedded and Standalone MapR 3.1 Embedded and Standalone Embedded and Standalone Apache (Hive 0.9.0) Embedded and Standalone Apache (Hive 0.7.1) Standalone Pivotal HD Standalone Pivotal HD Embedded (only Linux) Embedded and Standalone (only and Standalone (Linux and Linux) Windows) A.4. Supported business applications Business applications Versions Alfresco 2.1 SAP sap jco 2.* sap jco 3.* VtigerCRM Vtiger 5.0 Vtiger 5.1 Talend Installation and Upgrade Guide 29

36 Talend Installation and Upgrade Guide