Progress DataDirect for ODBC for Apache Hive Wire Protocol Driver

Transcription

1 Progress DataDirect for ODBC for Apache Hive Wire Protocol Driver Quick Start for UNIX/Linux Release 8.0.0

2

3 Copyright 2017 Progress Software Corporation and/or one of its subsidiaries or affiliates. All rights reserved. These materials and all Progress software products are copyrighted and all rights are reserved by Progress Software Corporation. The information in these materials is subject to change without notice, and Progress Software Corporation assumes no responsibility for any errors that may appear therein. The references in these materials to specific platforms supported are subject to change. Business Making Progress, Corticon, DataDirect (and design), DataDirect Cloud, DataDirect Connect, DataDirect Connect64, DataDirect XML Converters, DataDirect XQuery, Deliver More Than Expected, Icenium, Kendo UI, Making Software Work Together, NativeScript, OpenEdge, Powered by Progress, Progress, Progress Software Developers Network, Rollbase, SequeLink, Sitefinity (and Design), SpeedScript, Stylus Studio, TeamPulse, Telerik, Telerik (and Design), Test Studio, and WebSpeed are registered trademarks of Progress Software Corporation or one of its affiliates or subsidiaries in the U.S. and/or other countries. AccelEvent, Analytics360, AppsAlive, AppServer, Arcade, BravePoint, BusinessEdge, DataDirect Spy, DataDirect SupportLink, DevCraft, DigitalFactory, Fiddler, Future Proof, High Performance Integration, JustCode, JustDecompile, JustMock, JustTrace, OpenAccess, ProDataSet, Progress Arcade, Progress Profiles, Progress Results, Progress RFID, Progress Software, ProVision, PSE Pro, SectorAlliance, Sitefinity, SmartBrowser, SmartComponent, SmartDataBrowser, SmartDataObjects, SmartDataView, SmartDialog, SmartFolder, SmartFrame, SmartObjects, SmartPanel, SmartQuery, SmartViewer, SmartWindow, WebClient, and Who Makes Progress are trademarks or service marks of Progress Software Corporation and/or its subsidiaries or affiliates in the U.S. and other countries. Java is a registered trademark of Oracle and/or its affiliates. Any other marks contained herein may be trademarks of their respective owners. Please refer to the readme applicable to the particular Progress product release for any third-party acknowledgements required to be provided in the documentation associated with the Progress product. Updated: 2017/03/15 3

4 Copyright 4

5 Quick Start: Progress DataDirect for ODBC for Apache Hive Wire Protocol Driver for UNIX/Linux This quick start provides basic information that allows you to install and test connect with your driver. To take full advantage of the features and functionality available for your driver, refer to the product documentation library at: Progress DataDirect Documentation Library Note: OEM CUSTOMERS: Refer to the Progress DataDirect for ODBC Drivers Distribution Guide for information on installing, branding, unlocking, and distributing your branded drivers. This quick start covers the following topics: Before You Start on page 5 Requirements and Support on page 6 Downloading the Driver on page 6 Installing the ODBC Driver on UNIX/Linux on page 7 Environment Configuration on page 9 Configuring a Data Source on page 9 Testing the Connection on page 10 Tuning for Performance on page 10 Troubleshooting Setup/Connection Issues on page 12 Additional Resources on page 13 Before You Start Before you get started, you need the following: Appropriate user permissions to modify your environment and to read, write, and execute various files in the DataDirect for ODBC installation directory. Connection information: Host Name: The name or the IP address of the server to which you want to connect. Database: The name of the Apache Hive database to which you want to connect by default. The database must exist, or the connection attempt will fail. Logon ID: The default user ID that is used to connect to your database. Port Number: The port number of the server listener. 5

6 Chapter 1: Quick Start: Progress DataDirect for ODBC for Apache Hive Wire Protocol Driver for UNIX/Linux For licensed installations, you will also need the following information that was provided by Progress DataDirect: IPE Key (control number) Serial Number Requirements and Support Driver/Client Software Requirements: The driver has no client requirements. Supported Data Sources: Supports Apache Hive version 1.0 and higher against the following distributions: Amazon Elastic MapReduce (Amazon EMR), version 4.0 and higher Apache Hadoop Hive, version 1.0 and higher Cloudera's Distribution Including Apache Hadoop (CDH), version 5.4 and higher Hortonworks Data Platform (HDP), version 2.3 and higher IBM BigInsights, version 4.0 and higher MapR Distribution for Apache Hadoop, version 5.0 and higher Pivotal HD Enterprise (PHD), version 3.0 and higher For the latest information on supported data sources, visit the Progress DataDirect Supported Configurations page. Downloading the Driver To download the ODBC Apache Hive driver: 1. Visit the Progress DataDirect Connectors Download page. 2. Enter Hive in the search field. 3. Select ODBC for the interface. 4. When prompted for your OS and architecture, select the bitness and platform your application runs on. 5. Fill in the registration form with your contact information. 6. Review the End User License Agreement. If you agree, select the corresponding box; then, click Download. The installer program has been downloaded. See "Installing the ODBC Driver on UNIX/Linux" for instructions on installing the driver. See also Installing the ODBC Driver on UNIX/Linux on page 7 6

7 Installing the ODBC Driver on UNIX/Linux Installing the ODBC Driver on UNIX/Linux This section provides instructions for installing your downloaded files from a command-line interface. Depending on your environment, you can also install using GUI or silent installation options. Refer to the Progress DataDirect for ODBC Drivers Installation Guide for complete installation instructions. 1. Download the appropriate product file from the Web site into a temporary directory, for example, /tmp. 2. Switch the temporary directory; then, choose one of the following: 3. Extract the contents of the product file. Enter: tar -xvzf product_filename.tgz where product_filename is the name of the tarred product file name. The untarred files appear in the temporary directory. 4. Run the installer program by switching to the temporary directory that contains the untarred installer files and entering:./installer_filename.bin i console where: installer_filename is the full name of the installer's binary file you untarred in Step 3 on page The Introduction step appears. Press ENTER. 6. The product license agreement appears. Press SPACEBAR to page to the end of the agreement. At the end, you are asked to accept the agreement: Enter Y to accept the license agreement and continue with the installation. Enter N to abort the installation. Note: You can exit the installer program at any time by typing quit or return to the previous step by typing previous. 7. You are prompted to enter the absolute path to the installation directory. The default is: For 32-bit installations: opt/progress/datadirect/odbc_nn_32bit For 64-bit installations: opt/progress/datadirect/odbc_nn_64bit If you do not have /opt directory permissions, the installer program installs the drivers to your home directory by default. For example, the directory for 32-bit installation would be: /home_directory/progress/datadirect/odbc_nn_32bit where 7

8 Chapter 1: Quick Start: Progress DataDirect for ODBC for Apache Hive Wire Protocol Driver for UNIX/Linux nn is the product version number. For example, this value is 80 for the 8.0 version of the product. home_directory is the home directory for the user who is installing the product. 8. You are prompted for the type of installation: If you are installing an evaluation copy of the product (expires in 15 days), enter 1. Then, press Enter to confirm the evaluation information. Skip to Step 12 on page 8. If you are installing a OEM or licensed copy of the product, enter 2. Proceed to the next step. Note: If you were issued multiple product keys, you can type them sequentially, separating the keys with a space. For example: xxxxkey1 xxxxkey2 xxxxkey3 9. You are prompted to enter the IPE key (control number) that was provided by Progress DataDirect. Type the IPE key (control number), including dashes, and then press ENTER. Proceed to the next step. 10. The DataDirect Installer Customer Experience Improvement Program step appears. By choosing to participate in the program, you permit the installer to gather data about your installation, such as environment information and error details, that will be used to improve our products. Choose one of the following: If you would like to participate in the program, enter 1. If you do not want to participate in the program, enter 2. For more information about data collection, refer to the Progress Privacy Policy. 11. The installer prompts you for product registration information. Enter your name, your company name, and the serial number provided by Progress DataDirect; then, press ENTER. 12. The Pre-Installation Summary step provides the opportunity for you to review the information you have entered, and you are prompted to accept or change the information: Press ENTER to accept the information and begin the installation. To change the information, type back and then press ENTER. You are prompted for the information again. If you are installing with a licensed OEM IPE key, you are prompted for branding information. Please refer to the Progress DataDirect for ODBC Drivers Distribution Guide. 13. After completion of the installation, a message appears indicating that you have installed the software successfully. Press ENTER to close the installer. This completes the installation. After installation, you must do the following: Configure your environment Configure a data source See "Environment Configuration", "Setting the Library Path Environment Variable" and "Configuring a Data Source" for details. See also Environment Configuration on page 9 8

9 Environment Configuration Configuring a Data Source on page 9 Environment Configuration After installing your driver, your first step in setting up and configuring the driver is to configure your environment variables. The following procedures require that you have the appropriate permissions to modify your environment and to read, write, and execute various files. You must log in as a user with full r/w/x permissions recursively on the entire DataDirect Connect Series for ODBC installation directory. To configure the environment: 1. Check your permissions: You must log in as a user with full r/w/x permissions recursively on the entire product installation directory. 2. From your login shell, determine which shell you are running by executing: echo $SHELL 3. Run one of the following product setup scripts from the installation directory to set variables: odbc.sh or odbc.csh. For Korn, Bourne, and equivalent shells, execute odbc.sh. For a C shell, execute odbc.csh. After running the setup script, execute: env to verify that the installation_directory/lib directory has been added to your shared library path. 4. Set the ODBCINI environment variable. The variable must point to the path from the root directory to the system information file where your data source resides. The system information file can have any name, but the product is installed with a default file called odbc.ini in the product installation directory. For example, if you use an installation directory of /opt/odbc and the default system information file, from the Korn or Bourne shell, you would enter: ODBCINI=/opt/odbc/odbc.ini; export ODBCINI From the C shell, you would enter: setenv ODBCINI /opt/odbc/odbc.ini Configuring a Data Source The default odbc.ini file installed in the installation directory is a template in which you create data source definitions. You enter your site-specific database connection information using a text editor. Each data source definition must include the keyword Driver=, which is the full path to the driver. 9

10 Chapter 1: Quick Start: Progress DataDirect for ODBC for Apache Hive Wire Protocol Driver for UNIX/Linux The following examples show the minimum connection string options that must be set to complete a test connection, where xx represents iv for 32-bit or dd for 64-bit drivers, and yy represents the extension. The values for the options are samples and are not necessarily the ones you would use. [ODBC Data Sources] Apache Hive Wire Protocol=DataDirect 8.0 Apache Hive Wire Protocol Driver [Apache Hive Wire Protocol] Driver=ODBCHOME/lib/xxhiveyy.zz Database=default HostName=HiveServer LoginID=yourid MaxStringSize= PortNumber=10000 Connection Option Descriptions: Database: The name of the Apache Hive database to which you want to connect by default. The database must exist, or the connection attempt will fail. HostName: Either the name or the IP address of the server to which you want to connect. LogonID: The default user ID that is used to connect to your database. PortNumber: The port number of the server listener. The default port number for the Apache Hive server is Because of reported concurrency issues, you might want to use a different port number. MaxStringSize: (This option is required only for Microsoft Access and Tableau users.) The maximum size of columns of the String data type that the driver describes through result set descriptions and catalog functions. Specify the following value that corresponds to your environment: For Microsoft Access users, specify a value of 255. For Tableau users, specify a value from 255 to 4000 that suits your environment. Testing the Connection The driver installation includes an ODBC application called example that can be used to connect to a data source and execute SQL. The application is located in the installation_directory/example directory. To run the program after setting up a data source in the odbc.ini, enter example and follow the prompts to enter your data source name, user name, and password. If successful, a SQL> prompt appears and you can type in SQL statements such as SELECT * FROM table. If example is unable to connect, the appropriate error message is returned. Tuning for Performance The connection options described in this section directly affect the performance of your driver. To tune for performance, configure your driver according to the recommended settings and your environment. ArraySize Purpose: 10

11 Tuning for Performance Determines the number of cells the driver retrieves from a server for a fetch. When executing a fetch, the driver divides the value specified by the number columns in a particular table to determine the number of rows to retrieve. Performance Impact: To improve throughput, increase the value of Array Size. By increasing the value of Array Size, you increase the number of rows the driver will retrieve from the server for a fetch. In turn, increasing the number of rows that the driver can retrieve reduces the number, and expense, of network round trips. Note that improved throughput does come at the expense of increased demands on memory and slower response time. Furthermore, if the fetch size exceeds the available buffer memory of the server, an out of memory error is returned when attempting to execute a fetch. If you receive this error, decrease the value specified until fetches are successfully executed. Recommended Settings: Tune this setting to reflect the typical fetch size of your application. Smaller fetch sizes can improve the initial response time of the query. Larger fetch sizes improve overall fetch times at the cost of additional memory. BatchMechanism Purpose: Determines the mechanism that is used to execute batch operations. Performance Impact: Unlike the native batch mechanism, the multi-row insert mechanism only returns the total number of update counts for batch inserts. Therefore, setting BatchMechanism to MultiRowInsert offers substantial performance gains when performing batch inserts. Recommended Settings: If your application does not require individual update counts for each statement or parameter set in the batch, set to 2 (MultiRowInsert) for improved performance when executing batch inserts. CatalogMode Purpose: Determines whether the driver uses native catalog functions to retrieve information returned by the SQLTables, SQLColumns, and SQLStatistics catalog functions. Performance Impact: Apache Hive s native catalog functions return incorrect information in certain scenarios. To address this issue, by default, the driver uses a combination of driver-discovered information and native functions to retrieve more accurate catalog information than native functions alone. While using driver-discovered information improves accuracy, it does so at an expense to performance. If accurate catalog information is not required, you can improve performance by setting Catalog Mode connection option to 1 (Native). Recommended Settings: If accurate catalog information is not required, set to 1 (Native) for the best performance. If your application requires accurate catalog information, set to 0 (Mixed) for the optimal balance of performance and accuracy. DefaultLongDataBuffLen Purpose: The maximum length of data (in KB) the driver can fetch from long columns in a single round trip and the maximum length of data that the driver can send using the SQL_DATA_AT_EXEC parameter. 11

12 Chapter 1: Quick Start: Progress DataDirect for ODBC for Apache Hive Wire Protocol Driver for UNIX/Linux Performance Impact: To improve performance when your application fetches images, pictures, or long text or binary data, a buffer size can be set to accommodate the maximum size of the data. The buffer size should only be large enough to accommodate the maximum amount of data retrieved; otherwise, performance is reduced by transferring large amounts of data into an oversized buffer. If your application retrieves more than 1 MB of data, the buffer size should be increased accordingly. Recommended Settings: Set DefaultLongDataBuffLen to a size no larger than the maximum size of data retrieved. EncryptionMethod Purpose: Determines the method the driver uses to encrypt data sent between the driver and the database server. Performance Impact: Data encryption may adversely affect performance because of the additional overhead (mainly CPU usage) that is required to encrypt and decrypt data. Recommended Settings: If data encryption is not required, set to 0 (No Encryption) for improved performance. Troubleshooting Setup/Connection Issues This section describes common setup/connection issues you may encounter while trying to establish a database connection with the driver as well as some potential reasons for these issues. If you are experiencing a problem not described in this section, comprehensive troubleshooting resources are available in the "Troubleshooting" section of the User's Guide and Reference for your driver. In addition, Progress DataDirect offers a variety of options to meet your technical support needs. See Contacting Technical Support on page 13 for details. Common Setup/Connection Issues You are experiencing a setup/connection issue if you are encountering an error or hang while you are trying to make a database connection with the ODBC driver or are trying to configure the ODBC driver. Some common errors that are returned by the ODBC driver if you are experiencing a setup/connection issue include: Specified driver could not be loaded. Data source name not found and no default driver specified. Unable to connect to destination. Invalid username/password; logon denied. Troubleshooting the Issue Some common reasons that setup/connection issues occur are: The database and/or listener are not started. 12

13 Additional Resources The ODBCINI environment variable is not set correctly for the ODBC drivers on UNIX and Linux. The ODBC driver s connection attributes are not set correctly in the system information file on UNIX and Linux. See "Configuring a Data Source" for more information. For example, the host name or port number are not correctly configured. See also Configuring a Data Source on page 9 Additional Resources In addition to this quick start, the following resources enable you to take full advantage of the features and support offered for your driver. Product Documentation Library contains a comprehensive set of product documentation, including the following guides: Progress DataDirect Connect Series for ODBC Installation Guide details requirements and procedures for installing the product. Progress DataDirect for ODBC for Apache Hive Wire Protocol Driver User's Guide and Reference guides you through using and configuring the driver, provides detailed reference information, and explains the tools used to troubleshoot common problems. Progress Support Knowledgebase provides answers to questions, access to technical documentation, release notes, product alerts and other support information. Progress Community allows you to contribute, share, and network with other Progress users and employees. Technical Support provides technical support services, including maintenance services and opening a support case. Contacting Technical Support Progress DataDirect offers a variety of options to meet your support needs. Please visit our Web site for more details and for contact information: The Progress DataDirect Web site provides the latest support information through our global service network. The SupportLink program provides access to support contact details, tools, patches, and valuable information, including a list of FAQs for each product. In addition, you can search our Knowledgebase for technical bulletins and other information. When you contact us for assistance, please provide the following information: Your number or the serial number that corresponds to the product for which you are seeking support, or a case number if you have been provided one for your issue. If you do not have a SupportLink contract, the SupportLink representative assisting you will connect you with our Sales team. Your name, phone number, address, and organization. For a first-time call, you may be asked for full information, including location. The Progress DataDirect product and the version that you are using. The type and version of the operating system where you have installed your product. 13

14 Chapter 1: Quick Start: Progress DataDirect for ODBC for Apache Hive Wire Protocol Driver for UNIX/Linux Any database, database version, third-party software, or other environment information required to understand the problem. A brief description of the problem, including, but not limited to, any error messages you have received, what steps you followed prior to the initial occurrence of the problem, any trace logs capturing the issue, and so on. Depending on the complexity of the problem, you may be asked to submit an example or reproducible application so that the issue can be re-created. A description of what you have attempted to resolve the issue. If you have researched your issue on Web search engines, our Knowledgebase, or have tested additional configurations, applications, or other vendor products, you will want to carefully note everything you have already attempted. A simple assessment of how the severity of the issue is impacting your organization. 14