Ryft JDBC Connector. Installation and User Guide. Ryft Document Number: Document Version: Revision Date: November 2016

Transcription

1 Ryft JDBC Connector Installation and User Guide Ryft Document Number: 1193 Document Version: Revision Date: November Ryft Systems, Inc. All Rights in this documentation are reserved. RYFT SYSTEMS, INC.

2 Copyright (c) 2016, Ryft Systems, Inc. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. All advertising materials mentioning features or use of this software must display the following acknowledgement: This product includes software developed by Ryft Systems, Inc. Neither the name of Ryft Systems, Inc. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY RYFT SYSTEMS, INC. ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RYFT SYSTEMS, INC. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Ryft Systems, Inc. Page ii

3 Revision History Date Reason for Change Version Nov Initial Release Ryft Systems, Inc. Page iii

4 Contents Using This Guide... 5 Ryft Technical Support... 5 About Ryft Systems Inc : Overview : Install Ryft Connector for JDBC... 9 Summary of Steps... 9 Step 1: Create Directory on Java Classpath... 9 Step 2: Download/Unzip the file and Review the README file Step 3: Modify ryftone.server.ini File (Optional) Step 4: Test the JDBC Client/Server Connection Connection String Parameters Example: Connect to Ryft ONE Using Pentaho Install Ryft JDBC Driver Connect to Ryft Using Pentaho User Console : Troubleshooting Appendix A: Commonly Used Acronyms and Terms Appendix B: Open Source rd Parties Used in Deployment: Open Source rd Parties Used in Deployment: 3 rd Party Purchased Ryft-Developed Code Optional Ryft-Developed Open Source Ryft Systems, Inc. Page iv

5 Using This Guide This guide is for the System Administrator responsible for installing the Ryft Connector on your Ryft ONE server and/or clients, and making Ryft ONE accessible to your end users. It s also for the end user who will ultimately use their business intelligence and data visualization (BI/DV) tool to access the datasets on the Ryft ONE, taking advantage of all of the benefits of the Ryft ONE server. There are step-by-step instructions on installing the Ryft Connector and installing the sample datasets, and how to connect the BI/BA applications to the Ryft ONE. Additional resources: Ryft Open API Library User Guide and the Ryft ONE: User Guide. Ryft Technical Support You may access our first-tier support directly from our public website by using the Chat Widget and starting a chat with our support agents. All chat conversations are tracked and become help desk tickets in our support system. In addition, you can access our support system through the Ryft Support site: Log in with your credentials to see your past tickets, create new tickets, and to access limited-access content. For technical support, contact us: support@ryft.com Web: Phone: (RYFT ONE) About Ryft Systems Inc. Ryft is the performance leader in data-intensive computing with the first commercial HPC appliance to use hybrid x86/fpga compute, 48 TB of storage, and an Open API to accelerate and streamline existing software ecosystems by 100X. With more than a decade of experience delivering incredibly fast data analysis solutions to government intelligence agencies, Ryft is the only company that understands how to effectively apply the combination of FPGA compute acceleration and x86 integration to a broad set of data-intensive workloads from the Internet of Things, video cameras, web logs, customer data, and other sources. Industry heavyweights in retail, finance, defense, and health care trust Ryft to power a range of realtime intelligence applications. Ryft Systems, Inc. Page 5

6 This page intentionally left blank. Ryft Systems, Inc. Page 6

7 1 1: Overview Connect your business analytics/intelligence/visualization applications such as Tableau, Birst or Excel to the Ryft ONE server using Ryft Connectors for ODBC, JDBC or Apache Spark. This is done without the need for indexing or ETL and without the need to put the data into an SQL or nosql database inside the Ryft ONE. Instead, the Ryft ONE can work with structured files like XML and JSON, with future support for flat files, key values and CSV files, to make it look like the data is coming from an SQL database while being transparent to your business applications. The Ryft Connector: Supports the SQL-92 standard, which is supported by most major database vendors. Enables key features for analysis including: o Database tables and views o Column meta data o SQL commands (=,!=, LIKE, NOT LIKE) o Auto discovery of tables, views and metadata Client runs natively on Windows, Mac, and Linux-based infrastructures The Ryft Connector was built with the SimbaEngine SDK from Simba Technologies, which provides flexibility, convenience and connectivity to multiple BI tools, comprehensive support and excellent SQLquery coverage. Ryft Systems, Inc. Page 7

8 This guide focuses on the Ryft JDBC Connector. See Also: Ryft ODBC Connector: Installation and User Guide Ryft ONE Spark Integration and User Guide Ryft Systems, Inc. Page 8

9 2 2: Install Ryft Connector for JDBC The Ryft Connector for JDBC enables Java programs to execute the Ryft primitives and SQL statements, and to interact with the Ryft ONE server and datasets. The Ryft Connector for JDBC ( Ryft Connector ) supports LDAP, and is easy to download and install. NOTE: The Ryft JDBC Client requires Java 1.5 or higher. Follow the instructions provided by Ryft to download the zipped Ryft Connector for JDBC driver file, ryft1_jdbc_client_x.x.x.x, where x.x.x.x is the version number. Once downloaded, create a directory on the Java Classpath and unzip the file in the new directory. The ryft1_jdbc_client_x.x.x.x zip file contains 2 files: README RyftJDBCClient41 an executable Jar file. Summary of Steps Follow these steps to unzip and install the driver: 1. Create directory on Java Classpath 2. Download/unzip file and review README file 3. Modify ryftone.server.ini file (Optional) 4. Test JDBC Client/Server Connection Step 1: Create Directory on Java Classpath The Ryft JDBC Client requires Java 1.5 or higher. You can find a JDK at the Java website: Copy the RyftJDBCClient41.jar file to the target machine and make sure that it is on the java classpath. More information about setting the java classpath can be found here: Ryft Systems, Inc. Page 9

10 Step 2: Download/Unzip the file and Review the README file Doc #1193, V1.1.0/ November 2016 Download the file, according to the steps provided by Ryft, and unzip the file. Open the README file using your preferred viewer and review the contents. Here is a snippet: NOTE: The information provided in the Testing the JDBC Client/Server connection will be needed when integrating a BI tool, such as Pentaho, with the Ryft Server. You will need the IP address of the Ryft ONE server and the listening port number on the server. Ryft Systems, Inc. Page 10

11 Step 3: Modify ryftone.server.ini File (Optional) Doc #1193, V1.1.0/ November 2016 Next, you may choose to modify the.ryftone.server.ini file to match your installation. This is optional. You may want to update the following values in the file: Section Field Description/Notes [Server] LogLevel Logging Level. Default is 0, off. Available values are: 0 = Off 1 = Fatal; only fatal issues are logged 2 = Error; error conditions and fatal issues 3 = Warning; warnings, errors and fatal issues 4 = Informational; informational messages, warnings, etc. 5 = Debug; all debugging information, information messages, etc. 6 = Trace; function level entry points, debugging info, etc. LogPath Path to directory that will contain the log files. [Auth] Type Authorization type. Available values are: ldap if specified, set the next 4 values system none Ryft Systems, Inc. Page 11

12 Section Field Description/Notes LADPServer LDAPUser LDAPPassword Doc #1193, V1.1.0/ November 2016 URL for LDAP server (i.e. ldap://ldap.forumsys.com:389) Administrative user account name (i.e. read-only-admin) Password for the administrative user account listed above. LDAPBaseDN Base domain name used for account searches (i.e. dc=example,dc=com) [Network] ConnectionIdleTimeout Default is seconds ListenPort TCP port number used by the ODBC server. Modify the file, as needed, and then save and close the file. If your JDBC application runs on your desktop, such as Pentaho, then install the JDBC driver on your desktop. If you will use a JDBC application in a Linux environment, such as LibreOffice, then install the JDBC driver on your Linux server. Step 4: Test the JDBC Client/Server Connection You may need to provide the following, depending on the application you use to connect to the Ryft ONE: Classname: com.simba.client.core.jdbc41.jdbc41driver Connection URL: IP Address of Ryft ONE and port the server is listening on (e.g. jdbc:simba:// :7409 ). Connection String Parameters The Ryft Client for JDBC responds to configuration keywords similar to those for the Ryft Client for ODBC. However, since JDBC does not use the idea of a DSN, the user application must add the keywords and values to the connection string. Ryft Client for JDBC parses the connection string for configuration keywords, and extracts and acts on the values. The general form of the connection URL is: jdbc:simba://<serverhost>:<serverport>[;<property>=<value>[; ]] where [ ] indicates optional values. Note that additional values that need to be passed through the client, such as UID and PWD, can be specified in the property=value portion of the URL. Keyword ServerHost Description Required Data Type Description The hostname or IP address of the Ryft Server Yes String Ryft Systems, Inc. Page 12

13 Keyword Range Default Value Example Specification Comment Description Any valid hostname or IP address (No Default) or [2001:0db8:85a3:0000:0000:8a2e:0370:7334] This value must be present in the connection URL. Example: jdbc:simba://<host> where <HOST> is replaced with the appropriate value. Note that the use of [ ] is optional for IPv4 addresses, but required for IPv6 addresses. Ryft Client must be able to find the Ryft Server on the network and will try to make a network connection to this hostname or address. ServerPort Description Required Data Type The port on which Ryft Server is listening Yes Integer Range Default Value Example Specification Comment (No Default) ServerPort=1583 This value may be present in the connection URL. Example: jdbc:simba://<host>:<serverport> where the values in < > are replaced with the appropriate values. Ryft Client must be able to make a network connection to the host and port on which the Ryft Server is listening. LogLevel Description Required Data Type Range Level of log detail (how much information to log) No Enumeration Options are: OFF or 0 FATAL or 1 ERROR or 2 WARNING or 3 INFO or 4 DEBUG or 5 TRACE or 6 Ryft Systems, Inc. Page 13

14 Keyword Description Default Value OFF (0) Example Specification Comment LogLevel=DEBUG This value may be present in the connection URL. Example: jdbc:simba://<host>[...];loglevel=<levelvalue> where the values in < > are replaced with the appropriate values and [ ] indicates optional portions of the connection URL. With this keyword, you can control the amount of log output by controlling the kinds of events that are logged. Available options are: OFF (0): no logging occurs FATAL (1): only log fatal errors ERROR (2): log all errors WARNING (3): log all errors and warnings INFO (4): log all errors, warnings, and informational messages DEBUG (5): log method entry and exit points and parameter values for debugging TRACE (6): log all method entry points. LogPath Description Required Data Type Range Default Value Example Specification Comment Directory to store log files. No String Valid directory path or blank Blank. By default, will store log files in the current working directory. LogDirectory= C:\RyftJDBC This value may be present in the connection URL. Example: jdbc:simba://<host>[...];loglevel=<notoff>;logdirectory=<direc toryvalue> where the values in < > are replaced with the appropriate values and [ ] indicates optional portions of the connection URL. This keyword specifies where Ryft Client for JDBC will write all log files. If no path is specified then the log files will be written to the current working directory. Ryft Systems, Inc. Page 14

15 Example: Connect to Ryft ONE Using Pentaho Doc #1193, V1.1.0/ November 2016 This example shows you how to connect Pentaho to the Ryft ONE using the Ryft Connector for JDBC Install Ryft JDBC Driver Pentaho specifies where to install the Ryft JDBC driver. Refer to the Pentaho Help page for Adding a JDBC Driver, as shown. This step must be completed before adding a database connection. Connect to Ryft Using Pentaho User Console Start Pentaho and log into the system. From the main menu, click File > Manage Data Sources. Ryft Systems, Inc. Page 15

16 From Manage Data Sources, click New Data Source and the Data Source Wizard displays. Enter the Data Source Name for your Ryft ONE server, and then select Database Table(s) as the Source Type. The list of available database connections displays. You can name this for the table you connect to or the report you re going to create. So instead of using Ryft, as seen above, you can call it Ryft Passengers, for example, because you want to connect to the Ryft ONE Passengers data table. Pentaho only allows you to connect to one table at a time. Ryft Systems, Inc. Page 16

17 Ensure Reporting only is selected at the bottom, and then click +, the Add Connections icon, to add a database connection. Ryft Systems, Inc. Page 17

18 The Database Connection dialog box displays. Just beneath the Database Type selection box is the Adding Databases help link. Click the link to determine where to install the driver. For the Ryft ONE JDBC driver, copy it to the directory for the Pentaho BA Server on your local hard drive, as specified by Pentaho. Using the information provided in the Ryft JDBC README file, copy and paste the Connection URL into the Custom Connection URL field. Ensure you replace the generic the IP address in the Custom Connection URL with the specific IP address for your Ryft ONE server. Copy and paste the Class Name for the Ryft Client for JDBC in to the Custom Driver Class Name field. Ryft Systems, Inc. Page 18

19 The Database Type is Generic database and the Access is Native (JDBC). Enter the User Name and Password for your Ryft ONE server, and then click Test to verify connectivity. A message displays with the test results: Ryft Systems, Inc. Page 19

20 Click OK to the verification message. If the test failed, verify the information entered into the Database Connections fields and test again until you have a successful test. If the test was successful, click OK again on the Database Connection dialog box. The Data Source Wizard displays and now shows the newly added connection Ryft-310 in the list of Connections. Continue by specifying the Data Source Name and clicking Next to select from the Available Tables. Ryft Systems, Inc. Page 20

21 Select a table from the list of Available Tables and then click > to move it to the Selected Tables list on the right. Click Next to create the data source. Select Customize model now and then click OK. Pentaho displays the Data Source Model Editor and enables you to verify the table and fields Ryft Systems, Inc. Page 21

22 Once verified, click OK, and the Manage Data Sources screen displays. Verify that your newly added data source, Ryft, displays. Click Close. From the home page, select Create New > Interactive Report, and then select Ryft, your new data source: Ryft Systems, Inc. Page 22

23 Click OK and then create your report. Ryft Systems, Inc. Page 23

24 3 3: Troubleshooting The ODBC/JDBC Server creates the ryftone_server.log log file for all submitted transactions. By default, the log file is located in the /usr/local/ryft/bin/x8664 directory. You can change the location by updating the LogPath in the.ryftone.server.ini file. /usr/local/ryft/bin/x8664$ ls -al total drwxrwxr-x 2 ryftuser ryftuser 4096 Sep 6 17:10. drwxrwxr-x 4 ryftuser ryftuser 4096 Sep 6 16:38.. -rwxrwxr-x 1 ryftuser ryftuser Feb libsimbaclient.so -rw-r root root 5 Sep 6 17:14.r1odbcd.pid -rwsr-sr-x 1 root root May 25 19:34 ryft1_odbcctl -rwsr-sr-x 1 root root May 25 19:35 ryft1_odbcd -rw-rw-rw- 1 ryftuser ryftuser 588 Sep 6 16:48.ryftone.server.ini -rw-r--r-- 1 root root Sep 8 16:37 ryftone_server.log The logging level is controlled by the LogLevel parameter in the.ryftone.server.ini file. Options are: 0 = Off 1 = Fatal; only fatal issues are logged 2 = Error; error conditions and fatal issues 3 = Warning; warnings, errors and fatal issues 4 = Informational; informational messages, warnings, etc. 5 = Debug; all debugging information, information messages, etc. 6 = Trace; function level entry points, debugging info, etc. The default level for logging is "0" or no logging. The maximum setting is 6. The LogLevel should only be modified when you are troubleshooting an issue to capture necessary information. NOTE: If you modify the level, you must restart the ODBC server. /usr/local/ryft/bin/x8664$ more.ryftone.server.ini [Server] LogLevel=6 LogNamespace= LogPath=./ DriverLocale=en-US Ryft Systems, Inc. Page 24

25 DriverManagerEncoding=UTF-16 ErrorMessagesPath=../../errormessages [REST] RESTServer= #RESTUser=odbcuser #RESTPass=odbcpass [Auth] Type=none LDAPServer=ldap://ldap.forumsys.com:389 LDAPUser=read-only-admin LDAPPassword=password LDAPBaseDN=dc=example,dc=com [Admin] UseSignaling=Yes [Buffers] BufferAllocationSize=8192 PreAllocatedBuffers=1000 [Network] MaxConnections=64 ListenAddress= ConnectionIdleTimeout=86400 ListenPort=7409 UseSsl=0 SslCertfile= SslKeyFile= MaxDataChunkSize=65550 [Threads] MinWorkerThreads=10 TaskQueueTimeout=1000 Ryft Systems, Inc. Page 25

26 A Appendix A: Commonly Used Acronyms and Terms This guide frequently uses the following acronyms and terms: Word / Term Description API Application Programming Interface * Code that allows two software programs to communicate with each other. It defines the correct way for a developer to write a program that requests services from an operating system (OS) or other application. CLI Command Line Interface * A user interface to a computer s operating system or an application in which the user responds to a visual prompt by typing in a command on a specified line, receives response back from the system, etc. JDBC Java Database Connectivity A Java API that enables Java programs to execute SQL statements. This allows Java programs to interact with any SQL-compliant database. LDAP ODBC Spark RDD RDF Lightweight Directory Access Protocol A software protocol for enabling anyone to locate organizations, individuals, and other resources such as files and devices in a network, whether on the public Internet or on a corporate intranet. Open Database Connectivity A standard API for accessing database management systems. Built to be independent of database systems and operating systems. Spark Resilient Distributed Dataset A collection of elements partitioned across the nodes of the cluster that can be operated on in parallel. (See Spark Programming Guide: Record Definition File It defines how files with a particular extension should be treated by the Ryft system. These settings determine how the file is to be broken up for Ryft Systems, Inc. Page 26

27 Word / Term Description optimized reads/writes, and specifies whether the file is raw data, text, or structured text (records-based). When dealing with structured text, it is best to define an RDF for that file extension type, so that each record will stay intact as a single entity, and so that operations can be tailored to specific fields, if desired. REST API Representational State Transfer Application Programming Interface * An application program interface (API) that allows communication between a web-based client and server that employs representational state transfer (REST) constraints. It uses HTTP requests to GET, PUT, POST and DELETE data. ROL Ryft Open API Library The library that abstracts and manages Ryft ONE hardware resources. RHFS Ryft Hybrid File System The file system that manages data flow into and out of the Ryft ONE s advanced array of 48 solid state drives (SSDs). NOTE: * - Definitions from Ryft Systems, Inc. Page 27

28 B Appendix B: Open Source Ryft uses the following Open Sources in the deployment of the Ryft ONE server, obtained from third parties, that Ryft Systems, Inc. has developed and contribute as open source, and some that are optional open source for the Ryft ONE. 3 rd Parties Used in Deployment: Open Source Ansible simple IT automation platform that makes your applications and systems easier to deploy (if customer elects to deploy Ryft Cluster) Apache Spark (If customer elects to deploy the Spark connector) Consul cluster management software (if customer elects to deploy Ryft Cluster) DKMS driver management Fuse - file systems isql interactive SQL tool (only needed if customer elects to deploy the ODBC/JDBC Connector) SQL Lite used for settings database for core Ryft ONE: also used for simple database tool used for caching ODBC/JDBC connector. Swagger simple yet powerful framework for the RESTful API Ubuntu - open source operating system platform 3 rd Parties Used in Deployment: 3 rd Party Purchased Simba ODBC SDK (only needed if customer elects to deploy the ODBC connector) Ryft-Developed Code Ryft Open API the set of bindings description (headers, etc.) provided for C, C++, Java, see ryft.com/api for details (API document is a freely downloadable (no login required) PDF document.) Ryftprim an open source command line tool that wraps the Ryft Open API, which enables bash scripting that Ryft provides to invoke Ryft analytics functions from standard Ubuntu LTS Linux. Open source web-based demo software that Ryft created. JSON RESTFul API soon to be published on GitHub Ryft Systems, Inc. Page 28

29 Optional Ryft-Developed Open Source Doc #1193, V1.1.0/ November 2016 These are Ryft-developed open source that the customer may choose to add to the system. Open source ODBC/JDBC connector for SQL access to Ryft functionality; soon to be published on GitHub. Open source Spark connector for seamless Ryft ONE integration into Spark clusters. Published on GitHub and also available in Spark-Packages: Open source Consul connector for integration with Ryft ONE; soon to be published on GitHub. Ryft Systems, Inc. Page 29