Ryft REST API - Swagger.io

Size: px

Start display at page:

Download "Ryft REST API - Swagger.io"

Shawn Richards
5 years ago
Views:

1 Ryft REST API - Swagger.io User Guide Ryft Document Number: 1192 Document Version: Revision Date: June Ryft Systems, Inc. All Rights in this documentation are reserved. RYFT SYSTEMS, INC.

2 Copyright (c) 2017, Ryft Systems, Inc. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. All advertising materials mentioning features or use of this software must display the following acknowledgement: This product includes software developed by Ryft Systems, Inc. Neither the name of Ryft Systems, Inc. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY RYFT SYSTEMS, INC. ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RYFT SYSTEMS, INC. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Ryft Systems, Inc. Page ii

3 Revision History Date Reason for Change Version June 2017 Add rename and login March 2017 Add CURL & HTTP command examples October 2016 Initial Release Ryft Systems, Inc. Page iii

4 Contents Using This Guide... 5 Ryft Technical Support... 5 About Ryft Systems Inc : Ryft ONE RESTful API Web Interface... 7 GET /search Endpoint GET /count Endpoint PUT /rename Endpoint Files Endpoints DELETE /files Endpoint GET /files Endpoint POST /files Endpoint GET /version Endpoint Authentication POST /login Endpoint GET /token_refresh Endpoint : Generating a Python Client Requirement Ryft Python Client Example Step 1: Copy Ryft-server-swagger.json Interface Definition Page Step 2: Generate Client API Library for Desired Language Step 3: Replace Code Generator Display with swagger.json from Clipboard Step 4: Select Client Language and Save File Step 5: Setup Generated API Client Code Environment Run Code Examples Example: ryft_search_example.py Ryft Systems, Inc. Page iv

5 Using This Guide This guide is for the user who will use the Swagger interface to use the Ryft ONE REST API. Additional resources: Ryft REST API Command User & Reference Guide, Ryft Open API Library User Guide and the Ryft ONE: User Guide. Ryft Technical Support You may access our first-tier support directly from our public website by using the Chat Widget and starting a chat with our support agents. All chat conversations are tracked and become help desk tickets in our support system. In addition, you can access our support system through the Ryft Support site: Log in with your credentials to see your past tickets, create new tickets, and to access limited-access content. For technical support, contact us: support@ryft.com Web: Phone: (RYFT ONE) About Ryft Systems Inc. Ryft makes data analytics fast and simple with the world s top cloud and physical heterogeneous compute accelerators. With more than a decade of experience, Ryft helps deliver instant insights into any data by eliminating the latency associated with data indexing and transformation/curating. Today, industry leaders rely on Ryft to quickly and simply unlock the value hidden in their data for real-time actionable insight. Ryft Systems, Inc. Page 5

6 This page intentionally left blank. Ryft Systems, Inc. Page 6

1 1: Ryft ONE RESTful API Web Interface The Ryft ONE RESTful API incorporates Swagger to provide a simple, yet powerful, representation of the interface to the Ryft ONE.

7 1 1: Ryft ONE RESTful API Web Interface The Ryft ONE RESTful API incorporates Swagger to provide a simple, yet powerful, representation of the interface to the Ryft ONE. With the Swagger-enabled API, developers can easily try out the interface and use the Swagger SDK to easily generate clients from an extensive selection of languages. The RyftOne REST API page is available for both single server and clustered environments. The function is the same, regardless the environment. To access the interface, use the IP address of your Ryft ONE followed by port number :8765. Ryft Systems, Inc. Page 7

The page contains expandable sections for each of the endpoints: search, count, rename, files, version and the two special endpoints for JWT (JSON Web Token) authentication, login and token/refresh.

8 The page contains expandable sections for each of the endpoints: search, count, rename, files, version and the two special endpoints for JWT (JSON Web Token) authentication, login and token/refresh. The files endpoint has three operations: delete, get and post. Expand each to get more information, and then complete the appropriate parameter fields to execute the command. Expand a section and notice that the top of each section contains notes and the code to be executed. The Parameters section provides a list of all available parameters used by the endpoint, and the ability to input or select values for each parameter. Ryft Systems, Inc. Page 8

9 The bottom of each section has the Try it out! button. Press it to run the specified command with the parameters you have specified. In addition to returning the results in the Response Body section, it also displays the Curl code based on the parameters, and the Request URL used to connect to the specified server and the code to execute. Ryft Systems, Inc. Page 9

10 GET /search Endpoint Run a raw text or structured search against specified file(s). Return all data that match the specified parameter values. Scroll down the page to view the Parameters: Parameter Data Type Description query * String Search query string. At least one search query is REQUIRED. For example: (RAW_TEXT CONTAINS night ) file * Array[string] Set of files in the specified path to search. Designated file path must be relative to the /ryftone directory. At least one file is REQUIRED. Wild cards are supported such as wiki*.* or *.txt. List multiple values on individual lines. mode String Search mode. Available options are: es Exact search fhs Fuzzy hamming search feds Fuzzy edit distance search Ryft Systems, Inc. Page 10

11 Parameter Data Type Description ds Date search ts Time search Doc #1192, V1.1.0/ June 2017 ns Currency or numeric search rs Regular expression (regex) search ipv4 IPv4 search ipv6 IPv6 search surrounding Integer or line Number of characters before and after the matching query string to be returned. Use with RAW text search. Set to line to return the entire line. fuzziness Integer Fuzziness of the search. Measured as the maximum Hamming distance for fhs mode or the edit distance for feds mode. format String By default, structured search uses format=raw format. That means that found data are reported as base-64 encoded raw bytes. The options are: raw use for structured search xml json utf8 use for text search null If input file set contains XML data, the found records could be decoded by passing the format=xml query parameter. Records will then be translated from XML to JSON. The same is true for JSON data. For the text search use the format=utf8 option. It interprets raw bytes as UTF-8 string so it's easy to take a quick look at the results. cs Boolean Case-sensitive flag for query string. True or False. Default is false. reduce Boolean Reduce flag for fuzzy edit distance search. True or False. Default is true. fields String Comma-separated list of requested fields. If omitted, all fields are used. Use with structured search to minimize output. data String Name of data file to be kept. If the file already exists, it will be overwritten. Ryft Systems, Inc. Page 11

12 Parameter Data Type Description Doc #1192, V1.1.0/ June 2017 index String Name of index file to be kept. If the file already exists, it will be overwritten. delimiter String Characters to use between found records in the data file. Default is a carriage return. nodes Integer The number of RCAB processing nodes to use for this search. Maximum is 4. If omitted, all available nodes are used. local Boolean The server(s) on which to perform this search. True or False. Default is false; search is run locally. Select True to run on the entire cluster. stats Boolean Display search statistics with results. True or False. Default is false, no statistics are reported. limit Integer Limit the total number of returned records to the specified number. If none specified, all results are returned. Transform Array[string] Additional data transformation. Could be match("^.$"), replace("^(.)apple(.*)$", "$1orange$2") or script("filter_position") In this search example, the following parameters are set: query = Washington redskins files = wiki*.* mode = fhs surrounding =20 fuzziness = 2 format = utf8 cs = false Here is the result: Ryft Systems, Inc. Page 12

It found 13,651 matches. This is the Curl command that executes the search: curl -X GET --header 'Accept: application/json' 'http://172.16.14.3:8765/search?

13 It found 13,651 matches. This is the Curl command that executes the search: curl -X GET --header 'Accept: application/json' ' mode=fhs&surrounding=20&fuzziness=2&format=utf8&cs=false&reduce=true&local=true&stats= true' This is the request URL that executes the search. ode=fhs&surrounding=20&fuzziness=2&format=utf8&cs=false&reduce=true&local=true&stats=t rue Ryft Systems, Inc. Page 13

GET /count Endpoint Doc #1192, V1.1.0/ June 2017 Like search, run a search for specified query string, but in this case, only return the match statistics.

14 GET /count Endpoint Doc #1192, V1.1.0/ June 2017 Like search, run a search for specified query string, but in this case, only return the match statistics. Parameter Data Type Description query * String Search query string. REQUIRED. For example (RAW_TEXT CONTAINS night ) file * Array[string] Set of files in the specified path to search. Designated file path must be relative to the /ryftone directory. At least one file is REQUIRED. Wild cards are supported such as wiki*.* or *.txt. List multiple values on individual lines. mode String Search mode. Available options are: es exact search fhs fuzzy hamming search feds fuzzy edit distance search ds date search ts time search ns Currency or numeric search rs Regular expression (regex) search ipv4 IPv4 search Ryft Systems, Inc. Page 14

15 Parameter Data Type Description surrounding Integer or line ipv6 IPv6 search Doc #1192, V1.1.0/ June 2017 Number of characters before and after the matching query string to be returned. Use with RAW text search. Can set to line to return the entire line. fuzziness Integer Fuzziness of the search. Measured as the maximum Hamming distance for fhs mode or the edit distance for feds mode. cs Boolean Case-sensitive flag for query string. True or False. Default is false. reduce Boolean Reduce flag for fuzzy edit distance search. True or False. Default is true. data String Name of data file to be kept. If the file already exists, it will be overwritten. index String Name of index file to be kept. If the file already exists, it will be overwritten. delimiter String Characters to use between found records in the data file. Default is a carriage return. share-mode String Waiting time used for sharing mode when accessing data times. Default is wait-0s (zero seconds). Can be wait-10s, skip or ignore. nodes Integer The number of RCAB processing nodes to use for this search. Maximum is 4. If omitted, all nodes are used. local Boolean The server(s) on which to perform this search. True or False. Default is false; search is run locally. Select True to run on the entire cluster. performance Boolean Include performance metrics in the response results. True or False. Default is False. Ryft Systems, Inc. Page 15

In this example, the following parameters are used: query = Young Shik Rhee files = wiki*.* mode = fhs surrounding =10 fuzziness = 2 cs = false This is the result; it found 24 matches.

16 In this example, the following parameters are used: query = Young Shik Rhee files = wiki*.* mode = fhs surrounding =10 fuzziness = 2 cs = false This is the result; it found 24 matches. curl -X GET --header 'Accept: application/json' ' e=fhs&surrounding=10&fuzziness=2&cs=false&reduce=true&local=true' The only difference in the curl command is that the count endpoint is used instead of search so only the count or statistics display in the response. This is also true for the HTTP command. PUT /rename Endpoint Use the PUT /rename endpoint to rename or change the name of: an individual file, a directory, a catalog name during the upload to the Ryft ONE, or a file name within an existing catalog. Ryft Systems, Inc. Page 16

At least one of the parameters (file, directory, or catalog) must be specified. Parameter Data Type Description file String The full path of the source file to upload.

17 At least one of the parameters (file, directory, or catalog) must be specified. Parameter Data Type Description file String The full path of the source file to upload. The file extension cannot be changed using this endpoint. Wild cards are supported such as wiki*.* or *.txt. If used with the catalog parameter, the file will be renamed inside the catalog. dir String The full path of the source directory. catalog String The full path of the catalog file. If used with the file parameter, the catalog will not be renamed. new * String The new file name for the file, directory, catalog or a file within a catalog. This example demonstrates renaming a file in the /ryftone mount point from adldata1.txt to adldata2.txt. Ryft Systems, Inc. Page 17

18 Files Endpoints Three options are available for the Files endpoint. Click on each to expand the section. Each endpoint is covered in the following pages. Ryft Systems, Inc. Page 18

19 DELETE /files Endpoint Doc #1192, V1.1.0/ June 2017 Use the DELETE /files endpoint to delete directories and files on the node or cluster. For each parameter, you can list multiple values on new lines. Wildcards are supported. Parameter Data Type Description dir Array[string] The directory path to delete. Wildcards are supported. file Array[string] The file path to delete. Wildcards are supported. catalog Array[string] The catalog path to delete. Wildcards are supported. local Boolean True/False. Default is false. Return the file structure from the local node, only, or from the entire cluster. Provide values for one of the three parameters to define what should be deleted an entire directory, a specific file, or a catalog. You must provide the full path for each. Ryft Systems, Inc. Page 19

GET /files Endpoint Doc #1192, V1.1.0/ June 2017 Use the GET /files endpoint to obtain a list of the files and directories on the specified node or cluster.

20 GET /files Endpoint Doc #1192, V1.1.0/ June 2017 Use the GET /files endpoint to obtain a list of the files and directories on the specified node or cluster. Parameter Data Type Description dir String The folder path to return the list of files in the directory. Default is /ryftone the Ryftone directory. Only lists files from the /ryftone/<directory name> specified. hidden Boolean True/False. Default is false. Include or exclude hidden files. local Boolean True/False. Default is false. Return the file structure from the local node, only, or from the entire cluster. catalog String The pathname of the catalog to download the file. file String The filename. Ryft Systems, Inc. Page 20

21 In this example, list all files in the /ryftone/odbc directory on the local server (local=true). Ryft Systems, Inc. Page 21

POST /files Endpoint Use this endpoint to upload (post) files to the node or cluster. Parameter Data Type Description file String Set of files to post/upload. At least one file is REQUIRED.

22 POST /files Endpoint Use this endpoint to upload (post) files to the node or cluster. Parameter Data Type Description file String Set of files to post/upload. At least one file is REQUIRED. Use {{random}} for random names. Wild cards are supported, such as wiki*.* or *.txt. List multiple values on individual lines. catalog String The catalog to append the file(s) to. delimiter String Characters to use between file parts in the catalog. offset Long Optional. File part offset. length Long Optional. Ryft Systems, Inc. Page 22

Parameter Data Type Description File part length Doc #1192, V1.1.0/ June 2017 lifetime String The file or catalog lifetime. For example, 2h share-mode String Sharing mode used to access data files.

23 Parameter Data Type Description File part length Doc #1192, V1.1.0/ June 2017 lifetime String The file or catalog lifetime. For example, 2h share-mode String Sharing mode used to access data files. Default is wait-0s. Use values like wait-10s or ignore. local Boolean The server(s) on which to perform this search. True or False. Default is false; search is run locally. Select True to run on the entire cluster. GET /version Endpoint Run this to return the current build version on the local server. Returns the hash and version number. No input parameters. The following output is returned: Authentication If authentication is enabled, the ryft-server checks for Authorization HTTP header. If Authorization header contains Basic keyword the basic authentication is used. The ryft-server extracts username and password from the header and checks the user is authorized to access requested resources. Otherwise, if Authorization header contains Bearer keyword the JSON Web Token (JWT) is used. The ryft-server extracts JWT token from the header and uses it. There are two special endpoints for JWT authentication: Ryft Systems, Inc. Page 23

24 /login is used to get JWT token. /token/refresh is used to refresh existing token. POST /login Endpoint Example response below with token: Ryft Systems, Inc. Page 24

GET /token_refresh Endpoint In order to support JWT, ryft server exports two endpoints: Use /login to provide user credentials and get valid token back.

25 GET /token_refresh Endpoint In order to support JWT, ryft server exports two endpoints: Use /login to provide user credentials and get valid token back. Use /token/refresh to refresh existing token. By default, the token's lifetime is one hour but this timeout can be customized using the --jwt-lifetime command line option. There is a max of 10 lifetimes. Ryft Systems, Inc. Page 25

26 Ryft Systems, Inc. Page 26

27 2 2: Generating a Python Client The use of Swagger enables the user to easily generate a client to the RESTful JSON API, in an extensive selection of languages. This section shows how to generate a Python client. Requirement The Ryft ONE REST API works with Python version 2.6 or 2.7. Ryft Python Client Example This is an example of the Ryft Python client using swagger.json interface definition. The Ryft swagger.json interface definition is used to generate client code. This requires Python 2.7 and later. Here is a summary of the steps: 1. Go to the Ryft-server swagger.json interface definition page and copy contents to clipboard. 2. Generate client API library for desired language using swagger.json. 3. Replace code generated display with swagger.json from clipboard. 4. Select client language and save file 5. Setup generated API client code environment 6. Run sample programs Each step is covered in the following pages. Ryft Systems, Inc. Page 27

28 Step 1: Copy Ryft-server-swagger.json Interface Definition Page Doc #1192, V1.1.0/ June 2017 Go to the Ryft-server swagger.json interface definition page and copy contents of the entire page to clipboard. Use this syntax: Address_of_Host>:8765/swagger.json Step 2: Generate Client API Library for Desired Language Use Swagger s code generator web page to generate the code for your desired language. From the page, select Swagger Tools, Swagger Editor, and then Try the Live Demo link. Alternatively, go to Step 3: Replace Code Generator Display with swagger.json from Clipboard Replace the contents of the left window with the copied text from the swagger.json file. Ryft Systems, Inc. Page 28

29 Step 4: Select Client Language and Save File From the top-level swagger menu, select Generate Client and select Python (or your desired language). This example generates a Python API package and saves it to your local computer. Ryft Systems, Inc. Page 29

30 Step 5: Setup Generated API Client Code Environment You need these two commands to unzip the saved API package: If you don t have it, run sudo apt-get unzip. If you don t have it, run sudo apt-get install python-pip Now, issue these commands: $ unzip python-client-generated.zip $ cd ryft-python-client $ sudo python setup.py install Run Code Examples Doc #1192, V1.1.0/ June 2017 These examples are already installed on your Ryft ONE in the ~/install/ryft-python-client directory. Run Python examples in this project, or write your own. # run python examples from this repo, or write your own python ryft_version_example.py -a python ryft_search_example.py -a python ryft_files_example.py -a # run sample programs accepting more parameters python ryft_search.py -a -h --help Prints this short help message. -p --mode <primitive> Specifies the search mode to run, which can be one of: es exact_search fhs fuzzy_hamming_search (used by default) feds fuzzy_edit_distance_search ds date_search ts time_search ns numeric_search -f --file <filename> Specifes a /ryftone input file to search. -i Specifies case insensitive analysis for supported primitives. -n --nodes <#> Specifies 1-4 RCAP processing nodes to use. -d --fuzziness <#> Specifies the fuzzy search distance. -w --width <#> Specifies the surrounding text. -s -q <query> Specifies /ryftone files to search. -od filename Specifies /ryftone search output data file. -oi filename Specifies /ryftone search output index file. Rest Specific: -a --address Specifies ryft server eaddress. --local Specifies use local search. Opposite of -- cluster --cluster Specifies use cluster search. Opposite of -- local --format=<format> Specifies output formatting raw xml json utf8 --fields Specifies comma-separated list of fields to return --no-stats Disabled statistics output. --count Use /count endpoint instead of /search to print just statistics. -v Debug placeholder -vv Debug placeholder Ryft Systems, Inc. Page 30

31 Use this command to change default hostnames used by ryft*.py: perl -p -i -e 's/ryft311/ /' ryft*.py Example: ryft_search_example.py import swagger_client import sys, getopt # cmd line args host=" myopts, args = getopt.getopt(sys.argv[1:],"h:") for o, v in myopts: if o == '-h': host=v # api call a = swagger_client.apiclient(host=host) sa = swagger_client.apis.searchapi(a) sr = sa.search( files=["odbc/census/census*.censusxml"], query="(raw_text CONTAINS \"Aberdeen\")", local="true", stats="true", cs="true", fuzziness="0",format="utf8", surrounding="0", mode="es"); # show results print "stats=", sr.stats print "results=", sr.results Ryft Systems, Inc. Page 31

StorageGRID Webscale NAS Bridge Management API Guide

StorageGRID Webscale NAS Bridge 2.0.3 Management API Guide January 2018 215-12414_B0 doccomments@netapp.com Table of Contents 3 Contents Understanding the NAS Bridge management API... 4 RESTful web services