IBM DB2 JSON An overview of DB capabilities as a JSON document store

Transcription

1 IBM DB2 JSON An overview of DB capabilities as a JSON document store Olivier Bernin IBM Session Code: 4 April 16 th, 14:45 Platform: DB2 LUW

2 Agenda Introduction what's JSON, requirements, setting up,... Using IBM DB2 JSON Command line tool Java API NoSQL Wire Listener Under the bonnet

3 What is JSON? JavaScript Object Notation open standard format that uses human-readable text to transmit data objects consisting of attribute value pairs Alternative to XML Simple Human-readable Flexible Compact JSON is JavaScript Example

4 How is JSON used? Rich Internet Applications JSON (no schema) Mobile Applications Application Server Database Desktop Applications

5 What is DB2 JSON? A DB2 built-in solution that brings together the flexibility of the JSON data representation with the enterprise features and quality of service associated with RDBMs. In practice: Natively embedded in DB2 (out of the box) Uses the MongoDB programming model and query language Provides support for additional data types (date, etc...) 3 ways to access data Interactive command line shell Java driver access JSON data through familiar JDBC API MongoDB protocol driver allow access through a number of programming languages JavaScript, PHP, Python, Ruby, Perl, C, C++, etc...

6 DB2 JSON Architecture SQL Applications NoSQL CLP (db2nosql) MongoDB Wire Protocol NoSQL Wire Listener DB2 JSON Java API JDBC DB2 DB2 Engine JSON_VAL JSON UDFs

7 Setting up DB2 for JSON Software Requirements IBM DB2 Version 10.5 Fix Pack 1 for Linux, UNIX, and Windows Requires JDBC db2jcc.jar for the IBM Data Server Driver Package Requires the DB2 NoSQL JSON package: DB2 NoSQL JSON library nosqljson.jar Rhino JavaScript engine js.jar, which is required if you are using the command-line processor Java Runtime Environment 1.5 or greater

8 Setting up DB2 for JSON Configuration Steps Add java to the PATH Add db2jcc.jar to the CLASSPATH or the -cp in the script Other JAR files will be needed depending on technique used Ensure that the user has the following administrative privileges: CREATETAB and CREATE FUNCTION in the SYSTOOLS schema. DBADM authority for the target database Create or use an existing UTF-8 database with a 32K page size Ensure that the DFT_TABLE_ORG database configuration parameter is set to ROW

9 Using the command line tool - db2nosql Add the nosqljson.jar & js.jar to the CLASSPATH Start the client db2nosql -db <dbname> -user <user> -password <password> -hostname <host> -port <port> nosql> prompt Enable the database nosql>enable(true) Create required system objects. Run only once pe DB That's it!

10 A word about the programming model DB2 JSON deals with Collections of Documents A document is a JSON object Document always have an ID ( _id ) - maybe implicit A collection is a heap of document Collections do not enforce any schemas on documents!!! Documents can be added, removed, searched, etc... in collections Collections have names, optionally a namespace Collections are stored in the DB2 database Support for additional data types on top of native JSON ones $string, $number $int, $date, $timestamp, $binary, $oid Predicates for search, filtering, sorting, etc... are JSON objects Use special operators Example: { price : { $lt : 9.99 } }

11 Demo Creating a collection & inserting a document db.createcollection( movies ) db.movies.insert({ title: "2001: A Space Odyssey", director: "Stanley Kubrick", year: 1968, duration: 160, category: "Science Fiction", cast: [ { name: "Keir Dullea", character: "Dr. Dave Bowman" }, { name: "Gary Lockwood", character: "Dr. Frank Poole" }, { name: "William Sylvester", character: "Dr. Heywood R. Floyd" }], rating: 8.3})

12 Demo Creating a collection & inserting a document #IDUG

13 Demo Importing documents db.movies.importfile( movies.js )

14 Demo Search & query documents Find all movies directed by Stanley Kubrick db.movies.find({ director: "Stanley Kubrick" })

15 Demo Search & query documents Find all movies released in 1968, show the titles & directors db.movies.find({ year: 1968 }, { title: 1, director: 1 })

16 Demo Search & query documents Find all movies released before 2008, show the titles & directors db.movies.find({ year: { $lt: 2008 }}, { title: 1, director: 1 }))

17 Demo - Aggregation Find the average rating for all movies from a given director db.movies.group({ _id: "$director", avgrating: { $avg: "$rating"}} )

18 Using the command line shell a few more options Showing the current database connection: db -> Database: jdbc:db2:json Schema: TEST Setting the JSON collection namespace: use e.g. use mynamespace Default one is TEST Displaying the current namespace: show dbs

19 Connecting using the DB2 JSON Java API (JDBC) Enable the database using the db2nosql script Use the -setup enable option Use the com.ibm.nosql.json.api.nosqlclient class to obtain a DB object that represents a connection to the JSON store Establishes a JDBC connection to the DB2 database Pass an existing java.sql.connection or connection parameters (url, user, pwd, etc...) DB db = NoSQLClient.getDB(jdbcUrl, user, pwd)

20 Objects provided by the DB2 JSON Java API Same programming model as with the command line shell JSON Collection <=> com.ibm.nosql.json.api.dbcollection JSON Document <=> com.ibm.nosql.json.api.basicdbobject JSON Arrays <=> com.ibm.nosql.json.api.basicdblist Queries return cursor as com.ibm.nosql.json.api.basicdbobject Can be iterated over using Java foreach construct

21 An example Attached file - Sample.java

22 More advanced features Index Transactions Batch processing Send multiple documents to be processeds at once Performance Lazy fetch Fetch documents in successive groups instead of all at once Fire-and-forget mode Multi-threaded high-performance mode of update

23 Using the IBM NoSQL Wire Listener for DB2 The MongoDB Wire Protocol is a simple socket-based, request-response style protocol allowing the transmission over TCP/IP of JSON documents between clients and compatible JSON store Provides programming language independant access Requires a MongoDB driver for the onsidered language JavaScript, Python, C, C++, Perl, PHP, Ruby, etc... DB2 provides an IBM NoSQL Wire Listener allowing compatible drivers to connect to DB2 JSON Applications MongoDB API IBM NoSQL Wire Listener JSON Java API DB2

24 Getting started with the NoSQL Wire Listener Enable the database using the db2nosql script Use the -setup enable option Start the listener wplistener -start -mongoport <port> -dbname <db> -userid <user> -password <password> To stop it wplistener -shutdown -nosqlhost -mongoport <port> -dbname <db> -userid <user>

25 Under the bonnet The command line tool debug option: debug(true) Will display the SQL executing under the cover Creating a collection: db.createcollection( movies ) CREATE TABLE TEST."movies" ( ID CHAR(12) FOR BIT DATA NOT NULL, DATA BLOB(16M) INLINE LENGTH 25000, PRIMARY KEY(ID)) ORGANIZE BY ROW A table named after the collection is created 2 columns: ID & DATA, type is BLOB

26 Under the bonnet What does the data look like? SELECT * FROM TEST. movies ID DATA x'534645a535375ffaf80e7b65' x' c6... x'534645cc35375ffaf80e7b66' x' c6... x'534645cc35375ffaf80e7b67' x' c6... x'534645cc35375ffaf80e7b68' x' c6... Making sense of it... SELECT SYSTOOLS.BSON2JSON(DATA) AS JSON_DATA FROM TEST."movies" JSON_DATA {"title":"2001: A Space Odyssey","director":"Stanle... {"title":"e.t. The Extra-Terrestrial","director":"S... {"title":"raiders of the Lost Ark","director":"Stev... {"title":"lincoln","director":"steven Spielberg","y... SYSTOOLS.BSON2JSON() UDF created when database is enabled See what happens by running enable(false)

27 BSON? Binary JSON BSON [bee sahn], short for Binary JSON, is a binary encoded serialization of JSON-like documents. Following properties Lightweight Traversable Efficient - bsonspec.org More at bsonspec.org

28 Under the bonnet - Searching db.movies.find({ director: "Steven Spielberg" }) SELECT ID, DATA FROM TEST."movies" WHERE JSON_VAL(DATA, 'director', 's:2048:na')=? New SYSIBM.JSON_VAL function SYSIBM.JSON_VAL(<BSON_BLOB>, <JSON_PATH>, <TYPE_SPEC>) SELECT JSON_VAL(DATA, 'title', 's:32') AS TITLE, JSON_VAL(DATA, 'year', 'i') AS YEAR FROM TEST.\"movies\" TITLE YEAR : A Space Odyssey 1968 E.T. The Extra-Terrestrial 1982 Raiders of the Lost Ark 1981 Lincoln 2012

29 Under the bonnet What we have seen so far... JSON Documents are stored inside tables as BSON BLOBs Table represents the collection Schema name is collection namespace Set of new SQL functions provided to manipulate the documents JSON_VAL, BSON2JSON, JSON2BSON, etc... JSON_VAL used to parse BSON BLOB and extract a typed value What about performances??

30 Under the bonnet - Indexing Creating an index on a JSON document field db.movies.ensureindex({ director: 1 }) CREATE INDEX TEST."movies_xdirector" ON TEST."movies" (JSON_VAL(DATA,'director','s:1024:na')) Expression based-index Indexing the results of a functional expression on one or more columns New feature introduced with DB Expression-based indexes in IBM DB InfoCenter

31 IBM DB2 JSON on DB2 for z/os Supported from DB FP3 More required components Database enablement is installation task performed by Database or System administrator. No supported from script see APAR PM98357

32 Thank you! Questions?

33 Olivier Bernin IBM Session 4 IBM DB2 JSON An overview of DB capabilities as a JSON document store Please fill out your session evaluation before leaving!