JSON in PostgreSQL. Toronto Postgres Users Group Sept Steve Singer

Transcription

1 JSON in PostgreSQL Toronto Postgres Users Group Sept 2014 Steve Singer

2 What is JSON { "JSON" : { "Name" : "Javascript Object as Text format", "Allows" : { "Field1": "Nested objects "}, "{" : "good", "<" : "bad", "an_array" : [ 1, 2, 3 ], "love_postgresql" : true } }

3 Why JSON JavaScript Object Notation - Standard (RFC 7159) Popular Flexible Less Verbose than XML

4 Popular JSON based Databases MongoDB CouchDB ps://

5 PostgreSQL NoSQL Features with SQL reliability

6 When To JSON

7 When To JSON Entity Value Attributes (EVA) select * FROM config_options ; option value debug_level 5 database_host 1235.herokuapp.com database_name mydb site_name cool.global (4 rows)

8 When To JSON History Auditing create table customer ( id serial8, text, first_name text, last_name text ); create table customer_history ( history_id serial8, id int8, first_name text, last_name text

9 The JSON/JSONB Datatype create table customer_order ( order_id serial8 primary key, request json, response jsonb );

10 JSONB New in PostgreSQL 9.4 (Fall 2014 release?) JSON Available in PostgreSQL 9.3 (some features)

11 JSON or JSONB JSON JSONB Parsed at query time Maintains the original text representation Stored as a decomposed binary format. Not reparsed on each query Original text representation is lost Faster to insert/update Indexable

12 Adding Data insert into customer_order (request) values ('{ "order_time" : "2014/09/21 1:51:00", "order_amount" : 7.41, "order_qty" : 8 }'::json );

13 Selecting Data select request from customer_order; request { "order_time" : "2014/09/21 1:51:00",+ "order_amount" : 7.41, + "order_qty" : 8 + } (1 row)

14 Selecting individual fields (->) select request->'order_amount' from customer_order;?column? (1 row)

15 Adding A Response update customer_order set response= ' { "transaction_id" : 12345, "line_items" : [ { "item_num" : 1, "filled" : true }, { "item_num" : 2, "filled" : true } ] } ';

16 Querying Documents (contains) select request from customer_order where '{"transaction_id": }'; request { "order_time" : "2014/09/21 1:51:00",+ "order_amount" : 7.41, + "order_qty" : 8 + } (1 row)

17 That wasn't very fast Seq Scan on customer_order (cost= rows=100 width=32) (actual time= rows=1 loops=1) Filter: '{"transaction_id": 12346}'::jsonb) Rows Removed by Filter: Planning time: ms Execution time: ms (5 rows)

18 Creating Indexes CREATE INDEX idxgin ON customer_order USING gin and?

19 Bitmap Heap Scan on customer_order (cost= rows=100 width=32) (actual time= rows=1 loops=1) Recheck Cond: '{"transaction_id": 12346}'::jsonb) Heap Blocks: exact=1 -> Bitmap Index Scan on idxgin (cost= rows=100 width=0) (actual time= rows=1 loops=1) Index Cond: '{"transaction_id": 12346}'::jsonb) Planning time: ms Execution time: ms

20 Traversing The Tree (#>) Get object at specified path { "transaction_id" : 10, "line_items" : [ { "item_num" : 1, "filled" : true }], "payment" : { "method" : "visa", "auth_num" : "94142" } }

21 Traversing The Tree (#>) Get object at specified path select response #> '{"payment","auth_num"}' from customer_order where response #>> '{"payment","method"}' = 'visa';?column? "94142" (1 row)

22 Generating JSON select * FROM customer; id first_name last_name steve@ssinger.info Steve Singer 2 john@john.black John Black (2 rows)

23 Generating JSON select row_to_json(customer) FROM customer; row_to_json {"id":1, "first_name":"steve", "last_name":"singer" } {"id":2, " ":"john@john.black", "first_name":"john", "last_name":"black" } (2 rows)

24 Generating JSON select json_agg(customer) FROM customer; json_agg ger"}, + {"id":2," ":"john@john.black","first_name":"john","last_name":"black"}] (1 row)

25 Performance Comparisons Bulk Loading Postgres 2X faster than MongoDB Disk Space MongoDB used 35% more disk space Insert time PostgreSQL 3X faster on individual inserts Query Time PostgreSQL 3X faster Source:

26 Thanks to Andrew Dunstan Alexander Korotkov Oleg Bartunov Robert Hass Teador Sigaev \

27 Questions? Steve Singer Photos CC-BY-SA/NC see credits