3 / 120. MySQL 8.0. Frédéric Descamps - MySQL Community Manager - Oracle

Transcription

1 1 / 120

2 2 / 120

3 3 / 120 MySQL 8.0 a Document Store with all the benefits of a transactional RDBMS Frédéric Descamps - MySQL Community Manager - Oracle

4 4 / 120 Save the date!

5 5 / 120 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purpose only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied up in making purchasing decisions. The development, release and timing of any features or functionality described for Oracle s product remains at the sole discretion of Oracle.

6 6 / 120 about me - Who am I?

7 7 / 120 Frédéric MySQL Evangelist Hacking MySQL since 3.23 devops believer living in Belgium

8 8 / 120 RDBMS & MySQL Relational Databases

9 9 / 120 Relational Databases Data Integrity

10 10 / 120 Relational Databases Data Integrity normalization

11 11 / 120 Relational Databases Data Integrity normalization constraints (foreign keys,...)

12 12 / 120 Relational Databases Data Integrity normalization constraints (foreign keys,...) Atomicity, Consistency, Isolation, Durability

13 13 / 120 Relational Databases Data Integrity normalization constraints (foreign keys,...) Atomicity, Consistency, Isolation, Durability ACID compliant

14 14 / 120 Relational Databases Data Integrity normalization constraints (foreign keys,...) Atomicity, Consistency, Isolation, Durability ACID compliant transactions

15 15 / 120 Relational Databases Data Integrity normalization constraints (foreign keys,...) Atomicity, Consistency, Isolation, Durability ACID compliant transactions SQL

16 16 / 120 Relational Databases Data Integrity normalization constraints (foreign keys,...) Atomicity, Consistency, Isolation, Durability ACID compliant transactions SQL powerfull query language

17 17 / 120 NoSQL & MySQL NoSQL Databases

18 18 / 120 NoSQL or Document Store Schemaless

19 19 / 120 NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types,...

20 20 / 120 NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types,... very quick initial development

21 21 / 120 NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types,... very quick initial development Flexible data structure

22 22 / 120 NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types,... very quick initial development Flexible data structure embedded arrays or objects

23 23 / 120 NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types,... very quick initial development Flexible data structure embedded arrays or objects valid solution when natural data can t be modelized optimaly into a relational model

24 24 / 120 NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types,... very quick initial development Flexible data structure embedded arrays or objects valid solution when natural data can t be modelized optimaly into a relational model objects persistance without the use of any ORM - mapping oobject-oriented

25 25 / 120 NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types,... very quick initial development Flexible data structure embedded arrays or objects valid solution when natural data can t be modelized optimaly into a relational model objects persistance without the use of any ORM - mapping oobject-oriented JSON

26 26 / 120 NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types,... very quick initial development Flexible data structure embedded arrays or objects valid solution when natural data can t be modelized optimaly into a relational model objects persistance without the use of any ORM - mapping oobject-oriented JSON close to frontend

27 27 / 120 NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types,... very quick initial development Flexible data structure embedded arrays or objects valid solution when natural data can t be modelized optimaly into a relational model objects persistance without the use of any ORM - mapping oobject-oriented JSON close to frontend native in JS

28 28 / 120 NoSQL or Document Store Schemaless no schema design, no normalization, no foreign keys, no data types,... very quick initial development Flexible data structure embedded arrays or objects valid solution when natural data can t be modelized optimaly into a relational model objects persistance without the use of any ORM - mapping oobject-oriented JSON close to frontend native in JS easy to learn

29 29 / 120 How DBAs see data

30 30 / 120 How DBAs see data How Developers see data { } "GNP" : , "Name" : "Belgium", "government" : { "GovernmentForm" : "Constitutional Monarchy, Federation", "HeadOfState" : "Philippe I" }, "_id" : "BEL", "IndepYear" : 1830, "demographics" : { "Population" : , "LifeExpectancy" : }, "geography" : { "Region" : "Western Europe", "SurfaceArea" : 30518, "Continent" : "Europe" }

31 31 / 120 What if there was a way to provide both SQL and NoSQL on one stable platform that has proven stability on well know technology with a large Community and a diverse ecosystem?

32 32 / 120 DBMS or NoSQL?

33 33 / 120 DBMS or NoSQL? Why not both?

34 34 / 120 The MySQL Document Store! SQL is now optional!

35 35 / 120 A solution for all Developers: Operations: Business Owner: [x] schemaless [x] rapid prototying/simpler APIs [x] document model [x] transactions [x] performance management/visibility [x] robust replication, backup, restore [x] comprehensive tooling ecosystem [x] simpler application schema upgrades [x] don't lose my data == ACID trx [x] capture all my data = extensible/schemaless [x] product on schedule/time to market = rapid developement

36 36 / 120 MySQL Document Store the Solution

37 37 / 120

38 Built on the MySQL JSON Data type and Proven MySQL Server Technology Provides a schema flexible JSON Document Store No SQL required No need to define all possible attributes, tables, etc. Uses new X DevAPI Can leverage generated column to extract JSON values into materialized columns that can be indexed for fast SQL searches. Document can be ~1GB It's a column in a row of a table It cannot exceed max_allowed_packet Allows use of modern programming styles No more embedded strings of SQL in your code Easy to read Also works with relational Tables Proven MySQL Technology 38 / 120

39 X DevAPI Connectors for C++, Java,.Net, Node.js, Python, PHP working with Communities to help them supporting it too New MySQL Shell Command Completion Python, JavaScrips & SQL modes Admin functions New Util object A new high-level session concept that can scale from single MySQL Server to a multiple server environment Non-blocking, asynchronous calls follow common language patterns Supports CRUD operations 39 / 120

40 40 / 120 Starting using MySQL DS in few minutes

41 41 / 120 MySQL Shell Info

42 42 / 120 Migration from MongoDB to MySQL DS For this example, I will use the well known restaurants collection:

43 43 / 120 Migration from MongoDB to MySQL DS For this example, I will use the well known restaurants collection: We need to dump the data to a file and we will use MySQL Shell with the Python interpreter to load the data.

44 44 / 120 Migration from MongoDB to MySQL DS For this example, I will use the well known restaurants collection: We need to dump the data to a file and we will use MySQL Shell with the Python interpreter to load the data.

45 45 / 120 Dump and load using MySQL Shell & Python this example is inspired work: $ mongo --quiet --eval 'DBQuery.shellBatchSize=30000; db.restaurants. nd().shellprint()' \ perl -pe 's/(?:objectid ISODate)\(("[^"]+")\)/ $1/g' > all_recs.json

46 46 / 120 Dump and load using MySQL Shell & Python this example is inspired work: $ mongo --quiet --eval 'DBQuery.shellBatchSize=30000; db.restaurants. nd().shellprint()' \ perl -pe 's/(?:objectid ISODate)\(("[^"]+")\)/ $1/g' > all_recs.json

47 47 / 120

48 48 / 120 Let s query

49 49 / 120 Let s query That s too much records to show it here... let s limit it

50 50 / 120

51 51 / 120 Some more examples

52 52 / 120 Some more examples Let s add a selection criteria:

53 53 / 120 Using IN...

54 54 / 120 Syntax slightly different than MongoDB

55 > db.restaurants. nd({"cuisine": "French", "borough": { $not: /^Manhattan/} }, {"_id":0, "name": 1,"cuisine": 1, "borough": 1}).limit(2) { "borough" : "Queens", "cuisine" : "French", 55 / 120 Syntax slightly different than MongoDB

56 56 / 120 CRUD operations

57 57 / 120 CRUD operations for collections Add a document

58 58 / 120 CRUD operations for collections Modify a document

59 59 / 120 CRUD operations for collections Remove a document

60 60 / 120 MySQL Document Store Objects Summary

61 61 / 120 Document Store Full ACID! It relies on the proven MySQL InnoDB s strength & robustness:

62 62 / 120 Document Store Full ACID! It relies on the proven MySQL InnoDB s strength & robustness: innodb_ ush_log_at_trx_commit = 1

63 63 / 120 Document Store Full ACID! It relies on the proven MySQL InnoDB s strength & robustness: innodb_ ush_log_at_trx_commit = 1 innodb_doublewrite = ON

64 64 / 120 Document Store Full ACID! It relies on the proven MySQL InnoDB s strength & robustness: innodb_ ush_log_at_trx_commit = 1 innodb_doublewrite = ON sync_binlog = 1

65 65 / 120 Document Store Full ACID! It relies on the proven MySQL InnoDB s strength & robustness: innodb_ ush_log_at_trx_commit = 1 innodb_doublewrite = ON sync_binlog = 1 transaction_isolation = REPEATABLE-READ READ-COMMITTED...

66 66 / 120 Document Store Full ACID! It relies on the proven MySQL InnoDB s strength & robustness: innodb_ ush_log_at_trx_commit = 1 innodb_doublewrite = ON sync_binlog = 1 transaction_isolation = REPEATABLE-READ READ-COMMITTED... We do care about your data!

67 67 / 120 Document Store Full ACID! MySQL Document Store supports transactions

68 68 / 120 Document Store Full ACID! MySQL Document Store supports transactions

69 69 / 120

70 70 / 120 what about my old SQL? The hidden part of the iceberg

71 71 / 120 JSON datatype is behind the scene native datatype (since 5.7.8)

72 72 / 120 JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4

73 73 / 120 JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4 Conversion from "native" SQL types to JSON values

74 74 / 120 JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4 Conversion from "native" SQL types to JSON values JSON manipulation functions (JSON_EXTRACT, JSON_KEYS, JSON_SEARCH, JSON_TABLES,...)

75 75 / 120 JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4 Conversion from "native" SQL types to JSON values JSON manipulation functions (JSON_EXTRACT, JSON_KEYS, JSON_SEARCH, JSON_TABLES,...) Generated/virtual columns

76 76 / 120 JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4 Conversion from "native" SQL types to JSON values JSON manipulation functions (JSON_EXTRACT, JSON_KEYS, JSON_SEARCH, JSON_TABLES,...) Generated/virtual columns Indexing JSON data

77 77 / 120 JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4 Conversion from "native" SQL types to JSON values JSON manipulation functions (JSON_EXTRACT, JSON_KEYS, JSON_SEARCH, JSON_TABLES,...) Generated/virtual columns Indexing JSON data Foreign Keys to JSON data

78 78 / 120 JSON datatype is behind the scene native datatype (since 5.7.8) JSON values are stored in MySQL tables using UTF8MB4 Conversion from "native" SQL types to JSON values JSON manipulation functions (JSON_EXTRACT, JSON_KEYS, JSON_SEARCH, JSON_TABLES,...) Generated/virtual columns Indexing JSON data Foreign Keys to JSON data SQL Views to JSON data

79 79 / 120 How does the MySQL Document Store work?

80 80 / 120 What does a collection look like on the server?

81 81 / 120 _id Every document has a unique identifier called the document ID, which can be thought of as the equivalent of a table s primary key. The document ID value can be manually assigned when adding a document. If no value is assigned, a document ID is generated and assigned to the document automatically! Use getdocumentid() or getdocumentids() to get _ids(s)

82 82 / 120 Mapping to SQL examples createcollection('mycollection')

83 83 / 120 Mapping to SQL examples createcollection('mycollection') CREATE TABLE `test`.`mycoll` ( doc JSON, _id VARCHAR(32) GENERATED ALWAYS AS (doc->>'$._id') STORED PRIMARY KEY ) CHARSET utf8mb4;

84 84 / 120 Mapping to SQL examples createcollection('mycollection') CREATE TABLE `test`.`mycoll` ( doc JSON, _id VARCHAR(32) GENERATED ALWAYS AS (doc->>'$._id') STORED PRIMARY KEY ) CHARSET utf8mb4; mycollection.add({'test': 1234})

85 85 / 120 Mapping to SQL examples createcollection('mycollection') CREATE TABLE `test`.`mycoll` ( doc JSON, _id VARCHAR(32) GENERATED ALWAYS AS (doc->>'$._id') STORED PRIMARY KEY ) CHARSET utf8mb4; mycollection.add({'test': 1234}) INSERT INTO `test`.`mycoll` (doc) VALUES ( JSON_OBJECT('_id','663807fe367ee6114e0e5458bdac28bf', 'test',1234) );

86 86 / 120 Mapping to SQL examples (2) mycollection.find("test > 100")

87 87 / 120 Mapping to SQL examples (2) mycollection.find("test > 100") SELECT doc FROM `test`.`mycoll` WHERE (JSON_EXTRACT(doc,'$.test') > 100);

88 88 / 120 SQL and JSON Example

89 89 / 120 SQL and JSON Example same as:

90 90 / 120 SQL and JSON Example same as: we can use it in SQL:

91 91 / 120 Indexes in Document Store It's also possible to create indexes without using SQL syntax:

92 92 / 120 Indexes in Document Store It's also possible to create indexes without using SQL syntax:

93 93 / 120 SQL and JSON Example (2): validation

94 94 / 120 SQL and JSON Example (2): validation

95 95 / 120 SQL and JSON Example (2): validation

96 96 / 120 SQL and JSON Example (3): explain

97 97 / 120 SQL and JSON Example (3): explain

98 98 / 120 SQL and JSON Example (4): add index

99 99 / 120 SQL and JSON Example (4): add index

100 100 / 120 SQL and JSON Example (5): arrays

101 101 / 120 SQL and JSON Example (5): arrays

102 102 / 120 SQL and JSON Example (5): arrays

103 103 / 120 SQL and JSON Example (5): arrays

104 104 / 120 NoSQL as SQL JSON_TABLE

105 105 / 120 NoSQL as SQL (2) JSON_TABLE

106 106 / 120 NoSQL as SQL - aggregation

107 107 / 120 NoSQL as SQL - aggregation (2)

108 108 / 120 SQL and NoSQL - JOINS

109 109 / 120 SQL and NoSQL - JOINS

110 110 / 120 SQL and NoSQL - JOINS

111 111 / 120 SQL and NoSQL - JOINS (2)

112 112 / 120 SQL and NoSQL - JOINS (2)

113 113 / 120 SQL and NoSQL - JOINS (2)

114 114 / 120 SQL and NoSQL - and back to JSON Document

115 115 / 120 NoSQL or SQL JS> session.sql("select doc->>'$.name', doc->>'$.cuisine' from restaurants where doc->>'$.cuisine'='italian' limit 2") doc->>'$.name' doc->>'$.cuisine' Philadelhia Grille Express Italian Isle Of Capri Resturant Italian rows in set ( sec) JS> db.restaurants. nd("cuisine='italian'"). elds('name','cuisine').limit(2) [ { "cuisine": "Italian", "name": "Philadelhia Grille Express" }, { "cuisine": "Italian", "name": "Isle Of Capri Resturant" } ] 2 documents in set ( sec)

116 116 / 120 what do I gain? Conclusion

117 117 / 120 Conclusion This is the best of the two worlds in one product! Data integrity ACID Compliant Transactions SQL schemaless flexible data structure easy to start (CRUD)

118 118 / 120

119 119 / 120

120 120 / 120 Thank you! Any Questions? share your for MySQL on social media #DataOpsBarcelona