Fixing the Worst Database in the World

Transcription

1 Fixing the Worst Database in the World Heli Chris

2 Introduction, Heli Graduated from University of Helsinki (Master of Science, computer science), currently a doctoral student, researcher and lecturer (databases, Big Data, Multi-model Databases, methods and tools for utilizing semi-structured data for decision making) at University of Helsinki Worked with Oracle products since 1993, worked for IT since 1990 Data and Database! CEO for Miracle Finland Oy Oracle ACE Director Ambassador/EOUC (EMEA Oracle Users Group Community) Public speaker and an author Winner of Devvy for Database Design Category, 2015 Author of the book Oracle SQL Developer Data Modeler for Database Design Mastery (Oracle Press, 2015), co-author for Real World SQL and PL/SQL: Advice from the Experts (Oracle Press, 2016)

3 Helskyaho Heli, Valin-Raki Elise

4 Introduction, Heli Been an Oracle Designer user since 1996 and Data Modeler user since 2010 Been solving performance problems since 1993 Absolutely convinced that good database design and documentation will reduce this work. Absolutely convinced that it is easier to solve problems on database that has been documented

5 Why to design? "Data is the most valuable property in our company" "Why do we need to design the database? We already design the application!"

6 Why is designing the app not enough? Point of view (saving and retrieving data vs. UI) First increment vs. 20 years from now the whole picture vs. increments Different goals/targets: Code tables vs. Code files (how about the data integrity?) How about analysis, reports, everything else but the UI that the data is used for Same terminology, different meaning -> misunderstandings...

7 Why to model the data? To facilitate communication about the requirements To find the questions that should be asked To understand the requirements

8 Tables are the of your database

9 Get them wrong and

10 you need to be a SQL

11 with attrs as ( select pr.productno, attrs.* from products pr, json_table ( productinfo, '$' null on error columns ( description path '$.description',colour path '$.colour',weight path '$.weight',height path '$.dimensions.height',width path '$.dimensions.width',depth path '$.dimensions.depth' ) ) attrs union all

12 select productno, description, colour, weight, height, width, depth from ( select pr.productno, pa.attrname, pa.attrvalue from ( select productno from products ) pr left join ( select productno, attrname, attrvalue from product_attributes ) pa on pr.productno = pa.productno ) pivot ( min(attrvalue) for attrname in ( 1 height, 2 depth, 3 width, 4 weight, 5 colour, 6 description

13 ) ) ), product_details as ( select productno, min(description) descr, min(colour) clr, min(weight) wt, min(height) ht, min(width) wd, min(depth) dp from attrs group by productno ) select customers_ssn, descr, wt, clr,

14 case when ht like '%cm' then to_number(regexp_substr(ht, '^[0-9\.]*')) / 100 when ht like '%m' then to_number(regexp_substr(ht, '^[0-9\.]*')) else to_number(ht) / 100 end * case when wd like '%cm' then to_number(regexp_substr(wd, '^[0-9\.]*')) / 100 when wd like '%m' then to_number(regexp_substr(wd, '^[0-9\.]*') ) else to_number(dp) / 100 end *

15 case when dp like '%cm' then to_number(regexp_substr(dp, '^[0-9\.]*')) / 100 when dp like '%m' then to_number(regexp_substr(dp, '^[0-9\.]*') ) else to_number(dp) / 100 end volume, sum ( qty * howmuchtheypay ) from "Order" o join orderline ol on o.orderno = ol.order_orderno join product_details pd on ol.products_productno = pd.productno

16 where ( to_date ( o."date" default null on conversion error, 'DD-MON-YYYY' ) > add_months(sysdate, -1) or to_date ( o."date" default null on conversion error, 'MM/DD/YYYY' ) > add_months(sysdate, -1) or to_date ( o."date" default null on conversion error, 'YYYY/MM/DD' ) > add_months(sysdate, -1) ) group by customers_ssn, descr, ht, wd, dp, wt, clr

17 Help me Heli Helskyaho you're my only hope!

18 DEMO

19 Bad Primary s

20 Bad Primary s Anything non-unique (duh ) if you have to ask, it (probably) isn't! Things you (may) need to update addresses Stuff which is "unknown" in some cases

21 Always surrogate? Need more joins; slower SQL? Still need to add unique constraint over natural key Can enforce complex business rules with "natural keys"

22 School Teacher Scheduling Who is teaching what, when, and where? Teacher only in one place at a time Only one class in each room at a time

23 New Requirement! You can only schedule approved teachers for a class create table class_teachers ( class_teacher_id integer, teacher_id integer, class_id integer, primary key ( class_teacher_id ), unique ( teacher_id, class_id ) ); Surrogate key

24 Teachers must be approved for classes No teacher_id! How validate teacher only has one class at a time? \_( ツ )_/

25 No surrogate: can implement rules! Teacher only in one place at a time Only one class in each room at a time

26 Wrong Data Types

27 Wrong Data Types: Invalid Data create table bad_data_types ( date_yyyymmdd integer ); insert into bad_data_types values ( ); insert into bad_data_types values ( ); insert into bad_data_types values (-1);

28 alter table bad_data_types add constraint is_date check ( to_number ( to_date ( date_yyyymmdd, 'yyyymmdd' ), 'yyyymmdd' ) = date_yyyymmdd ); ORA-01481: invalid number format model

29 alter table bad_data_types add constraint is_date check ( to_number ( to_date ( date_yyyymmdd, 'yyyymmdd' ), 'yyyymmdd' ) = date_yyyymmdd ) novalidate; Stops new invalid dates Existing data may still be dodgy!

30 Wrong Data Types: More Complex SQL with dates as ( select to_date ( date_yyyymmdd, 'yyyymmdd' ) dt from bad_data_types where regexp_like ( date_yyyymmdd, '[0-9]{4}(1[0-9] 0[1-9])(3[0-1] [1-2][0-9] 0[1-9])' ) ) select * from dates where dt = :date_var Still not quite right

31 Defaults to NLS settings! validate_conversion ( <expression> as <data type>, [ <format mask> ], [ <nls_parameters> ] ) 1 = Success! 0 = Failure

32 select to_date(date_yyyymmdd, 'yyyymmdd') from bad_data_types where validate_conversion ( date_yyyymmdd as date, 'yyyymmdd' ) = 1;

33 this......must match this cast ( <expression> as <data type> [ default <value> on conversion error ], [ <format mask> ], [ <nls_parameters> ] ) Also... to_date() to_number() to_timestamp() etc.

34 select to_date( date_yyyymmdd default ' ' on conversion error, 'yyyymmdd' ) from bad_data_types; 31-DEC-9999

35 Wrong Data Types: Performance how to screw up the optimizer's estimates 1-JAN DEC-2017 = 1 DAY = 8,870

36

37 Flexible Easy to add new attributes Less time requirement gathering

38 Flexible Flexible Easy to add new attributes Less time requirement gathering Don't know contents until run-time More complex SQL

39 Functions ( ) json_value extract single value json_query return JSON fragment json_table convert JSON to relational table

40 Functions ( ) json_value extract single value json_query return JSON fragment json_table convert JSON to relational table but how do you know what to extract?

41 Data Guide

42 Data Guide ( ) alter table products add constraint product_desc_is_json_c check ( product_description_json is json ); create search index products_ji on products ( product_description_json ) for json parameters ( 'dataguide on' );

43 Data Guide ( ) begin DBMS_JSON.add_virtual_columns( ); end; / 'PRODUCTS', 'product_description_json', DBMS_JSON.get_index_dataguide('PRODUCTS', 'product_description_json', DBMS_JSON.FORMAT_HIERARCHICAL)

44 Data Guide ( ) begin DBMS_JSON.create_view ( 'products_v', 'PRODUCTS', 'product_description_json', DBMS_JSON.get_index_dataguide( 'PRODUCTS', 'product_description_json', DBMS_JSON.FORMAT_HIERARCHICAL) ); end;

45 Oracle Data Modeler Helps You Build Better Databases Use it!

46 Oracle Data Modeler #MakeDataGreatAgain Gratisography