CS317 File and Database Systems

Size: px

Start display at page:

Download "CS317 File and Database Systems"

Bryce Rice
5 years ago
Views:

1 CS317 File and Database Systems Lecture 8 Normalization, Bottom-Up from UNF to BCNF October 17, 2017 Sam Siewert

Inside a Datacenter E.g. Green House 1 Rack is 42 Rack Units [U = 1.75 ], 6.125 Feet High, Stacked HDD Storage 2U to 4U, 3.5 HDD SSD Storage 1U to 2U, 2.

Handling, Optional Liquid Cooling E.g. Emerson Liebert 120/240VAC Power Conditioning and Distribution E.g. Eaton and Pulizzi PDU DC Telco Rack Alternatives [Higher Efficiency, Less Convenient] Sam Siewert Typical Datacenter Rack [E.

2 Inside a Datacenter E.g. Green House 1 Rack is 42 Rack Units [U = 1.75 ], Feet High, Stacked HDD Storage 2U to 4U, 3.5 HDD SSD Storage 1U to 2U, 2.5 SSD Computing 1U to 4U Servers Standard Rack Depth = 36, Width = 19 or 23 Hot Rows [Fan Exhaust], Cold Rows [Front Panels] Power, Chillers for Air Handling, Optional Liquid Cooling E.g. Emerson Liebert 120/240VAC Power Conditioning and Distribution E.g. Eaton and Pulizzi PDU DC Telco Rack Alternatives [Higher Efficiency, Less Convenient] Sam Siewert Typical Datacenter Rack [E.g. DreamWorks, Microsoft, Green House, NCAR, NOAA, DoE, Xerox, Amazon, ] Sam Siewert 2

10KW Per Rack, 2 x 20Amp 240VAC PDU Dual Circuit for Dual Power Supply Computing and Storage Hot Swap Power Units in Servers and Storage Enclosures

g. 8 Cores/CPU, 4 CPU Sockets, HT3 or QPI, 32 per U Network Port Density and Bandwidth GigE, 10GE, 40G Infiniband or Bonded 10GE, 100G CEE Ports Per

3 Basic Datacenter Figures of Merit [2014] Power Density 10 KW Per Rack 120/240VAC, 20Amp Circuits, 1-Phase Loads E.g. 10KW Per Rack, 2 x 20Amp 240VAC PDU Dual Circuit for Dual Power Supply Computing and Storage Hot Swap Power Units in Servers and Storage Enclosures Storage Density TB/U, 1PB Rack is 24 to 48 TB/U TB HDDs, 180TB in 4U, 45TB/U RAID10 is Striped and Mirrored Storage (Typical for DBMS) E.g. LC SFP Compute Density 2 or 4 Socket 1U/2U Servers E.g. 8 Cores/CPU, 4 CPU Sockets, HT3 or QPI, 32 per U Network Port Density and Bandwidth GigE, 10GE, 40G Infiniband or Bonded 10GE, 100G CEE Ports Per Server, Copper [Twinax, TP, IB] or Multi-mode Optical [LC/SC SFP/SFP+ connectors], Typically 2 to 4 or More SFP Transceivers Copper or Optical LC/SC Connectors for Optical Switch Port Density Sam Siewert 3

4 Significance of Normalization Structured (Normalized), Indexed, Searchable, High Veracity Data E.g. 2TB HDD Could Hold 6.8 Kilobytes of Data on Every Person in US 1 Petabyte [RAID 10 42U Rack] Stores 3 Megabytes for Every US Citizen E.g. All Documented Life Events [Legal, Travel, Residency] 1000 Racks, 1 Exabyte Structured + Unstructured Data [ and Audio] NSA Utah Estimated to Store up to 3 to 12 Exabytes by Forbes 180 Petabytes for 24 hour Audio on 300 million People 300 Petabytes for 1 Year of Phone Conversations on all US Citizens [Forbes] Unstructured BLOB Files [Documents, Images, Audio, Video] Audio with Compression [10 to 200 hours of MP3 per 640MB CD] Easily 24 hours on an MP3 CD of Intelligible Conversation Images - JPEG Lossy [10:1 to 20:1], PNG Lossless [4:1 to 10:1] MPEG Compression is 20:1 to 100:1 [Lossy] for I-frame and MVQ B/P-frames 24 Hours of SD Video is about 50 Gigabytes of Data [10 SD DVDs] 14,305 Racks of RAID-10 Disk for a Day of 30Hz SD Video of All US Citizens Structured Data Financial & Legal Transactions, Records Unstructured Documents, , Audio, Snapshots, [Some Video] Not Only Capture and Store, but Search! Sam Siewert 4

5 Ex #4 Posted Reminders Assignment #3 Returned Next Week Assignment #5, Physical DB Design and Project! Assignment #6, Complete DBMS Project FINAL Design Schema for DBMS project in a small team Logical design focus Normalization Physical is MySQL on PRClab Combine Network Applications with DBMS in C/C++, JDBC, or Python - Add Stored Programs and Triggers Add Views Create Transactions where needed Sam Siewert 5

Normalization Concern is Duplication of Data in DBMS and Hazards Wastes

Second Insert of baddress, must match that Already Existing for SA9 Delete

baddress Change for B005 or B003 Foreign Keys are Exception (Expected

6 Normalization Concern is Duplication of Data in DBMS and Hazards Wastes Space Duplicate Data Insert Hazard - New Staff Row Also Assigned to B007, Second Insert of baddress, must match that Already Existing for SA9 Delete Hazard SA9 Quits, Row Delete, Lose B007 baddress Modification Hazard baddress Change for B005 or B003 Foreign Keys are Exception (Expected Redundancy for Relational Model) 14.3 UNF [1NF] Redundant Attribute Data Sam Siewert 6

7 The Process of Normalization [Follow Rules for Relational Table Design and Hints coming from ER/EER Information Model] UNF Paper, Spreadsheet 7

8 The Process of Normalization [Bottom-Up Tables] 8

9 UNF -> 3NF Minimizes Update Anomalies [Insert, Update, Delete], Page 420 to 426 One Client Renting Multiple Properties Typical of Spreadsheet, Paper UNF cno cname pno paddr start finish Rent ono oname CR76 John Kay PG4, PG16 6 Lawrence Street, 5 Novar Drive 7/1/12, 9/1/13 8/31/13, 9/1/14 350, 50 CO40, CO93 Tina Murphy, Tony Shaw CR56 Aline Stewart PG4, PG36, PG16 6 Lawrence Street, 2 Manor Road, 5 Novar Drive 9/1/11, 10/1/12, 11/1/14 6/10/12, 12/1/13, 8/10/ CO40 CO93 CO93 Tina Murphy, Tony Shaw, Tony Shaw Client Rental PropertyOwner 2NF Client Rental PropertyForRent Owner 3NF Sam Siewert 9

10 UNF -> 1NF UNF Table with ONE or MORE Repeating Groups [Tuple Sub-set] 1NF Relation where Intersection of Each Row and Column has ONE Value UNF cno cname pno paddr start finish Rent ono oname CR76 John Kay PG4, PG16 6 Lawrence Street, 5 Novar Drive 7/1/12, 9/1/13 8/31/13, 9/1/14 350, 50 CO40, CO93 Tina Murphy, Tony Shaw CR56 Aline Stewart PG4, PG36, PG16 6 Lawrence Street, 2 Manor Road, 5 Novar Drive 9/1/11, 10/1/12, 11/1/14 6/10/12, 12/1/13, 8/10/ CO40 CO93 CO93 Tina Murphy, Tony Shaw, Tony Shaw NF (Still Suffers all 3 Hazards) cno cname pno paddr start finish Rent ono oname CR76 John Kay PG4 6 Lawrence Street 7/1/12 8/31/ CO40 Tina Murphy CR76 John Kay PG16 5 Novar Drive 9/1/13 9/1/14 50 CO93 Tony Shaw CR56 Aline Stewart PG4 6 Lawrence Street 9/1/11 6/10/ CO40 Tina Murphy CR56 Aline Stewart PG36 2 Manor Road 10/1/12 12/1/ CO93 Tony Shaw CR56 Aline Stewart PG16 5 Novar Drive 11/1/14 8/10/ CO93 Tony Shaw Sam Siewert 10

11 1NF -> 2NF 1NF Relation where Intersection of Row and Column Has ONE Value 2NF 1NF Relation where Every Non-Primary Key Attribute is Fully Functionally Dependent on the PK [ER 1..1 to 1..1 Relations] NF (Suffers all 3 Hazards) cno cname pno paddr start finish Rent ono oname CR76 John Kay PG4 6 Lawrence Street 7/1/12 8/31/ CO40 Tina Murphy CR76 John Kay PG16 5 Novar Drive 9/1/13 9/1/14 50 CO93 Tony Shaw CR56 Aline Stewart PG4 6 Lawrence Street 9/1/11 6/10/ CO40 Tina Murphy CR56 Aline Stewart PG36 2 Manor Road 10/1/12 12/1/ CO93 Tony Shaw CR56 Aline Stewart PG16 5 Novar Drive 11/1/14 8/10/ CO93 Tony Shaw NF (Still Suffers Update Hazard Due to Transitive Dependency pno -> ono -> oname) cno [PK] CR76 CR56 cname John Kay Aline Stewart cno pno start finish CR76 PG4 7/1/12 8/31/13 CR76 PG16 9/1/13 9/1/14 CR56 PG4 9/1/11 6/10/12 CR56 PG36 10/1/12 12/1/13 CR56 PG16 11/1/14 8/10/15 pno paddress rent ono oname PG4 6 Lawrence Street 350 CO40 Tina Murphy PG16 5 Novar Drive 450 CO93 Tony Shaw Sam Siewert PG36 2 Manor Road 375 CO93 Tony Shaw 11

12 Figure Alternate 1NF cno [PK] CR76 CR56 cname John Kay Aline Stewart cno pno paddr start finish Rent ono oname CR76 PG4 6 Lawrence Street 7/1/12 8/31/ CO40 Tina Murphy CR76 PG16 5 Novar Drive 9/1/13 9/1/14 50 CO93 Tony Shaw CR56 PG4 6 Lawrence Street 9/1/11 6/10/ CO40 Tina Murphy CR56 PG36 2 Manor Road 10/1/12 12/1/ CO93 Tony Shaw CR56 PG16 5 Novar Drive 11/1/14 8/10/ CO93 Tony Shaw Sam Siewert 12

13 2NF -> 3NF [Also BCNF] 2NF 1NF Relation where Every Non-Primary Key Attribute is Fully Functionally Dependent on the PK [ER 1..1 to 1..1 Relations] 3NF 2NF Relation where no Non-PK Attribute is Transitively Dependent on a PK NF (Still Suffers Update Hazard Due to Transitive Dependency pno -> ono -> oname) cno [PK] CR76 CR56 cname John Kay Aline Stewart cno pno start finish CR76 PG4 7/1/12 8/31/13 CR76 PG16 9/1/13 9/1/14 CR56 PG4 9/1/11 6/10/12 CR56 PG36 10/1/12 12/1/13 CR56 PG16 11/1/14 8/10/15 pno paddress rent ono oname PG4 6 Lawrence Street 350 CO40 Tina Murphy PG16 5 Novar Drive 450 CO93 Tony Shaw PG36 2 Manor Road 375 CO93 Tony Shaw pno [PK] paddress rent ono [FK] PG4 6 Lawrence Street 350 CO40 PG16 5 Novar Drive 450 CO93 PG36 2 Manor Road 375 CO93 ono [PK] CO40 CO93 oname Tina Murphy Tony Shaw Sam Siewert 13

14 Lossless-Join Property of 3NF Fundamental Point P. 425 Lossless-Join Reversibility 3NF is a Process to Apply Relational Algebra Projections Creates a Lossless-Join Decomposition (Reducing or Eliminating Insert, Delete, Update Hazards) Using Natural Join (in a View) we Can Reverse View [Stored Query] Easily Regenerate 1NF Version UNF Could Be Re-created Via Application Report Generation 1. Elimination of Repeating Groups -> 1NF 2. 1NF -> Every Non-PK [CK?] Attribute Fully Functionally Dependent on PK [any CK?] -> 2NF (No Partial Dependencies Allowed) 3. 2NF -> No Non-PK [CK?] is Transitively Dependent on the PK [any CK?] -> 3NF Sam Siewert 14

15 Reminder SK, CK, PK, AK, FK SK Attribute of Set of Attributes that UNIQUELY identifies Tuple in Relation CK An SK, s.t. no Proper Subset is also an SK [Minimal] UNIQUIENESS CK uniquely Identifies all Tuples in Relation IRREDUCIBILITY No Proper Subset of CK has UNIQUENESS PK CK Selected to ID Tuples UNIQUELY in Relation AK CK Not Selected to be PK FK An Attribute or Set of Attributes in R 1 that Matches CK in R 1 or R 2..N Sam Siewert 15

16 Is 3NF Good Enough? Recall that PK Selection is From Set of CKs in Relation Dependencies on Remaining CKs not Used as PK? Strengthen 2NF and 3NF Definitions to Include ANY CK 1. 1NF -> Every Non-CK Attribute Fully Functionally Dependent on ANY CK -> 2NF (No Partial Dependencies Allowed) 2. 2NF -> No Non-CK is Transitively Dependent on ANY CK -> 3NF Even With STRONG 2NF & 3NF, Dependencies Can Still Cause Redundancy BCNF Considers Common Cases Sam Siewert 16

17 BCNF BCNF Relation is BCNF If-and-only-if Every Determinant is a CK Determinant Attribute or Group of Attributes on Which some OTHER Attribute is Fully Functionally Dependent 3NF allows A -> B if B is PK and A is not CK BCNF REQUIRES A to be CK [Further Constrains] Issues Arise when Relation contains 2+ Composite CKs CKs Overlap [Common Attribute] Stopping at 3NF Preferred [Sometimes] to Avoid Loss of Dependencies Sam Siewert 17

18 ClientInterview 3NF Relation BCNF Example cno intdate inttime staffno roomno CR76 5/13/14 10:30 SG5 G101 CR56 5/13/14 12:00 SG5 G101 CR74 5/13/14 12:00 SG37 G102 CR56 7/1/14 10:30 SG5 G102 ClientInterview 3 Candidate Keys 1. (cno, intdate) - PK 2. (staffno, intdate, inttime) - CK 3. (roomno, intdate, inttime) CK intdate is Overlap between 3 CKs (creates Hazard) (staffno, intdate) determinant is not a CK for ClientIntervew ClientInterview has following functional dependencies 1. (cno, intdate) -> inttime, staffno, roomno 2. (staffno, intdate, inttime) -> cno 3. (roomno, intdate, inttime) -> staffno, cno 4. staffno, intdate -> roomno Sam Siewert 18

19 ClientInterview 3NF Relation BCNF Example cno intdate inttime staffno roomno CR76 5/13/14 10:30 SG5 G101 CR56 5/13/14 12:00 SG5 G101 CR74 5/13/14 12:00 SG37 G102 CR56 7/1/14 10:30 SG5 G102 Interview Relation cno intdate inttime staffno CR76 5/13/14 10:30 SG5 CR56 5/13/14 12:00 SG5 CR74 5/13/14 12:00 SG37 CR56 7/1/14 10:30 SG5 Interview Relation staffno intdate roomno SG5 5/13/14 G101 SG37 5/13/14 G102 SG5 7/1/14 G102 Sam Siewert 19

Normalization. { Ronak Panchal }

Normalization. { Ronak Panchal } Normalization { Ronak Panchal } Chapter Objectives The purpose of normailization Data redundancy and Update Anomalies Functional Dependencies The Process of Normalization First Normal Form (1NF) Second