Evolving Database Design and Architecture Patterns and Practices

Transcription

1 Evolving Database Design and Architecture Patterns and Practices Pramod Sadalage ThoughtWorks 1

2 Patterns of Database Changes 2

3 Patterns of Database Changes Architecture 2

4 Patterns of Database Changes Architecture Structure 2

5 Patterns of Database Changes Architecture Structure Data Quality 2

6 Patterns of Database Changes Architecture Structure Data Quality Referential Integrity 2

7 Patterns of Database Changes Architecture Structure Data Quality Referential Integrity Database Code 2

8 Timeline of Change Deploy new changes, migrate data, put in scaffolding code Remove old schema, scaffolding code Original Transition Result { Implement the refactoring { Transition Period (old and new) { Refactoring completed 3

9 Architecture Patterns Changes that improve the over all manner in which external programs interact with the database 4

10 Add Read Method 5

11 Introduce Read Only Table 6

12 Encapsulate Table with View 7

13 Migrate Method from Database 8

14 Introduce Calculation Method 9

15 Migrate Method to Database 10

16 more at 11

17 Structural Change Patterns Change the structure of the database schema, for better database design 12

18 Split Column 13

19 Replace One-To-Many with Associative Table 14

20 Rename Table 15

21 Merge Columns 16

22 Replace Column 17

23 Split Table 18

24 more at 19

25 Data Quality Patterns Changes that improve the quality of the information with in a database or ensure the consistency and usage of data 20

26 Make Column Non-Nullable 21

27 Introduce Column Constraint 22

28 Add Lookup Table 23

29 more at 24

30 Referential Integrity Patterns Changes ensure Referential Data maintained is making Data Quality sure is improved 25

31 Add Foreign Key Constraint 26

32 Introduce Cascading Delete 27

33 more at 28

34 Database Code Like code refactoring (Fowler refactor 1999) database code to improve design the of database code i.e Stored Procs and Triggers 29

35 Practices Without good development Practices, using and implementing the Patterns is going to be difficult. 30

36 Configuration Management 31

37 Configuration Management Allow for common code ownership 31

38 Configuration Management Allow for common code ownership All database artifacts belong in source control repository 31

39 Configuration Management Allow for common code ownership All database artifacts belong in source control repository Include setup/config data 31

40 Configuration Management Allow for common code ownership All database artifacts belong in source control repository Include setup/config data Publish database artifacts with Continuous Integration 31

41 Give everyone a database Sandbox 32

42 Give everyone a database Sandbox Reduce waste and waiting time 32

43 Give everyone a database Sandbox Reduce waste and waiting time Automate data tasks 32

44 Give everyone a database Sandbox Reduce waste and waiting time Automate data tasks Allows local changes and experimenting 32

45 Give everyone a database Sandbox Reduce waste and waiting time Automate data tasks Allows local changes and experimenting Improves productivity 32

46 Give everyone a database Sandbox Reduce waste and waiting time Automate data tasks Allows local changes and experimenting Improves productivity Spinning up new environments is easy 32

47 33

48 Behavior of Database 34

49 Behavior of Database Like objects, database has behavior 34

50 Behavior of Database Like objects, database has behavior Develop database objects using BDD style tests 34

51 Behavior of Database Like objects, database has behavior Develop database objects using BDD style tests Allows easy changes to the database 34

52 Behavior of Database Like objects, database has behavior Develop database objects using BDD style tests Allows easy changes to the database Protects against changes that affect dependent functionality 34

53 35

54 Requirements: Store a Vehicle with Unique VIN Number Model Year 2005 and above Model Year Not Null Model Name Not Null Make Not Null Miles not above

55 Requirements: Store a Vehicle with Unique VIN Number Model Year 2005 and above Model Year Not Null Model Name Not Null Make Not Null Miles not above

56 Requirements: Store a Vehicle with Behavior Tests: Unique VIN Number Model Year 2005 and above Model Year Not Null Model Name Not Null Make Not Null Miles not above shouldnotsaveduplicatevin() shouldsavemodelyear2010() shouldnotsavemodelyear2004() shouldnotsavenullmodelname() shouldnotsavenullmake() shouldsavemiles5000() shouldnotsavemiles12000() 35

57 CREATE TABLE vehicle(!i d N U M B E R ( 1 8 ) N O T N U L L,! vin VARCHAR2(32) NOT NULL,! name VARCHAR2(32) NOT NULL,! make VARCHAR2(32) NOT NULL,! year NUMBER(4) NOT NULL,! miles NUMBER(10) NULL, CONSTRAINT chk_vehicle_year_gt_2005!!! CHECK (year> 2004)); CONSTRAINT chk_vehicle_miles_lt_10001!!! CHECK (miles< 10001)); CREATE UNIQUE INDEX uidx_vehicle_vin!!! ON vehicle(vin);! ALTER TABLE VEHICLE ADD CONSTRAINT!!! pk_vehicle PRIMARY KEY (id); 36

58 Tracking Changes 37

59 Tracking Changes Write each change as a delta (migration) script 37

60 Tracking Changes Write each change as a delta (migration) script Migration scripts are development time activity not deployment time project 37

61 Tracking Changes Write each change as a delta (migration) script Migration scripts are development time activity not deployment time project Package delta scripts for automated deployment 37

62 Tracking Changes Write each change as a delta (migration) script Migration scripts are development time activity not deployment time project Package delta scripts for automated deployment Same scripts for: developers, QA, UAT and Production 37

63 38

64 ALTER TABLE customer ADD customeridentifier VARCHAR2(12); UPDATE customer SET customeridentifier = customernumber; --If No Transistion Period ALTER TABLE customer DROP COLUMN customernumber; --//@UNDO ALTER TABLE customer ADD customernumber NUMBER(10); UPDATE customer SET customernumber = customeridentifier; ALTER TABLE customer DROP COLUMN customeridentifier; 38

65 Deployed Database Version 39

66 Continuous Integration 40

67 Continuous Integration Test application code and database at one place 40

68 Continuous Integration Test application code and database at one place Generate code and database artifacts 40

69 Continuous Integration Test application code and database at one place Generate code and database artifacts Integrate application and database changes in an independent environment 40

70 Check in application code and database migration scripts Local dev environment Central integration environment PROD CT QA ANT Maven Make Rake Source Control Continuous Integration Engine PROD UAT QA Environment Dev DB Update and build Dev Database 41

71 Check in application code and database migration scripts Local dev environment Central integration environment Migration scripts PROD CT QA ANT Maven Make Rake Source Control Continuous Integration Engine PROD UAT QA Environment Dev DB Update and build 41

72 Check in application code and database migration scripts Local dev environment Migration scripts Central integration environment Migration scripts PROD CT QA ANT Maven Make Rake Source Control Continuous Integration Engine PROD UAT QA Environment Dev DB Update and build Integration Database 41

73 Check in application code and database migration scripts Local dev environment Migration scripts Central integration environment Migration scripts PROD CT QA ANT Maven Make Rake Source Control Continuous Integration Engine PROD UAT QA Environment Dev DB Update and build Integration Database 41

74 Check in application code and database migration scripts Local dev environment Migration scripts Central integration environment Migration scripts PROD CT QA ANT Maven Make Rake Source Control Continuous Integration Engine PROD UAT QA Environment Dev DB Update and build Integration Database Apply migration scripts 41

75 Check in application code and database migration scripts Local dev environment Migration scripts Migration scripts Central integration environment Migration scripts PROD CT QA ANT Maven Make Rake Source Control Continuous Integration Engine Artifacts War Jar PROD UAT QA Environment Dev DB Update and build Integration Database Apply migration scripts 41

76 Check in application code and database migration scripts Local dev environment Migration scripts Migration scripts Central integration environment Migration scripts Migration scripts PROD CT QA ANT Maven Make Rake Source Control Continuous Integration Engine Artifacts War Jar PROD UAT QA Environment Dev DB Update and build Integration Database Apply migration scripts War Jar 41

77 Deployment 42

78 Deployment Database migration/upgrade should be a development time task not deployment time task 42

79 Deployment Database migration/upgrade should be a development time task not deployment time task Package all the migration scripts, during Continuous Integration cycle 42

80 Deployment Database migration/upgrade should be a development time task not deployment time task Package all the migration scripts, during Continuous Integration cycle Apply these migration scripts 42

81 43

82 Pair with the Data Team 44

83 Pair with the Data Team Break down the silos 44

84 Pair with the Data Team Break down the silos Allows continuous reviews 44

85 Pair with the Data Team Break down the silos Allows continuous reviews Understand performance implications early 44

86 Pair with the Data Team Break down the silos Allows continuous reviews Understand performance implications early Put database code and application code in same repository 44

87 Resources bit.ly/qconbddd bit.ly/evolvedb 45

88