Databases and Database Programming. Copyright 2017 by Robert M. Dondero, Ph.D. Princeton University

Transcription

1 Databases and Database Programming Copyright 2017 by Robert M. Dondero, Ph.D. Princeton University 1

2 Objectives You will learn about: Databases and database management systems Transactions SQL and SQLite Database programming in Java and Python Relational database design Beyond relational databases (if time) 2

3 Objectives Note: Comprehensive coverage is impossible! Please supplement with the reading 3

4 Agenda DBs and DBMSs Transactions SQL and SQLite DB programming in Java and Python Relational DB design Beyond relational DBs (if time) 4

5 DB and DBMS Some definitions... Database (DB) A structured collection of data An abstract view of a file or collection of files stored persistently (disk, flash drive, ) 5

6 DB and DBMS Database management system (DBMS) Software that maintains a database Usually, a server Listens on a known host at a known port Clients contact server to perform queries and updates 6

7 Motivation for DB and DBMS Question: Why not simply use files? Answer: Centralized control Database administrators (DBAs) control the data 7

8 Motivation for DB and DBMS A good DBMS used by good DBAs can: Reduce redundancy Avoid inconsistencies Facilitate data sharing Enforce standards Apply security restrictions Maintain integrity Balance conflicting requirements Insure safety (backups) An Introduction to Database Systems, C. J. Date 8

9 Types of DBs In historical order... Navigational DBs Relational DBs 9

10 Navigational DBs Data are linked into tree or network/graph structure User/program is given root node User/program follows links to desired data Example: 10

11 Example Navigational DB BOOKS isbn title quantity ORDERS custid quantity author AUTHORS CUSTOMERS custname street zipcode ZIPCODES city state 11

12 Example Nav DB Query Which customers purchased the book whose ISBN is 123? BOOKS (isbn, title, quantity) 123 The Practice of Programming 500 ORDERS (custid, quantity) CUSTOMERS (custname, street, zipcode) Princeton 114 Nassau St Harvard 1256 Mass Ave

13 Navigational DB Limitations In the example: Good: Given book, easy to find customers Bad: Given customer, hard to find books In general: Queries are biased DB designer must anticipate queries to create appropriate links 13

14 Relational DBs Who: Edgar Codd When: Where: IBM What: Applied some mathematics Eliminate all links!!! 14

15 Relational DB Structure Relational database structure Formally: Database consists of relations which have tuples and attributes Informally: Database consists of tables which have rows and columns Example... 15

16 Relational DBs BOOKS isbn title quantity 123 The Practice of Programming The C Programming Language Algorithms in C 650 ORDERS isbn custid quantity AUTHORS isbn author 123 Kernighan 123 Pike 234 Kernighan 234 Ritchie 345 Sedgewick CUSTOMERS custid custname street zipcode 111 Princeton 114 Nassau St Harvard 1256 Mass Ave MIT 292 Main St ZIPCODES zipcode city state Princeton NJ Cambridge MA Cambridge MA 16

17 Relational DBs No links Queries are unbiased However: DBA can create links (indices), based upon anticipated usage patterns DBMS can create links (indices), based upon observed usage patterns Today, relational DBs are (still!) the most popular kind 17

18 Agenda DBs and DBMSs Transactions SQL and SQLite DB programming in Java and Python Relational DB design Beyond relational DBs (if time) 18

19 Motivation for Transactions Problem 1: Recovery DBMS must recover from HW/SW failures Problem 2: Concurrency DBMS must handle updates from multiple concurrent clients Both problems are solved by transactions 19

20 Problem 1: Recovery Recovery DBMS must recover from HW/SW failures DBMS must preserve DB consistency in the presence of HW/SW failures 20

21 Problem 1: Recovery Example: Customer 222 has purchased 1 copy of book 123 Must change ORDERS: add 1 to quantity of appropriate row Must change BOOKS: subtract 1 from quantity of appropriate row 21

22 Problem 1: Recovery Time Increment quantity of the ORDERS row whose isbn is 123 and custid is 222 Decrement quantity of the BOOKS row whose isbn is 123 HW/SW error DB becomes inconsistent!!! 22

23 Solution 1: Transactions Transaction A logical unit of work A sequence of operations that the DBMS performs atomically DBMS executes all or none of the operations Transforms DB from one consistent state to another Without necessarily preserving consistency at intermediate points 23

24 Solution 1: Transactions Pattern: (1) Execute BEGIN stmt to begin transaction (2) Perform update(s) (3) Execute: COMMIT stmt to commit updates to DB and end transaction, or ROLLBACK stmt to discard updates and end transaction 24

25 Solution 1: Transactions BEGIN Time Increment quantity of the ORDERS row whose isbn is 123 and custid is 222 HW/SW error Decrement quantity of the BOOKS row whose isbn is 123 COMMIT DBMS rolls back first update DB remains consistent 25

26 Problem 2: Concurrency Concurrency Multiple processes may access DB concurrently DBMS must maintain DB consistency in the presence of concurrent updates 26

27 Problem 2: Concurrency Time Process A Fetch row R (quan=100) quan+=5 Update row R (quan=105) Process B Fetch row R (quan=100) quan+=5 Update row R (quan=105) Process A's update is lost; DB is inconsistent 27

28 Solution 2: Transactions Transaction A recovery mechanism (as previously described), and A locking mechanism Oversimplification... Process can ask DBMS to lock DB rows that are involved in active transaction Other processes cannot access locked DB rows 28

29 Solution 2: Transactions Time Process A BEGIN Fetch row R (quan=100) Acquire lock on R quan += 5 Update row R (quan=105) COMMIT Release lock on R BEGIN Process B Fetch row R (quan=105) Acquire lock on R Fetch row R (quan=105) Acquire lock on R quan += 5 Update row R (quan=110) COMMIT Release lock on R B cannot access row R until A commits 29

30 The ACID Test DBMS transaction mechanism must pass the ACID test Atomicity: each transaction must complete or rollback completely Consistency: each transaction must not violate any consistency constraints (see later this lecture) Isolation: transactions should not interfere with each other; same results if transactions are executed sequentially or in parallel Durability: once committed, transaction remains persistent even if HW/SW failures occur 30

31 Agenda DBs and DMBSs Transactions SQL and SQLite DB programming in Java and Python Relational DB design Beyond relational DBs (if time) 31

32 SQL Who: Donald Chamberlin & Raymond Boyce When: 1970s Where: IBM Why: Manipulate data in IBM s System R relational DBMS 32

33 SQL SQL Structured Query Language Has been standardized ISO/IEC :2008 Now the de facto standard for communicating with relational DBMSs 33

34 SQLite Who: D. Richard Hipp When: 2000 Where: General Dynamics Why: Eliminate distinct DBMS process, eliminate DBAs 34

35 SQLite SQLite A popular free relational DBMS Uses SQL (no surprise!) Extends SQL with additional statements As DBMSs typically do Lightweight (no surprise!) Good choice for learning SQL 35

36 Typical DBMS Structure Application program Socket DBMS program File I/O Data store DBMS is a program App pgm & DBMS pgm run in distinct processes App process & DBMS process communicate via sockets (see upcoming Networks lecture) 36

37 SQLite Structure Application program SQLite module File I/O Data store DBMS is a module App pgm & DBMS module run in same process App pgm & DBMS module communicate via function/method calls 37

38 Using the SQLite Client SQLite command-line client From a courselab shell prompt: sqlite3 filename Uses existing database filename, or Creates database filename Example: sqlite3 bookstore.sqlite Type SQLite/SQL statements at sqlite> prompt 38

39 Using the SQLite Client Suggestion: Install SQLite command-line client on your computer Mac and MS Windows Download from Linux Use package mgr to install sqlite3 (or similarly named) package 39

40 SQLite/SQL Statements SQLite statements Begin with. Keywords are case sensitive Standard SQL statements Do not begin with. Keywords are case insensitive Must end with semicolon 40

41 SQLite: Fundamentals.help.help.quit.quit.tables.tables.schema [table].schema.schema books 41

42 SQLite Data Types Some SQLite data types: SQLite Java Python INTEGER int or long int REAL double float TEXT String unicode BLOB Object object NULL type null type NoneType 42

43 SQLite: Reading/Writing.read file.read bookstore.sql.output file.output xxx.sql Subsequent dump goes to file.dump.dump 43

44 SQL: Creating/Destroying Tables DROP TABLE [IF EXISTS] table; drop table books; CREATE TABLE [IF NOT EXISTS] table (column datatype, ); CREATE TABLE books (isbn TEXT, title TEXT, quantity INTEGER); 44

45 SQL: Altering Table Schema ALTER TABLE table specification [, specification] ; ALTER TABLE books ADD price INTEGER FIRST; ALTER TABLE books ADD pages INTEGER AFTER quantity; ALTER TABLE books DROP price, DROP pages; ALTER TABLE table ADD INDEX(column); ALTER TABLE books ADD INDEX (isbn); Enables fast search on isbn column Matters for large tables (see COS 333 assignments) 45

46 SQL: Selecting Table Data SELECT expr, FROM table, [WHERE condition] [ORDER BY column [ASC DESC]]; SELECT * FROM books; SELECT * FROM authors; SELECT * FROM customers; SELECT * FROM orders; SELECT * FROM zipcodes; SELECT isbn, title FROM books; SELECT * FROM books ORDER BY quantity DESC; Note: Result is a table 46

47 SQL: Selecting Table Data SELECT * FROM books WHERE quantity = 650; SELECT * FROM books WHERE quantity >= 650; SELECT * FROM orders SELECT * FROM orders WHERE isbn = 123 AND custid = 222; SELECT * FROM orders WHERE isbn = 123 OR custid = 222; SELECT * FROM books WHERE title LIKE 'The%'; SELECT * FROM books WHERE title LIKE '%of%'; SELECT * FROM books WHERE title LIKE 't_e%'; Note: Result is a table 47

48 SQL: Selecting Table Data SELECT books.title, authors.author from books, authors; Cartesian product; 15 rows, but only 5 are meaningful! SELECT books.title, authors.author from books, authors WHERE books.isbn = authors.isbn; SELECT title, author from books, authors WHERE books.isbn = authors.isbn; SELECT custname, title, orders.quantity FROM books, customers, orders WHERE books.isbn = orders.isbn AND orders.custid = customers.custid; Note: Result is a table 48

49 SQL: Changing Table Data INSERT INTO table (column, ) VALUES (expr, ); INSERT INTO books (isbn, title, quantity) VALUES('456', 'Core Java', 120); DELETE FROM table [WHERE condition]; DELETE FROM books; -- Be careful!!! DELETE FROM books WHERE title LIKE 'The%'; UPDATE table SET column1=expr1 [, column2=expr2 ] [WHERE condition]; UPDATE books SET quantity=60 WHERE isbn=123; UPDATE books SET quantity=quantity+1 WHERE isbn=123; 49

50 SQL: Transactions BEGIN; Begins a transaction COMMIT; Commits and ends the active transaction ROLLBACK; Rolls back and ends the active transaction 50

51 UPDATE books SET quantity = WHERE isbn = 123;.quit SQL: Transactions BEGIN; UPDATE books SET quantity = WHERE isbn = 123;.quit BEGIN; UPDATE books SET quantity = WHERE isbn = 123; COMMIT;.quit BEGIN; UPDATE books SET quantity = WHERE isbn = 123; ROLLBACK;.quit 51

52 Agenda DBs and DBMSs Transactions SQL and SQLite DB programming in Java and Python Relational DB design Beyond relational DBs (if time) 52

53 Database Drivers Program must use a database driver to communicate with DBMS Program Database driver DBMS 53

54 Java Database Drivers Generally: Java program JDBC interface DBMS-specific impl Socket Database protocol DBMS File I/O Data store 54

55 Java Database Drivers Example: Java program JDBC interface Oracle JDBC impl Socket Oracle protocol Oracle DBMS File I/O Data store 55

56 Java SQLite Database Driver In our case: Java program JDBC interface sqlite-jdbc.jar SQLite protocol SQLite DBMS File I/O Data store sqlite-jdbc.jar must be in CLASSPATH 56

57 Python Database Drivers Generally: Python program DBAPI DBMS-specific impl Socket Database protocol DBMS File I/O Data store 57

58 Python Database Drivers Example: Python program DBAPI Oracle DBAPI impl Socket Oracle protocol Oracle DBMS File I/O Data store 58

59 Python SQLite Database Driver In our case: Python program DBAPI sqlite3 SQLite protocol SQLite DBMS File I/O Data store sqlite3 module must be installed 59

60 AuthorSearch Programs Illustrate: Database connectivity Execution of SELECT statements Handling of result tables 60

61 AuthorSearch in Java See AuthorSearch.java Accessing a SQLite database Use of cursor to iterate over result table 61

62 AuthorSearch in Python See authorsearch.py Accessing a SQLite database Use of cursor to iterate over result table Cursor provides each row as list 62

63 AuthorSearch in Python Commentary In relational model, columns are (in principle) unordered Don t execute SELECT * from code Python sqlite3 module should provide each row as a dict, not a list 63

64 Purchase Programs Illustrate: Database connectivity Execution of UPDATE statements Transactions 64

65 Purchase in Java See Purchase.java Execution of UPDATE statements Transactions connection.setautocommit(false); Commands connection not to automatically commit after each update Program must explicitly commit First update implicitly begins a transaction connection.commit() Commits and ends transaction 65

66 Purchase in Python See purchase.py Execution of UPDATE statements Transactions First update implicitly begins a transaction connection.commit() Commits and ends transaction 66

67 Recovery Programs Illustrate: Using transactions for recovery See Recovery.java, recovery.py Error occurs at unpredictable time between updates Database consistency is preserved 67

68 SQL Injection Attacks The problem (via an example)... Consider this SQL statement from AuthorSearch programs: SELECT books.isbn, title, quantity FROM books, authors WHERE books.isbn = authors.isbn AND author = 'someauthor' 68

69 SQL Injection Attacks Suppose (malicious?) user provides this someauthor: junk' OR 'x'='x 69

70 SQL Injection Attacks SELECT books.isbn, title, quantity FROM books, authors WHERE books.isbn = authors.isbn AND author = 'someauthor' junk' OR 'x'='x SELECT books.isbn, title, quantity FROM books, authors WHERE books.isbn = authors.isbn AND author = 'junk' OR 'x'='x' Note: AND has higher precedence than OR 70

71 SQL Injection Attacks Resulting statement: Selects all rows of Cartesian product Accesses data that user is not authorized to access??? Causes denial-of-service??? Implements a SQL injection attack 71

72 SQL Injection Attacks The problem (via another example)... Consider this SQL statement from Purchase programs: UPDATE orders SET quantity = quantity+1 WHERE isbn = someisbn AND custid = somecustid 72

73 SQL Injection Attacks Suppose (malicious?) user provides this someisbn 123 and this somecustid: 222 OR 'x'='x' 73

74 SQL Injection Attacks UPDATE orders SET quantity = quantity+1 WHERE isbn = someisbn AND custid = somecustid OR 'x'='x UPDATE orders SET quantity = quantity+1 WHERE isbn = 123 AND custid = 222 OR 'x'='x' Note: AND has higher precedence than OR 74

75 SQL Injection Attacks Resulting statement: Updates all rows of ORDERS table Corrupts the DB!!! Implements a SQL injection attack For more examples: 75

76 Prepared Statements A solution... Prepared statements Program commands driver to compile SQL statement, with placeholders Program sends user data to driver Driver populates placeholders with user data SQL statement parsing is unrelated to user data E.g. Presence of OR in user data does not affect parsing 76

77 Prepared Pgms Illustrate: Prepared statements See AuthorSearchPrepared.java See authorsearchprepared.py See PurchasePrepared.java See purchaseprepared.py Prepared statements prohibit SQL injection attacks Always use prepared statements 77

78 Agenda DBs and DBMSs Transactions SQL and SQLite DB programming in Java and Python Relational DB design Beyond relational DBs (if time) 78

79 Example Database DB0 Design: BOOKS: isbn, title, authors, quantity ORDERS: isbn, custid, custname, street, city, state, zipcode, quantity 79

80 Example data: Example Database DB0 BOOKS isbn title authors quantity 123 The Practice of Programming {Kernighan Pike} The C Programming Language {Kernighan Ritchie} Algorithms in C Sedgewick 650 ORDERS isbn custid custname street city state zipcode quantity Harvard 1256 Mass Ave Cambridge MA Harvard 1256 Mass Ave Cambridge MA Princeton 114 Nassau St Princeton NJ

81 Normal Forms Observation: Design seems wrong Note redundancy Can we be more formal? Are there rules that we can apply to: Determine that the design is wrong? Make it right? 81

82 Normal Forms Database researchers have developed theories to help DBAs: Develop right designs, or... Consciously decide to use a wrong design In particular, Codd proposed 3 normal forms 82

83 First Normal Form Informally... Def: A table is in first normal form iff each column contains only atomic values 83

84 DB0 Not in First Normal Form BOOKS isbn title authors quantity 123 The Practice of Programming {Kernighan Pike} The C Programming Language {Kernighan Ritchie} Algorithms in C Sedgewick 650 ORDERS isbn custid custname street city state zipcode quantity Harvard 1256 Mass Ave Cambridge MA Harvard 1256 Mass Ave Cambridge MA Princeton 114 Nassau St Princeton NJ

85 Example Database DB1 Design BOOKS: isbn, title, quantity AUTHORS: isbn, author ORDERS: isbn, custid, custname, street, city, state, zipcode,quantity 85

86 Example Database DB1 Example data: BOOKS isbn title quantity 123 The Practice of Programming The C Programming Language Algorithms in C 650 AUTHORS isbn author 123 Kernighan 123 Pike 234 Kernighan 234 Ritchie 345 Sedgewick ORDERS isbn custid custname street city state zipcode quantity Harvard 1256 Mass Ave Cambridge MA Harvard 1256 Mass Ave Cambridge MA Princeton 114 Nassau St Princeton NJ

87 Candidate Keys Def: A candidate key for a table is a minimal set of columns that uniquely identifies any particular row of that table 87

88 Candidate Keys in DB1 Candidate keys: BOOKS isbn title quantity 123 The Practice of Programming The C Programming Language Algorithms in C 650 AUTHORS isbn author 123 Kernighan 123 Pike 234 Kernighan 234 Ritchie 345 Sedgewick ORDERS isbn custid custname street city state zipcode quantity Harvard 1256 Mass Ave Cambridge MA Harvard 1256 Mass Ave Cambridge MA Princeton 114 Nassau St Princeton NJ

89 Primary Keys Def: We choose one candidate key to be the primary key of the table; the others are called alternate keys of the table In DB1: Each table has only one candidate key It s obvious which candidate key to choose as the primary key 89

90 Functional Dependence Def: A column C2 of a table is functionally dependent on a column C1 iff, for each row in the table, the value of C1 determines the value of C2 In DB

91 DB1 Functional Dependencies BOOKS isbn title quantity AUTHORS isbn author ORDERS quantity isbn custid custname street zipcode city state 91

92 Second Normal Form Informally... A table is in second normal form iff: It is in first normal form, and No non-candidate-key column is dependent on any proper subset of any candidate key 92

93 DB1 Not in Second Norm Form BOOKS isbn title quantity AUTHORS isbn author ORDERS quantity isbn custid custname street zipcode city state 93

94 Example Database DB2 Design BOOKS: isbn, title, quantity AUTHORS: isbn, author CUSTOMERS: custid, custname, street, city, state, zipcode ORDERS: isbn, custid, quantity 94

95 Example Database DB2 BOOKS isbn title quantity 123 The Practice of Programming The C Programming Language Algorithms in C 650 ORDERS isbn custid quantity AUTHORS isbn author 123 Kernighan 123 Pike 234 Kernighan 234 Ritchie 345 Sedgewick CUSTOMERS custid custname street city state zipcode 111 Princeton 114 Nassau St Princeton NJ Harvard 1256 Mass Ave Cambridge MA MIT 292 Main St Cambridge MA

96 DB2 in Second Norm Form BOOKS isbn title quantity AUTHORS isbn author ORDERS CUSTOMERS isbn custid quantity custid custname street city zipcode state 96

97 Third Normal Form Informally... A table is in third normal form iff: It is in second normal form, and Every non-candidate-key column is nontransitively dependent on the candidate keys 97

98 DB2 not in Third Norm Form BOOKS isbn title quantity AUTHORS isbn author ORDERS CUSTOMERS isbn custid quantity custid custname street city zipcode state 98

99 Example Database DB3 Design BOOKS: isbn, title, quantity AUTHORS: isbn, author CUSTOMERS: custid, custname, street, zipcode ZIPCODES: zipcode, city, state ORDERS: isbn, custid, quantity 99

100 Example Database DB3 BOOKS isbn title quantity 123 The Practice of Programming The C Programming Language Algorithms in C 650 ORDERS isbn custid quantity AUTHORS isbn author 123 Kernighan 123 Pike 234 Kernighan 234 Ritchie 345 Sedgewick CUSTOMERS custid custname street zipcode 111 Princeton 114 Nassau St Harvard 1256 Mass Ave MIT 292 Main St ZIPCODES zipcode city state Princeton NJ Cambridge MA Cambridge MA 100

101 DB3 in Third Norm Form BOOKS isbn title quantity AUTHORS isbn author CUSTOMERS custname custid street zipcode ORDERS isbn quantity custid ZIPCODES city zipcode state 101

102 Relational DB Design Wrap-Up Some additional points... Database designers routinely violate normal forms But a good one does so: Only with a purpose Knowing the consequences 102

103 Relational DB Design Wrap-Up DBMS can enforce additional consistency constraints; e.g.: Primary key values cannot be null Primary key values must be unique within a table Within a table, foreign key values must correspond to primary key values in another table E.g. orders.isbn is a foreign key with respect to books.isbn 103

104 Relational DB Design Wrap-Up There is a substantial mathematical theory of relational database design More precise definitions of normal forms Several additional normal forms Relational algebra, relational calculus Foreign keys, integrity rules... See An Introduction to Database Systems (C. J. Date) 104

105 Agenda DBs and DBMSs Transactions SQL and SQLite DB programming in Java and Python Relational DB design Beyond relational DBs (if time) 105

106 Relational DB Limitations Relational DBs: Create an impedence mismatch Relational data model (tables, rows, columns) differs from OOP data model (objects, object references, object composition, class inheritance) Awkward to map relational data to (software) objects (e.g., cursors) Object-relational mapping (ORM) software exists See 106

107 Relational DB Limitations Relational DBMSs: Often are more complex than necessary Many apps don t need query neutrality, strict ACID transactions, One size does not fit all 107

108 DB Data Models Data Model Relational Non-Relational (NoSQL) Key-Value Stores Document Stores Wide-Column Stores Some info for next 5 slides derived from 108

109 Key-Value Stores Key-value store Values: Arbitrary bytes Data structure: key-value pairs Persistent hash table, or search tree (for ordering) Simple; fast Examples: Redis, Memcached, Microsoft Azure Cosmos DB, Hazelcast, Ehcache 109

110 Document Stores Document store Values: Documents Documents sometimes have internal structure (e.g. JSON) which DBMS can process Data structure: key-value pairs Can access document by key or sometimes by content Examples: MongoDB, Amazon DynamoDB, Couchbase, CouchDB, MarkLogic 110

111 Wide-Column Stores Wide-column store Values: Arbitrary bytes Data structure: Multidimensional associative array Typically sparsely populated Origin: Google BigTable Examples: Cassandra, HBase, Microsoft Azure Cosmos DB 111

112 Popular DBMS and Data Models According to as of Sept 2017: Rank DBMS DB Data Model Score 1 Oracle Relational MySQL Relational Microsoft SQL Server Relational PostgreSQL Relational MongoDB Document Store DB2 Relational Microsoft Access Relational Cassandra Wide-Column Store Redis Key-Value Store SQLite Relational

113 DBMS Topologies Topology Centralized Distributed Shared Memory Shared Disk Shared Nothing 113

114 DBMS Topologies Centralized One DBMS process Distributed More than one DBMS process Shared memory Multiple DBMS processes share the same memory and file system Shared disk Multiple DBMS processes use distinct memory but share the same file system Shared nothing Multiple DBMS processes, each having its own distinct memory and file system Data is split or replicated across servers 114

115 The CAP Theorem Who: Eric Brewer Where: UC Berkeley When:

116 The CAP Theorem Desired qualities of a distributed sharednothing DBMS: Consistency: Every read from node Y receives the most recent write to node X Availability: Every read from node Y receives a response, but maybe not the most recent write to node X Partition tolerance: The system continues to have same behavior even if its nodes become partitioned 116

117 The CAP Theorem The CAP theorem It is impossible for a distributed data store to simultaneously provide more than 2 of those 3 qualities Any distributed DBMS must have P Any distributed DBMS cannot have both C and A 117

118 The CAP Theorem Node X Node Y write read User 1 User 2 Normal case User 1 writes to X X communicates change to Y User 2 reads from Y 118

119 The CAP Theorem Node X Node Y write read User 1 User 2 Suppose X and Y become partitioned Suppose we claim partition tolerance (P) 119

120 The CAP Theorem Node X Node Y write read User 1 User 2 Option 1: User 1 writes to X User 2 read from Y returns immediately Consistency (C)? No Availability (A)? Yes Partition tolerance (P)? Yes 120

121 The CAP Theorem Node X Node Y write read User 1 User 2 Option 2: User 1 writes to X User 2 read from Y blocks Consistency (C)? Yes Availability (A)? No Partition tolerance (P)? Yes 121

122 DBMS Topologies Topology Centralized Distributed Shared Memory Shared Disk Shared Nothing CP AP 122

123 DBMS Topologies Shared nothing CP AP Provide consistency and partition tolerance Goal: data quality (i.e. provide consistent data) Provide availability and partition tolerance Goal: data quantity (i.e. provide data quickly) 123

124 Model/Topology Summary DB data models Relational NoSQL Key-value store Document store Wide column store DBMS topologies Centralized Decentralized Shared memory Shared disk Shared nothing CP AP

125 Model/Topology Combinations Which model/topology combination would you use for: A small bank? A large bank? The Google search engine? 125

126 Model/Topology Combinations Some combinations of model + topology are difficult E.g.: relational + shared nothing Split vertically: OK Split horizontally & key identifies server: OK Split horizontally & key does not identify server (sharding): hard Motivation for NoSQL DBMSs 126

127 Summary We have covered... DBs and DBMSs Transactions SQL and SQLite DB programming in Java and Python Relational DB design Beyond relational DBs (if time) 127