How They Work. Larry Page (one of the founders of Google) wrote the following in July 1996:

Size: px

Start display at page:

Download "How They Work. Larry Page (one of the founders of Google) wrote the following in July 1996:"

Rosamond Peters
5 years ago
Views:

1 Databases How They Work Back in 1996 Larry Page (one of the founders of Google) wrote the following in July 1996: I am almost out of disk space. I have downloaded about 24 million unique URLs and about 100 million links I think I will need 8 gigs more to store everything Current retail prices are about $1000/4 gigs I have only about 15% of the pages but it seems promising.

2 Why Databases? Now (2006) there are tens of billions (1 billion = 10^9) of documents on the Internet. How can anyone find a needle in this haystack? Answer: Pointers to documents are stored in databases in the form of keywords so that specific documents can be found and retrieved by using a combination of search terms. Data Storage Background Suppose there are 20 billion (=20x10 9 ) documents and the first 100 characters of each are stored using 1 byte (=8 bits) per character => 20x10 9 x100 = 2x10 12 bytes or 2 trillion (= 2000 billion) bytes of storage are needed. A 200 GB (gigabyte) hard drive costs about $200 => need 10 drives, $2000 which is quite reasonable.

3 Reading Hard Drive Data is written in the form of concentric tracks onto the platters in the hard drive. Each track can hold kbytes (kilo-bytes) and the platters rotate at approx 7200 rpm (=7200/60=120 rev/sec). Assuming 167 kbytes/track, the read rate is 120x167x10 3 =20 MB/s (megabytes/sec). But 200GB/20MB/s = 2x10 11 /2x10 7 = 10 4 sec. Note that sec = 2.78 hours for 200 GB. Direct search/read from hard drives not feasible! Database Organization Desirable properties of a database: Answer queries fast with options to narrow or broaden search results. Easy to update and maintain without introducing inconsistencies. Flexible and easy to expand as data and users evolve. Scalable as size increases (by orders of magnitude, e.g., from millions to billions).

4 Card File of Mini Zoo Name: Habitat: Colors: Speed: Caretaker: (6-8 am, 3-6 pm) Name: Habitat:, Colors: Speed: Caretaker: (10 am - 4 pm) Name: Habitat:, Colors:,, Speed: Caretaker: (10 am - 4 pm) Name: Habitat: Colors:,, Speed: Caretaker: (6-7 am, 3-4 pm) Name: Habitat: Colors: Speed: Caretaker: (10 am - 4 pm) Name: Habitat: Colors:, Red Speed: Caretaker: (6-8 am, 3-6 pm) Name: Habitat: Colors: Yellow Speed: Caretaker: (6-7 am, 3-4 pm) Suppose changes his schedule. Then we need to go through all cards and enter the change. Database Queries We could ask some of the following questions: How many animals are fast? Which animals live in water and are grey? How many animals have as caretaker? Which animals are either green or fast?

5 Card File --> Flat File Suppose we convert the card file to an electronic database on a computer. A straightforward solution is to enter each card into a table that has as many columns as there are features to record. For example, since some animals have 3 colors, we need 3 columns, labeled color1, color2, and color3. Flat File Database # Name Habitat1 Habitat2 Color1 Color2 Color3 Speed Caretaker Yellow Red (6-8 am, 3-6 pm) (10 am - 4 pm) (10 am - 4 pm) (10 am - 4 pm) (6-7 am, 3-4 pm) (6-8 am, 3-6 pm) (6-7 am, 3-4 pm) Fixed number of columns. Wastes space if most animals have one or two colors. Suppose an exotic bird has four colors, or some animals have two caretakers that take turns. This means that the whole database needs to be enlarged.

6 Flat File Database Now we can search electronically, but we still need to look at every record to find, for example, which animals are black. Also, if suddenly we have two caretakers that take turns in caring for some of the animals, we need to add a column to each record in the database. Hierarchical Database It would be nice to be able to find a record quickly by successive refining of the search query. For example, we could ask for all animals that live on land. And then we could ask for all animals that live on land and are fast.

7 Hierarchical Database DBMS 6-7 am 3-4 pm Yellow 6-7 am 3-4 pm Red 6-8 am 3-6 pm 6-8 am 3-6 pm Hierarchical Database But there is still a lot of redundancy. For example, duck is listed twice. What if we just ask How many animals are fast?

8 Network Database (6-7, 3-4) (10-4) (6-8, 3-6) Red Yellow Network Database Has very little redundancy. We can easily ask How many animals does take care of? Gets very complex quickly.

9 Relational Database Speed Table Name Speed Caretaker Table Name Caretaker Habitat Table Name Habitat Color Table Name Color Caretaker Schedule Table Caretaker Schedule Red Yellow 6-7 am 3-4 pm 10 am - 4 pm 6-8 am 3-6 pm Relational Database All information in the data base is represented as values in tables. Each property (e.g., color or caretaker) appears once and only once as a column in a table. Each datum in the database is accessible by a combination of table name, column name, and primary key (e.g., fish, duck) value.

10 After Sorting wrt Column 2 Speed Table Name Speed Caretaker Table Name Caretaker Habitat Table Name Habitat Color Table Name Color Caretaker Schedule Table Caretaker Schedule 6-7 am 6-8 am 10 am - 4 pm 3-4 pm 3-6 pm Red Yellow = {, } = {,, } = {,,,, } AND = {} AND = {, } OR = {,,,,, } Sets, Boolean Operators Set A Set B A AND B A OR B Note: A OR B = A + B -(A AND B)

11 Sets, Boolean Operators Set A NOT A A AND NOT A A OR NOT A Empty set (nothing) Everything (whole database) Truth Tables 0: false, 1: true

Worksheet - Storing Data

Unit 1 Lesson 12 Name(s) Period Date Worksheet - Storing Data At the smallest scale in the computer, information is stored as bits and bytes. In this section, we'll look at how that works. Bit Bit, like