CSCI-GA Database Systems Lecture 8: Physical Schema: Storage

Transcription

1 CSCI-GA Database Systems Lecture 8: Physical Schema: Storage Mohamed Zahran (aka Z)

2 View 1 View 2 View 3 Conceptual Schema Physical Schema 1. Create a model of the enterprise (e.g., using ER) 2. Create a logical implementation (using a relational model and normalization) Disk What happens under the hood?

3 Requirements Analysis Conceptual Design Uses a file system to store the relations Requires knowledge of hardware and operating systems characteristics Logical Design Schema Refinement Application & Security Design Physical Design

4 First, Let s Look at a Typical Hierarchy Level Access time Typical size Registers "instantaneous" under 1KB Level 1 Cache 1-3 ns 64KB per core Level 2 Cache 3-10 ns 256KB per core Level 3 Cache ns 2-20 MB per chip Main Memory ns 4-32 GB per system Hard Disk 3,000,000-10,000,000 ns over 1TB In DB, we care about those two levels

5 File Structures Indexes Physical Design Criteria Storage media Performance Hard-disk drives Solid-state Disks

6 Disk Drives Platters Tracks Platter Sectors Track To access data: seek time: position head over the proper track rotational latency: wait for desired sector (RPM) transfer time: grab the data (one or more sectors)

7 Hard Disks spinning platter of special material mechanical arm with read/write head must be close to the platter to read/write data data is stored magnetically disks are random access meaning data can be read/written anywhere on the disk

8 A Conventional Hard Disk Structure

9

10 Hard Disk Architecture Surface = group of tracks Track = group of sectors Sector = group of bytes Cylinder: several tracks on corresponding surfaces

11 Disk Sectors and Access Each sector records Sector ID Data (512 bytes, 4096 bytes proposed) Error correcting code (ECC) Used to hide defects and recording errors Synchronization fields and gaps Access to a sector involves Queuing delay if other accesses are pending Seek: move the heads Rotational latency Data transfer Controller overhead

12 Example of a Real Disk: MKxx59GSM from Toshiba Size: 1TB 5,400 RPM 2.5 Number of platters: 3 Number of data heads: 6 Interface: SATA Transfer rate to host: 3GB/sec Average seek time: 12ms Track-to-Track seek: 2ms

13 Disks: Other Issues Average seek and rotation times are helped by locality. Disk performance improves about 10%/year Capacity increases about 60%/year Common disk interfaces/controllers: SCSI, IDE, SATA

14 The Disk: A View from the Top A sequence of blocks A physical unit of access is always a block All blocks are of the same size Actually, a block consists of 1 or more sectors A file: Logically: a series of records, of similar or different sizes Physically: a series of, no necessarily contiguous blocks The file system helps us to: Find the first block Find the last block Find the next block Find the previous block

15 Relation File Each tuple is a record in the file. Logical view Records There is one extra layer that we will discuss shortly! Physical view Assumptions: There can be several records in a block No record spans more than one block Blocks Sectors

16 Example Relation E# Salary Records Records First block of the file Blocks Left-over Space

17 Processing a Query: What Happens Under The Hood? SELECT E# FROM R WHERE SALARY > 1500; Read from disk into RAM all relevant blocks Get the relevant information from blocks What is the cost of this? Additional processing then produce the answer

18 A Simple Cost Model Assumptions Reading or Writing a block costs one time unit Processing is free Justification Accessing the disk is much more expensive than any reasonable CPU processing of queries Implication Goal: minimize number of block accesses Good Heuristic: Organize the physical database so that you make as much use as possible from any block you read/write

19 Why Not Store Everything in Main Memory?

20 Example What is the cost of accessing: E# 2 and E# 9? What is the cost of accessing: E# 2 and E# 4? Array in RAM Blocks on disk

21 What Is The Best Place for the Next Block?

22 RAID Disk Array: Arrangement of several disks that gives abstraction of a single, large disk. Goals: Increase performance and reliability. Two main techniques: Data striping: Data is partitioned; size of a partition is called the striping unit. Partitions are distributed over several disks. Redundancy: More disks => more failures. Redundant information allows reconstruction of data if a disk fails.

23 RAID Levels Level 0: No redundancy Level 1: Mirrored (two identical copies) Each disk has a mirror image Parallel reads, a write involves two disks. Maximum transfer rate = transfer rate of one disk Better reliability but no protection against data corruption of viruses

24 RAID Levels Level 0+1: Striping and Mirroring Parallel reads, a write involves two disks. Maximum transfer rate = aggregate bandwidth Level 1+0 is the same as 0+1 with reverse in mirroring and stripping.

25 RAID Levels Level 2: Bit-Interleaved Uses hamming code for error correction Can recover data from single-bit corruption Out of fashion! For Error correction (Hamming code)

26 RAID Levels Level 3: Byte-Interleaved with Parity Striping Unit: One byte. One check disk. Each read and write request involves all disks; disk array can process one request at a time.

27 RAID Levels Level 4: Block-Interleaved Parity Striping Unit: One disk block. One check disk. Parallel reads possible for small requests, large requests can utilize full bandwidth Writes involve modified block and check disk

28 RAID Levels Level 5: Block-Interleaved Distributed Parity Similar to RAID Level 4, but parity blocks are distributed over all disks Level 6 is similar to 5 but with extra parity block

29 What Is This Story of Solid State Disks (SSD)? No moving parts hence called solid state reads and writes to a medium called NAND flash memory Faster startup: no spinning Extremely low read latency Deterministic: performance does not depend on the location of the data

30 BUT Much more expensive than hard disks (~3$/GB vs. 0.15$/GB) Limited write erase time Slower write speeds High capacity SSDs may have significant higher power requirements SSD can get slower as it ages

31 SSD Organization SSD has its own page SSD pages are getting larger: 8KB, 16KB, 32KB Sector addressable Most SSD uses 4KB sectors

32 Comparison of typical hard-drives and SSD More up-to-date- SSD: Samsung SSD 840 Pro Series: 256 GB Read XFR Rate: ~510 MB/s Write XFR Rate: ~500 MB/s

33 Query Optimization and Execution Relational Operators Files and Access Methods Buffer Management Disk Space Management DB Application DBMS OS DISK Controller Important: Tow flavors of DBMS Relying on OS file systems Uses its own or extend OS Data must be in RAM for DBMS to operate on it!

34 Disk Space Manager Abstractly: deals with pages as a unit of data (read, write, allocate, deallocate) Page size usually chosen as disk block Keeps track of: Which blocks are in use List of bitmap Which pages are on which page blocks Hides the details of the hardware and OS and makes higher level layers think in term of pages

35 Buffer Manager Memory may not be able to hold all needed pages the software layer responsible for bringing pages from disk to main memory as needed Implements replacement policy Higher levels of the DBMS code can be written without worrying about whether data pages are in memory or not.

36 Buffer Manager It does its job as follows: Partitions the memory into frames (1 frame holds 1 page) The collection of all frames/pages is called buffer pool Page Requests from Higher Levels BUFFER POOL disk page free frame MAIN MEMORY DISK DB choice of frame dictated by replacement policy

37 Hierarchy of Data Tuples/Records Relations Files Pages Blocks Sectors

38 We can try to minimize the number of block accesses for frequent queries. Indexing Oversimplification: Tries to provide where blocks containing useful records are File Organization Oversimplification: Tries to provide When you read a block you get many useful records

39 Conclusions From the application to the disk, each layer sees the data differently Disks provide cheap, non-volatile storage. Performance and cost are the main issues