Sep. 22 nd, 2008 Sang-Won Lee. Toward Flash-based Enterprise Databases

Transcription

1 Towards Flash-based Enterprise Databases Sep. 22 nd, 2008 Sang-Won Lee Sungkyunkwan University, Korea 1 SKKU VLDB Lab.

2 SKKU VLDB Lab. Research Directions Vision: Flash is disk, disk is tape, and tape is dead. (by Jim Gray) Strategy: Do only what only you can do. (by E. W. Dijkstra) Research Goal: Database systems on flash memory pioneer Recent Achievements FAST: One of the most efficient FTL mechanisms (ACM Transactions on Embedded Computing Systems, 2007) IPL: An innovative database storage scheme on flash memory (ACM SIGMOD 2007) A Case for Flash Memory SSD in Enterprise Database Applications (ACM SIGMOD 2008) 2 SKKU VLDB Lab.

3 Flash-based Database Research: Motivations 3 SKKU VLDB Lab.

4 Real Motivations SCI, SCI, SCI, 4 SKKU VLDB Lab.

5 The VERY Real Motivations Rule of the game in storage market is changing Hardware: Dr. Masuoka, Samsung, Toshiba,.. Hwang s Law vs. Moore s Law SW: OS, File System, FTL, DBMS, 5 SKKU VLDB Lab.

6 Flash Memory Market Mobile phone, digital camera, ipod, MP3 etc ( 1 ~ 10 G) Flash already wins over harddisk PC, Laptop : over 10G 2008: the war begins Enterprise server storage: 100G ~ 100T 2009 ~ 2010? In my opinion, flash is more attractive in enterprise rather than PC / Labtop market 6 SKKU VLDB Lab.

7 Flash Memory Solid State Disk (SSD) Provides a block-device interface identical to a hard disk, but uses flash memory as a storage medium 50 years champion VS A new challenger! Identical Interface 7 SKKU VLDB Lab.

8 A Quick Intro to Flash Memory

9 Flash Organization 9 SKKU VLDB Lab.

10 Flash Performance: Chip Level Flash characteristics Erase-before-write ( No in-place update) Asymmetric read/write speed No mechanical latency 10 SKKU VLDB Lab.

11 Sequential Append-only Writes vs. Random Overwrites E.g. 2K page write: new write vs. overwrite in a naïve way New write: 0.2ms Data Blocks: block mapping Free Blocks Overwrite 63 2K-reads = 6.3ms 63 2K-writes = 12.6ms 1 2K-write = 0.2ms 1 erase = 1.5ms Total = 20.6ms 11 SKKU VLDB Lab.

12 Flash Performance: SSD Level (Source: Jim Gray s slide) 12 SKKU VLDB Lab.

13 NAND FlashSSD Architecture 13 SKKU VLDB Lab.

14 Flash Translation Layer ( FTL ) A software layer that makes flash memory appear to the system like a disk drive Address mapping: logical sector <-> physical sector Garbage collection Power-off off recovery Wear leveling and bad block management. 14 SKKU VLDB Lab.

15 Flash SSD Block Diagram e.g. Samsung SSD somewhat old model Pata/ Sata/ Scsi/. More Powerful CPU, Bigger SDRAM Parallelism / Interleaving for Large Bandwidth 15 SKKU VLDB Lab.

16 Opportunities in Flash-based DBMS Research Flash-aware Applications Flash-aware DBMS (Storage, Indexing, Buffer Mgmt., Query Processing ) Flash-aware OSs/File Systems/Device Drivers F T L Flash Chips (SLC, MLC) 16 SKKU VLDB Lab.

17 FAST: A Log Buffer based Flash Translation Layer using Fully Associative Sector Translation Sang-Won Lee, et al. ACM Transactions on Embedded d Computing Systems, July 2007

18 Seoul National University FTL: Hybrid Mapping 18 SKKU VLDB Lab.

19 Seoul National University FTL: Hybrid Mapping Data Blocks Free Blocks Every 2K writes can be buffered in 0.2 ms, not in 20.6 ms When a log block is full, it should be merged Merge operation Log Buffer Blocks: Write buffers based on Locality and Log Structured File System idea 19 SKKU VLDB Lab.

20 Associativity in CPU Cache Directed mapping vs. Fully associative mapping Cache Block Block No. No. Mapping = (Block No.) mod (# of Blocks in Cache) A Block can be placed anywhere in the cache Memory Block No (a) Associativity between memory and CPU cache 20 SKKU VLDB Lab.

21 SNU FTL = BAST BAST: Block Associative Sector Translation Flash Memory Original Data Blocks Log Blocks = Cache for Writes (b) Log blocks: a cache for write operations Sectors from one logical block can be mapped only to one log block Flash Memory Original Data Blocks Log Blocks (c) Block associativity in log block scheme 21 SKKU VLDB Lab.

22 Disadvantages of BAST What if there are many hot blocks? 1. Choose one victim log blocks Data Blocks 2. Merge the victim block 3. Assign a new log block for the new hot block Free Blocks Log Buffer Blocks: Write buffers 22 SKKU VLDB Lab.

23 Disadvantages of BAST(2) Low space utilization of victim blocks Log block thrashing Lo og block ut tilization(% ) # of Log blocks A B C 23 SKKU VLDB Lab.

24 FAST What if we take Fully Associative Sector Translation in log buffer blocks? Sectors from one logical block can be mapped to any log block Flash Memory Original Data Blocks Log Blocks (d) Fully associativity b/w logical sectors and log blocks 24 SKKU VLDB Lab.

25 Sector Writes and Reads in FAST Flash Memory Sectors from one logical block can be mapped to any log block Original Data Blocks Log Blocks Block Mapping Table Sector Mapping Table Sector (over-)write Sector read Append the write sector in log blocks If the log blocks are full, 1. Merge a log block in FIFO manner 2. Get a free block and make it a new log block Search the sector mapping table for the most recent sector If not found, visit the original data blocks via block mapping table 25 SKKU VLDB Lab.

26 Performance Evaluation e Count Eras bast fast o-fast Digital Camera # of Log blocks Eras se Count bast fast LINUX o-fast # of Log blocks 26 SKKU VLDB Lab.

27 Performance Evaluation(2) Era ase Count bast fast o-fast Symbian # of Log blocks e Count Eras bast fast o-fast RANDOM # of Log blocks 27 SKKU VLDB Lab.

28 Design of Flash-Based DBMS: An In-Page Logging Approach SIGMOD 2007 Sang-Won Lee, Bongki Moon (Univ. of Arizona)

29 Traditional DBMS on Flash Memory: Problem (FTL) + pure disk-based DBMS traditional in-place update Because of no update-in-place characteristics, most block write results in a copy-back operation (> 20ms) SQL: Update / Insert / Delete Update Buffer Mgr. Page : 4KB Block : 128KB Dirty Block Write Flash Memory Data Block Area 29 SKKU VLDB Lab.

30 In-page Logging: Basic Ideas Traditional In-Place Update Log-Structured Approach In-Page Logging Approach * No in-place update * No mechanical latency * Fast Read Speed 30 SKKU VLDB Lab.

31 In-page Logging: Basic Ideas(2) Database update and log in traditional DBMS architecture SQL Buffer Log Buffer pi pj DB pi LOG pj 31 SKKU VLDB Lab.

32 In-page Logging: Basic Ideas(3) Write performance: 100x faster in best case Update / Insert / Delete update-in-place Sector : 512B Buffer Mgr. physio-logical log Page : 4KB Block : 128KB 1 sector write = 200 us Flash Memory Data Block Area 32 SKKU VLDB Lab.

33 In-page Logging: Basic Ideas(4) Read When a page Pi is read in after replace-out Buffer Mgr. Pi Re-construct most up-to-date in-memory copy Re-apply the physiological action against the original copy (CPU overhead!) Read Original copy of Pi All log sectors in the flash block (IO overhead) Flash Memory 15 pages: log area (8K): 16 sectors Criticisms: 2x slow read speed It still may be fast enough Locality of updates 33 SKKU VLDB Lab.

34 In-page Logging: Basic Ideas(5) Merge operation: new internal operation in IPL A Physical Flash Block Merge up-to-date 15 data pages log area (8K): 16 sectors clean log area B old B new Merge policies: eager vs. lazy merge Refer to paper 34 SKKU VLDB Lab.

35 IPL Simulation with TPC-C C Performance trend with varying buffer sizes (1G.100u, 8KB log area) 35 SKKU VLDB Lab.

36 In-page Logging: The Unexplored Beauty Traditional In-Place Update Log-Structured Approach In-Page Logging Approach * No in-place update * No mechanical latency * Fast Read Speed * Redo-based Log New Recovery & CC Approach 36 SKKU VLDB Lab.

37 A Case for Flash Memory SSD in Enterprise Database Applications SIGMOD 2008 Sang-Won Lee, Bongki Moon, Chanik Park, Jae-Myung Kim, Sang-Woo Kim 37 SKKU VLDB Lab.

38 Major I/Os in Database SQL Buffer Cache Log Buffer Data File Temporary Tablespace Undo(Rollback) Data Redo Log Access pattern: random Next challenge Access pattern: Append-only sequential writes, then random reads 38 SKKU VLDB Lab.

39 Redo Log 39 SKKU VLDB Lab.

40 Transaction Concept and Log begin_transaction ti select statements; // read Buffer SQL Log Buffer insert/update/deletes; // update pi commit or rollback; DB LOG 40 SKKU VLDB Lab.

41 Transaction Concept and Log (2) Tx1 Tx2 Tx_N SQL Redo Log Buffer Buffer Log Buffer Commit pi DB LOG Redo Log File 512 Byte Block (include wastage) 41 SKKU VLDB Lab.

42 Transaction Commit: Final Performance Bottleneck Transaction response time = read / write + CPU + commit Negligible read / write time Large buffer cache or main memory DBMS CPU time (dual core Intel CPU) TPC-B: 50 us TPC-C: ~ 200 us Commit time = 8 ~ 10ms Final performance bottleneck 42 SKKU VLDB Lab.

43 TPS vs. CPU TPS in TPC-B: HDD vs. SSD CPU saturation knee point 80% 70% % TPS % 40% 30% CPU% % % 0 vu1 vu5 vu10 vu15 vu20 vu25 vu30 vu35 vu40 vu45 vu50 S-SSD(CPU%) HDD(CPU%) S-SSD(TPS) HDD(TPS) 0% 43 SKKU VLDB Lab.

44 TPS in TPC-B: HDD vs. SSD CPU-bound vs. IO-bound: 7200 vs RPM SATA TPS vu1 vu5 vu10 vu15 vu20 vu25 vu30 vu35 vu40 vu45 vu50 vu55 vu60 S SSD(Quad TPS) 10k SATA(Quad TPS) 7200 SATA(Quad TPS) S SSD(Dual TPS) 10k SATA(Dual TPS) 7200 SATA(Dual TPS) 44 SKKU VLDB Lab.

45 Flash as Log Device Communications of the ACM, July 2008 Flash memory storage Adam Sun 45 SKKU VLDB Lab.

46 Rollback Segment 46 SKKU VLDB Lab.

47 Undo Data in Oracle Undo tablespace and rollback segments Tx1 Tx2 Tx_N Buffer Cache pi Log Buffer Data File Temporary Undo(Rollback) Redo Log Tablespace Data 47 SKKU VLDB Lab.

48 Rollback Segment t2 C B 100 t1 t0 A 50 A 100 A 200 Rollback segment Stores undo(old) information Transaction rollback; MVCC Each transaction is assigned to a rollback segment in round-robin Rollback segment disk Each rollback segment is written in (almost) append-only mode Asynch mode by DBWR For read consistency, the rollback segments is randomly accessed 48 SKKU VLDB Lab.

49 Undo Area I/O Pattern (Write) During TPC-C C run (Semi-)Sequential write, then random read 800 Logical sector addres ss (x1000) Time (second) 49 SKKU VLDB Lab.

50 Undo Area I/O Pattern (Read) Full scan of a table while running TPC-C C 50 SKKU VLDB Lab.

51 Temporary Tablespace 51 SKKU VLDB Lab.

52 Temporary Tablespace Index creation Temporary table Sort / Sort merge join Hash join 52 SKKU VLDB Lab.

53 External Merge Sort 1 st phase: run generation 200M Source Data Disk 2 nd phase: merge 2M (in-memory sort) Main memory buffers Disk 100 runs INPUT 1 INPUT 2... OUTPUT.. INPUT 100. Disk Main memory buffers Disk SKKU VLDB Lab.

54 IO Patterns in External Merge Sort Sequential write, then random read 54 SKKU VLDB Lab.

55 External Merge Sort Experiment set up 200M data 100B record 10B sort key In-memory sort area: 2M 55 SKKU VLDB Lab.

56 External Merge Sort Cluster Unit of IO in merge step Disk SSD Change the cluster size 2K ~ 128K Optimal cluster size 64K (hdd) vs. 2K (sdd) Execution Time (sec) Cluster Size in Merge Step (KB) Implications 56 SKKU VLDB Lab.

57 Hash Join Original Relation OUTPUT 1 Partitions... INPUT 2 hash function h B B-1 Disk B main memory buffers Disk 1. Partitioning (or building) phase Partitions of R & S hash fn h2 Hash table for partition Ri (k < B-1 pages) Join Result h2 Disk Input buffer for Si Output buffer B main memory buffers 2. Probing (or matching) phase Disk 57 SKKU VLDB Lab.

58 IO Patterns in Hash Join 58 SKKU VLDB Lab.

59 FlashSSDs in Enterprise Databases 59 SKKU VLDB Lab.

60 Next Challenges in Database SQL Buffer Cache Log Buffer Data File Temporary Tablespace Undo(Rollback) Data Redo Log Access pattern: random writes 60 SKKU VLDB Lab.

61 Flash Perspectives from DB Community There has been discussion of sea-changes in DBMS design arising from new storage technologies replacing disk. Flash memory appears to be both technologically viable and economically supported by a broad market. Flash is the first new persistent storage medium to succeed in this regard in more than three decades. [Hellerstein, Stonebraker and Hamilton (2007)]. Flash is disk, disk is tape, and tape is dead. 61 SKKU VLDB Lab.

62 Some DB Community Papers DAMON 2008: from CPU to Storage in research Modeling the Performance of Algorithms on Flash Memory Devices Ross, Kenneth Fast Scans and Joins using Flash Drives Shah, Mehul A.; Stavros Harizopoulos; Janet L. Wiener; Goetz Graefe VLDB 2008 Flashing Up The Storage Layer, Ioannis Koltsidas (University of Edinburgh), Stratis Viglas (University of Edinburgh) Online Maintenance of Very Large Random Samples on Flash Storage Suman Nath (Microsoft Research), Phillip Gibbons (Intel Research, Pittsburgh, USA) 62 SKKU VLDB Lab.

63 Flash-based DBMS: Research Groups Ken Columbia Sam MIT Graefe HP HKUST INRIA: PicoDBMS team PBFilter: Indexing Flash-Resident Data through Partitioned Summaries Minesota Microsoft / Edinburg university (VLDB 2008) 63 SKKU VLDB Lab.

64 If you want truly to understand d something, try to change it Kurt Lewin 64 SKKU VLDB Lab.