Topics in Computer System Performance and Reliability: Storage Systems!

Transcription

1 CSC 2233: Topics in Computer System Performance and Reliability: Storage Systems! Note: some of the slides in today s lecture are borrowed from a course taught by Greg Ganger and Garth Gibson at Carnegie Mellon University 1

2 Who am I? 2

3 What makes storage systems so cool? 1. Combines so many topic areas: hardware meets OS meets networking meets distributed systems meets security meets AI meets HCI 3

4 What makes storage systems so cool? 1. Combines so many topic areas 2. This is where great jobs are! Designers and implementers still needed not just testing J Continuing growth area for the future The Internet is a network, but the web is a storage system Strong existing companies: EMC, NetApp, Core competency for Internet services: Google, Microsoft, Amazon, and still support for start-ups 4

5 What makes storage systems so cool? 1. Combines so many topic areas 2. Great careers 3. Still so much room to contribute: performance actually matters here in fact, it dominates other parts of system performance in many cases and reliability too storage management wide open and, storage starting to take over computation 5

6 Amdahl s Law Speedup limited to fraction improved obvious, but fundamental, observation % reduction in BLUE yields only 45% reduction in total What does this mean for storage systems? 6

7 Technology Trends Normalized value relative to CPU Performance Network Bandwidth Memory Bandwidth Disk Bandwidth Network Latency Disk Latency Year 7

8 Consequence: storage performance dominates Assume 50 seconds CPU & 50 seconds I/O CPU improves by 2X every 2 years 8

9 Consequence: storage performance dominates Assume 50 seconds CPU & 50 seconds I/O CPU improves by 2X every 2 years 9

10 Storage systems: fun quotes I/O certainly has been lagging in the last decade Seymour Cray, 1976 Also, I/O needs a lot of work David Kuck, 1988 In 3 to 5 years, we will start seeing servers as peripherals to storage SUN Chief Technology Officer, 1998 Scalable I/O is perhaps the most overlooked area of high-performance computing R&D Suggested R&D topic report for

11 Remainder of the course Devices: Hard disk drives, solid state drives Local file systems File system organizations File system integrity/consistency NVM file systems Distributed file systems Parallel file systems Extremely scalable storage (Google & Co) Reliability & fault tolerance 11

12 Logistics & Administratives Class time: Wed 10am 12pm Office hours: By appointment Class web page Still undergoing updates 11 weeks of lectures Course project due end of the semester 12

13 Grading 30% class participation Participation in class discussions (Read all papers prior to class) Class presentation of research paper Possibly quizzes (10%) 70% class project No exams, no homework, no paper summaries 13

14 Class project Can be done in team of two or alone Start looking for a partner now! We will suggest possible projects (see course web page) Output: workshop quality research paper (10-12 pages) Even better: conference quality paper Use latex template on course web page All reports will be published as tech-report We will help you get there --- multiple milestones: And meetings with TA/instructor 14

15 Paper presentation Each of you will present at least one paper in class Format of the presentation: 25 min presentation of paper contents 5-15 min paper review Good points Bad points 10 min class discussion that you lead! Prepare questions! Keep the class engaged! 15

16 Paper presentation What I do not want: A long laundry list of all things the paper did What I do want: A lecture style presentation of the paper Including background material your fellow class mates might need to understand the paper A critical discussion of the paper Strength & Weaknesses Prepare questions! What you get: Feedback! 16

17 Purpose of presentation Wrong answers: To give a verbal version of the paper, cramming all its content into 30 min To impress people with your technical depth and thoroughness In fact, no one cares about these things The goal is to filter out the main points of the paper and present them well By the end, everybody in the audience should remember 2-3 takehome messages 17

18 What s on each slide? Control level of detail Each slide should have one basic point There should NOT be tons of text Use sentence fragments Use pictures everywhere you possibly can! A picture says more than 1000 words Saves text and thus slides Much easier to process 18

19 Rest of today: Some review 19

20 What are storage systems all about? Memory/storage hierarchy 20

21 Memory/storage hierarchies Balancing performance with cost Small memories are fast but expensive Large memories are slow but cheap Exploit locality to get the best of both worlds locality = re-use/nearness of accesses allows most accesses to use small, fast memory Capacity L1/2 CACHE L3 CACHE DRAM SSD Performance HARD DISK 21

22 Example memory hierarchy values Notice the huge access time gap between DRAM and disk SSDs (tens of microsecs) 22

23 What are storage systems all about? Memory/storage hierarchy Combining many technologies to balance costs/benefits For long time not the focal point of storage system design More interesting in recent years with SSDs and NVMs arriving on the market 23

24 What are storage systems all about? Memory/storage hierarchy Combining many technologies to balance costs/benefits For long time not the focal point of storage system design More interesting in recent years with SSDs and NVMs arriving on the market Persistence Storing data for lengthy periods of time To be useful, it must also be possible to find it again later this brings in data organization, consistency, and management issues This is where the serious action is and it does relate to the memory/storage hierarchy 24

25 Why persistence is important Some statistics: Among companies who lose data in a disaster, 50% never re-open and 90% are out of business within two years Even smaller incidents can be costly Reproducing some tens of megabytes of accounting data can take several weeks and cost tens of thousands of dollars Bad PR! 25

26 What is a storage system: Big Picture Application Bob1Bob2Bob3Bob4Bob2Bob3Bob4 Bob2 Bob3Bob4Bob4 The storage system Application gives keeps the data objects data objects & their and returns one upon IDs to storage request (by ID) Storage System Bob1 Bob2 Bob3 Bob4 26

27 Storage Systems & Interfaces What is a Storage System? Hardware (devices, controllers, interconnect) and Software (file system, device drivers, firmware) dedicated to providing management of and access to persistent storage. One view: defined by collection of interfaces 27

28 Storage Software Interfaces Program File system Device driver I/O controller Physical Media High level of abstraction No abstraction Understands files and directories HDD understands platters, cylinders, tracks, sectors 28

29 What s inside a disk? 29

30 Disk structure top view of single platter Surface organized into tracks Tracks organized into sectors 30

31 Disk service time components After BLUE read Seek for RED Rotational latency After RED read Components: Seek Rotational latency Data transfer 31

32 Seek time Time required to move head over desired track A real seek profile: Note that this is not linear! 32

33 Seek time Seek times not linear because they have up to four components: Accelerate Coast at max velocity If going far enough to reach max velocity Decelerate Settle onto correct trace Takes extra time to settle before writing 33

34 What is the average seek time? Watch out for misrepresentations What it is not: Seek time for average of possible distances Seek time for any LBN to any other What it is: Depends on workload Very different for sequential versus random workloads 34

35 Where does the disk head s time go? Seek time, rotational latency, transfer time? Random 4KB requests 35

36 Impact of request sizes? Seek time, rotational latency, transfer time? 36

37 Impact of locality? Seek time, rotational latency, transfer time? 37

38 Where does the disk head s time go? Seek time: 1 6ms, depending on distance Improving at 7-10% per year Rotation speeds: 7,200-15,000 RPM Average latency of 2-4ms Improving at 7-10% per year Data rates: MB/s Average sector transfer time of 25us Improving at 30-40% per year 38

39 What s inside a disk? The mechanics: What are all those needed for? The electronics (just like a small computer): A processor DRAM Control ASIC 39

40 Disk drive what s in a sector? Data Typically 512 bytes Sync bytes = pattern to notify controller that data follows Header (ID information) Cylinder, head, and sector number ECC (error correcting codes) At such high densities, problems occur ECC detects and corrects on the fly Tri-state guarantee of sector writes All written All not written Sector destroyed NEVER: partially modified Servo = bit pattern used for centering on track 40

41 How is functionality implemented? Some in ASIC logic: Error detection and correction Servo processing Motor-seek control Some in firmware running on control processor 41

42 How is functionality implemented? Some in ASIC logic: Error detection and correction Servo processing Motor-seek control Some in firmware running on control processor Request processing, queueing, scheduling LBN to PBN mapping 42

43 How to map LBN to PBN The view of the OS The reality: 43

44 LBN to physical mapping for single surface 44

45 Extending the mapping to a multi-surface disk 45

46 First complication: zones Real disks don t have constant number of sectors per track 46

47 Multiple zones 47

48 Computing physical location from LBN An example zone breakdown.. First, figure out which zone contains the LBN i.e. which cylinder Then determine surface number Then determine sector number 48

49 Second complication: defect management Disks keep spare sectors Those are used in case portions of the media become unusable (both before and after installation) 49

50 Second complication: defect management First approach: remap broken sector, don t touch anything else 50

51 Second complication: defect management Second approach, slip mapping past broken sector 51

52 Third complication: skew 52

53 Third complication: skew 53

54 Third complication: skew It takes time to switch from one track to another Sequential transfers suffer full rotation 54

55 Same request with track skew of one sector Track skew prevents unnecessary rotation 55

56 Same request with track skew of one sector Track skew prevents unnecessary rotation 56

57 How is functionality implemented? Some in ASIC logic: Error detection and correction Servo processing Motor-seek control Some in firmware running on control processor Request processing, queueing, scheduling LBN to PBN mapping Zones Defects Skew 57