CIS Operating Systems I/O Systems & Secondary Storage. Professor Qiang Zeng Spring 2018

Similar documents
CIS Operating Systems I/O Systems & Secondary Storage. Professor Qiang Zeng Fall 2017

I/O Handling. ECE 650 Systems Programming & Engineering Duke University, Spring Based on Operating Systems Concepts, Silberschatz Chapter 13

Operating Systems. V. Input / Output

I/O Systems. Amir H. Payberah. Amirkabir University of Technology (Tehran Polytechnic)

Device-Functionality Progression

Chapter 12: I/O Systems. I/O Hardware

Chapter 13: I/O Systems. Operating System Concepts 9 th Edition

Module 12: I/O Systems

Chapter 13: I/O Systems

Chapter 10: Mass-Storage Systems

Chapter-6. SUBJECT:- Operating System TOPICS:- I/O Management. Created by : - Sanjay Patel

by I.-C. Lin, Dept. CS, NCTU. Textbook: Operating System Concepts 8ed CHAPTER 13: I/O SYSTEMS

Ref: Chap 12. Secondary Storage and I/O Systems. Applied Operating System Concepts 12.1

Chapter 12: Mass-Storage Systems. Operating System Concepts 8 th Edition,

Input/Output Systems

CIS Operating Systems File Systems. Professor Qiang Zeng Spring 2018

Chapter 12: I/O Systems

Chapter 13: I/O Systems

Chapter 12: I/O Systems. Operating System Concepts Essentials 8 th Edition

Input/Output. Today. Next. Principles of I/O hardware & software I/O software layers Disks. Protection & Security

Operating System: Chap13 I/O Systems. National Tsing-Hua University 2016, Fall Semester

CIS Operating Systems File Systems. Professor Qiang Zeng Fall 2017

Chapter 13: I/O Systems

Chapter 13: I/O Systems

CSE 120. Overview. July 27, Day 8 Input/Output. Instructor: Neil Rhodes. Hardware. Hardware. Hardware

Chapter 13: I/O Systems

Module 12: I/O Systems

Lecture 13 Input/Output (I/O) Systems (chapter 13)

Chapter 13: I/O Systems

Chapter 13: I/O Systems. Chapter 13: I/O Systems. Objectives. I/O Hardware. A Typical PC Bus Structure. Device I/O Port Locations on PCs (partial)

Chapter 13: I/O Systems

CS330: Operating System and Lab. (Spring 2006) I/O Systems

Disk Scheduling. Chapter 14 Based on the slides supporting the text and B.Ramamurthy s slides from Spring 2001

Lecture 15: I/O Devices & Drivers

Input-Output (I/O) Input - Output. I/O Devices. I/O Devices. I/O Devices. I/O Devices. operating system must control all I/O devices.

Chapter 12: Secondary-Storage Structure. Operating System Concepts 8 th Edition,

Chapter 13: I/O Systems

Disk Scheduling. Based on the slides supporting the text

V. Mass Storage Systems

Outline. Operating Systems: Devices and I/O p. 1/18

Today: I/O Systems. Architecture of I/O Systems

CSE 153 Design of Operating Systems

File. File System Implementation. Operations. Permissions and Data Layout. Storing and Accessing File Data. Opening a File

CSE 153 Design of Operating Systems Fall 2018

CSE 4/521 Introduction to Operating Systems. Lecture 24 I/O Systems (Overview, Application I/O Interface, Kernel I/O Subsystem) Summer 2018

I/O Device Controllers. I/O Systems. I/O Ports & Memory-Mapped I/O. Direct Memory Access (DMA) Operating Systems 10/20/2010. CSC 256/456 Fall

I/O SYSTEMS. Sunu Wibirama

Free Space Management

I/O Management and Disk Scheduling. Chapter 11

UNIT 4 Device Management

Introduction Disks RAID Tertiary storage. Mass Storage. CMSC 420, York College. November 21, 2006

Comp 204: Computer Systems and Their Implementation. Lecture 18: Devices

I/O Systems and Storage Devices

Input/Output. Chapter 5: I/O Systems. How fast is I/O hardware? Device controllers. Memory-mapped I/O. How is memory-mapped I/O done?

I/O Systems. Jo, Heeseung

Operating Systems 2010/2011

CS370 Operating Systems

Overview of Mass Storage Structure

Devices. Today. Comp 104: Operating Systems Concepts. Operating System An Abstract View 05/01/2017. Devices. Devices

Chapter 12: Mass-Storage

Che-Wei Chang Department of Computer Science and Information Engineering, Chang Gung University

Operating Systems. V. Input / Output. Eurecom

Chapter 13: I/O Systems

EIDE, ATA, SATA, USB,

Module 13: Secondary-Storage Structure

CS420: Operating Systems. Mass Storage Structure

Operating System 1 (ECS-501)

UC Santa Barbara. Operating Systems. Christopher Kruegel Department of Computer Science UC Santa Barbara

Module 11: I/O Systems

Input Output (IO) Management

CS370 Operating Systems

I/O Systems. 04/16/2007 CSCI 315 Operating Systems Design 1

I/O AND DEVICE HANDLING Operating Systems Design Euiseong Seo

Input/Output Management

CHAPTER 12 AND 13 - MASS-STORAGE STRUCTURE & I/O- SYSTEMS

CS3600 SYSTEMS AND NETWORKS

OPERATING SYSTEMS CS3502 Spring Input/Output System Chapter 9

I/O Systems. Jinkyu Jeong Computer Systems Laboratory Sungkyunkwan University

Hard Disk Drives (HDDs) Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University

Disc Allocation and Disc Arm Scheduling?

CSE 380 Computer Operating Systems

Hard Disk Drives (HDDs)

CS 134. Operating Systems. April 8, 2013 Lecture 20. Input/Output. Instructor: Neil Rhodes. Monday, April 7, 14

I/O, Disks, and RAID Yi Shi Fall Xi an Jiaotong University

Chapter 10: Mass-Storage Systems

Ricardo Rocha. Department of Computer Science Faculty of Sciences University of Porto

Operating Systems 2010/2011

OPERATING SYSTEMS CS136

Chapter 10: Mass-Storage Systems. Operating System Concepts 9 th Edition

Chapter 11. I/O Management and Disk Scheduling

Chapter 14: Mass-Storage Systems. Disk Structure

Chapter 13: Mass-Storage Systems. Disk Scheduling. Disk Scheduling (Cont.) Disk Structure FCFS. Moving-Head Disk Mechanism

Chapter 13: Mass-Storage Systems. Disk Structure

Lecture 9. I/O Management and Disk Scheduling Algorithms

Ricardo Rocha. Department of Computer Science Faculty of Sciences University of Porto

I/O & Storage. Jin-Soo Kim ( Computer Systems Laboratory Sungkyunkwan University

CS 471 Operating Systems. Yue Cheng. George Mason University Fall 2017

CS370 Operating Systems

Mass-Storage Systems. Mass-Storage Systems. Disk Attachment. Disk Attachment

Lecture 16: Storage Devices

Transcription:

CIS 3207 - Operating Systems I/O Systems & Secondary Storage Professor Qiang Zeng Spring 2018

Previous class Memory subsystem How to allocate physical memory? How to do address translation? How to be quick? CPU cache such as TLB Page cache How to be lazy? Copy-on-write Demand paging Memory-mapped file I/O CIS 3207 - Operating Systems 2

Outline I/O subsystem: hardware aspect Terms: controller, bus, port How are the device registers and memory addressed? I/O subsystem: software aspect Device drivers programmer s perspective: polling, interruptdriven, and DMA User-space programmer s perspective: blocking, nonblocking, & asynchronous I/O subsystem: performance aspect Vectored I/O Caching, Buffering and Spooling Some slides are courtesy of Dr. Abraham Silberschatz and Dr. Thomas Anderson CIS 3207 - Operating Systems 3

Northbridge/Southbridge architecture CIS 3207 - Operating Systems 4

Intel Ivy Bridge CIS 3207 - Operating Systems 5

Simplified architecture CIS 3207 - Operating Systems 6

Layers: User Big program-> picture libs -> core kernel -> Kernel I/O subsystem -> driver->controller->drive User programs Libraries CIS 3207 - Operating Systems 7

Controller A hardware module, inside or outside computer, that controls (or, interfaces with) a peripheral component or device. Inside: e.g., a memory controller that manages access to memory, a Network Interface Controller that connects a computer to a network Outside: e.g., the controller in an external hard disk Controller contains registers and memory that bridge the device and the computer E.g., a keyboard controller s memory contains the chars that the user keyed in CIS 3207 - Operating Systems 8

Controller: two forms Controller boards Network interface controller Graphics controller Chips Northbridge: contains mem controller Southbridge: hub of I/O controllers Keyboard controller Interrupt Controller CIS 3207 - Operating Systems 9

Bus and port Bus: logically, a set of wires used to pass information between computer components. E.g., PCI (Peripheral Component Interconnect) PCIe (PCI express) Port: an interface to connect a computer with other computers or peripheral devices Many controllers have ports to connect I/O devices E.g., VGA, DVI, HDMI, Thunderbolt, PS/2, USB ports Not to be confused with TCP/UDP ports Not to be confused with I/O ports (addresses) CIS 3207 - Operating Systems 10

CIS 3207 - Operating Systems 11

Simplified architecture CIS 3207 - Operating Systems 12

Controller registers Control register can be written to start a command or to change the mode of a device Status register Whether the current command has completed Whether it has data to read (e.g., network packets) Whether there is an error Data-in register Read to get input Data-out register Written to send output CIS 3207 - Operating Systems 13

How are the controller registers and memory addressed Port-mapped I/O Memory-mapped I/O CIS 3207 - Operating Systems 14

Port-mapped I/O Special instructions and a dedicated address space Disadvantage 1: it increases the complexity of CPU for the support of special instructions Disadvantage 2: those special instructions are very limited; e.g., in X86 they are in and out Advantage: it doesn t occupy the physical address space Try cat /proc/ioports CIS 3207 - Operating Systems 15

Port-mapped I/O example MOV DX,12345h IN AL,DX ;reads I/O port 12345h CIS 3207 - Operating Systems 16

Memory-mapped I/O Not to be confused with memory-mapped file I/O, which maps a disk file to the virtual address space of a process Controller register and memory are mapped into the same physical address space as RAM, and are accessed the same way as RAM Advantage: CPU is simpler; all instructions, e.g., MOV and ADD and addressing modes used for accessing RAM can be used for I/O Disadvantage: controller s memory occupies portion of the physical address space CIS 3207 - Operating Systems 17

Port-mapped vs. memory-mapped I/O MOV DX,12345h IN AL,DX ;reads I/O port 12345h MOV AL,[DX] ;virtual address 12345h is first translated to a physical address, which may point to controller memory, BIOS, RAM or nothing CIS 3207 - Operating Systems 18

Questions On 32-bit system, why the system reports around 3GB memory even if 4GB RAM is installed? Called 3G Barrier issue Mainly due to memory-mapped I/O What is the Solution? PAE (Physical Address Extension) supports 36-bit physical address space 64-bit systems CIS 3207 - Operating Systems 19

cat /proc/iomem CIS 3207 - Operating Systems 20

Outline I/O subsystem: hardware aspect Terms: controller, bus, port How is the device memory addressed? I/O subsystem: software aspect Device drivers programmer s perspective: polling, interrupt-driven, and DMA User-space programmer s perspective: blocking, nonblocking, & asynchronous I/O subsystem: performance aspect Vectored I/O Caching, Buffering and Spooling Some slides are courtesy of Dr. Abraham Silberschatz and Dr. Thomas Anderson CIS 3207 - Operating Systems 21

How to deal with various devices? Take the secondary storage as an example, it may be A Western Digital RAID A Seagate hard disk drive An Intel solid state drive An Amazon Elastic Block Store volume Building an operating system that handles each case individually would be too complex What should we do? Layering CIS 3207 - Operating Systems 22

Device drivers What if all the second storage systems expose the same set of interface Then the other part of the kernel doesn t need to care which storage drive is being used That is exactly what device drivers do A device driver is a piece of code that operates a particular type of device, and exports some standard interface to the kernel (or other programs) E.g., each manufacture (Intel, WD, and Seagate) provides a device driver that exports the same standard block device interface, so that the kernel can access the various devices based on the uniform interface CIS 3207 - Operating Systems 23

Device driver types Block devices Provide buffered access (i.e., data is conveyed through buffer) Usually seekable (i.e., supports random access) Character devices Provide unbuffered access (i.e., raw access) Usually not seekable (i.e., data is like a stream) Network devices Sending and receiving data packets CIS 3207 - Operating Systems 24

Interfaces $ ls al /dev crw-rw-rw- 1 root wheel 4, 64 Oct 27 22:23 ttyt0 crw-rw-rw- 1 root wheel 18, 3 Oct 27 22:23 Modem brw-r----- 1 root operator 1, 0 Oct 27 22:23 disk0 The network interface doesn t fit the Unix everything is a file principle, and does not exist in the /dev directory CIS 3207 - Operating Systems 25

Driver-device communication Polling Interrupt-driven I/O DMA (Direct Memory Access) CIS 3207 - Operating Systems 26

Polling For each byte of I/O 1. Polling the busy bit in the status register until 0 2. CPU sets read or write bit and, if write, copies data into data-out register 3. CPU sets command-ready bit 4. Controller sets busy bit, executes transfer 5. Controller clears busy bit, error bit, and command-ready bit when transfer done Step 1 is busy-wait cycle to wait for I/O from device It wastes a lot of CPU cycles Recall the busy-wait lock in concurrent programming Can someone notify CPU when the busy bit = 0? CIS 3207 - Operating Systems 27

Interrupt-driven I/O CPU Interrupt-request line is set by I/O device Checked by processor after each instruction Dispatch handling to correct handler according to the interrupt number Recall the Interrupt Descript Table Interrupt chaining if more than one device uses the same interrupt number An interrupt is generated by devices when Input is ready Output is done CIS 3207 - Operating Systems 28

Direct Memory Access Contrast to programmed I/O (polling and interruptdriven I/O), which relies on CPU to issue instructions to move each byte/word CPU initiates DMA command; once initiated, CPU is free to do other work and the DMA controller will take care of the I/O The initiation is costly, so DMA is not suitable for moving a small amount of data, for example Keyboard and mouse DMA is now frequently used for Network, graphics, disk I/O CIS 3207 - Operating Systems 29

Three steps of DMA CPU tells the DMA controller the following info. Source and destination addresses Read or write mode Count of bytes DMA controller arranges data transfer between device and memory according the info. When done, DMA controller issues an interrupt to signal completion CIS 3207 - Operating Systems 30

Addr. translation is important for security Each TLB and page table entry has a Supervisor bit Entries for kernel memory have the bit set If a user space program (ring 3) requests to translate a virtual address using these entries, a permission violation Page Fault will be triggered This is how kernel (ring 0) memory gets protected Each process uses its own page table to translate addresses, Process A cannot access the content of Process B unless they share Finally, when a process writes to a read-only page, e.g., for libc code, the address translation again will trigger a permission violation Page Fault CIS 3207 - Operating Systems 31

Insecure memory access with DMA DMA devices access memory using physical addresses, so they don t go through address translation Do you sense something uncomfortable? When you leave your laptop (even you lock it), it may be attacked nearly arbitrarily DMA attack a kind of physical attacks that allows you to read/write all physical memory https://youtu.be/hdhpy7rpujm Interception tool https://github.com/carmaa/inception CIS 3207 - Operating Systems 32

Six Step Process to Perform DMA Transfer 5. DMA controller transfers bytes to buffer X, increasing memory address and decreasing C until C 0 6. when C 0, DMA interrupts CPU to signal transfer completion 1. device driver is told to transfer disk data to buffer at address X 2. device driver tells disk controller to transfer C bytes from disk to buffer at address X DMA/bus/ interrupt controller CPU cache CPU memory bus PCI bus x memory buffer IDE disk controller disk disk 3. disk controller initiates DMA transfer 4. disk controller sends each byte to DMA controller disk disk CIS 3207 - Operating Systems 33

Application I/O Interface I/O system calls encapsulate device behaviors in generic classes Device-driver layer hides differences among I/O controllers from kernel New devices talking already-implemented protocols need no extra work Each OS has its own I/O subsystem structures and device driver frameworks Devices vary in many dimensions Character-stream or block Sequential or random-access Synchronous or asynchronous (or both) Sharable or dedicated Speed of operation read-write, read only, or write only CIS 3207 - Operating Systems 34

Different styles of userspace I/O Blocking - process suspended until I/O completed Easy to use and understand Recall the first project: it is good to send the whole file Nonblocking - I/O call reads what is available, or writes what it can write until the device buffer is full, and then Returns quickly with count of bytes read or written select() to find if data ready then read() or write() to transfer Asynchronous - process runs while I/O executes I/O subsystem signals process when I/O completed Make use of multi-threading CIS 3207 - Operating Systems 35

Sync vs. Async Synchronous Asynchronous CIS 3207 - Operating Systems 36

Outline I/O subsystem: hardware aspect Terms: controller, bus, port How are the device registers and memory addressed? I/O subsystem: software aspect Device drivers programmer s perspective: polling, interrupt-driven, and DMA User-space programmer s perspective: blocking, nonblocking, & asynchronous I/O subsystem: performance aspect Caching, Buffering and Spooling I/O scheduling Some slides are courtesy of Dr. Abraham Silberschatz and Dr. Thomas Anderson CIS 3207 - Operating Systems 37

Vectored I/O Vectored I/O allows one system call to perform multiple I/O operations For example, Unix readv()and writev() accepts a vector of multiple buffers to read into or write from This scatter-gather method better than multiple individual I/O calls Decreases the number of system calls CIS 3207 - Operating Systems 38

Several important generic ideas Buffering - store data in memory while transferring between devices To cope with device speed mismatch usually between producer and concumer To enable I/O scheduling To assemble multiple parts into a whole CIS 3207 - Operating Systems 39

Several important generic ideas Spooling - placing data in temporary working area for another program to process Simultaneous Peripheral Operation On-Line Process can access the resource without waiting. After writing the data on spool, process can perform other tasks. And a dedicated spooling process sends data on spool to the device E.g: print spooling and Mail spooling etc. Difference with buffering: Spooling is a special kind of buffering; it ensures data of different files/mails doesn t interleave Caching - faster device holding copy of data Key to performance Difference with buffering: data in cache is repetitively accessed, while data in buffer is consumed like a stream CIS 3207 - Operating Systems 40

Summary I/O subsystem: hardware aspect Terms: controller, bus, port How are the device registers and memory addressed? I/O subsystem: software aspect Device drivers programmer s perspective: polling, interrupt-driven, and DMA User-space programmer s perspective: blocking, nonblocking, & asynchronous I/O subsystem: performance aspect Vectored I/O Caching, Buffering and Spooling CIS 3207 - Operating Systems 41

Back-up slides CIS 3207 - Operating Systems 42

Spinning disk track t spindle sector s arm assembly cylinder c read-write head platter arm rotation CIS 3207 - Operating Systems 43

Disk Performance Disk Latency = Positioning time ( = Seek Time + Rotation Time) + Transfer Time Seek Time: time to move disk arm over tracks (1-20ms) Rotation Time: time to rotate disk under disk head Per rotation: 4 15ms (depending on price of disk) On average, only need to wait half a rotation Transfer Time: time to transfer data onto/off of disk Disk head transfer rate: 50-100MB/s (5-10 usec/sector) Host transfer rate dependent on I/O connector (USB, SATA, ) CIS 3207 - Operating Systems 44

Disk Scheduling The operating system is responsible for using hardware efficiently for the disk drives, this means minimizing the seek time Seek time seek distance CIS 3207 - Operating Systems 45

Disk Scheduling (Cont.) Let s illustrate scheduling algorithms with a request queue; the numbers below are cylinder numbers: 98, 183, 37, 122, 14, 124, 65, 67 Current head pointer 53 CIS 3207 - Operating Systems 46

FCFS Illustration shows total head movement of 640 cylinders CIS 3207 - Operating Systems 47

SSTF Shortest Seek Time First selects the request with the minimum seek time from the current head position It may cause starvation of some requests Illustration shows total head movement of 236 cylinders CIS 3207 - Operating Systems 48

SCAN The disk arm starts at one end of the disk, and moves toward the other end, servicing requests until it gets to the other end of the disk, where the head movement is reversed and servicing continues. SCAN algorithm Sometimes called the elevator algorithm Issue: after the arm reaches one end, and the requests near that end are probably sparse (since that area has been serviced just now), while the other end may have dense requests CIS 3207 - Operating Systems 49

SCAN (Cont.) Illustration shows total head movement of 208 cylinders CIS 3207 - Operating Systems 50

C-SCAN Provides a more uniform wait time than SCAN The head moves from one end of the disk to the other, servicing requests as it goes When it reaches the other end, however, it immediately returns to the beginning of the disk, without servicing any requests on the return trip Treats the cylinders as a circular list that wraps around from the last cylinder to the first one CIS 3207 - Operating Systems 51

C-SCAN (Cont.) CIS 3207 - Operating Systems 52

C-LOOK LOOK a version of SCAN, C-LOOK a version of C-SCAN Arm only goes as far as the last request in each direction, then reverses direction immediately, without first going all the way to the end of the disk CIS 3207 - Operating Systems 53

C-LOOK (Cont.) CIS 3207 - Operating Systems 54

Other second storage Solid state storage: It has no moving parts and stores data using circuits; thus good bandwidth and lower power consumption Flash PCM (Phase Change Memory) Faster and more scalable Memristor Much denser storage CIS 3207 - Operating Systems 55

Writing assignments Compare the ideas buffering and caching When to use blocking I/O and when to use non-blocking I/O What are the advantages and disadvantages of DMA What is the benefit of having the drivers to expose the same set of interfaces to the kernel? CIS 3207 - Operating Systems 56