CSCI2467: Systems Programming Concepts

Transcription

1 CSCI2467: Systems Programming Concepts Slide set 1: A Tour of Computer Systems (CS:APP Chapter 1) Instructor: M. Toups Spring 2018

2 Overview 1 A Tour of Computer Systems Systems Information is bits plus context Programs Are Translated by Other Programs into Different Forms Processors Read and Interpret Instructions Stored in Memory Caches & Memory Hierarchy The Operating System manages processes, memory, and more Wrapping up the intro 2 Lab 0 Prerequisites review Remote access with ssh The terminal Text editors Part 1 Part 2 Part 3 Handin notes 3 Preview of next week Reading Data surprises!

3 Systems vs. Applications Applications software provides services to a user Systems software provides a platform for applications and services to build upon Application programs Operating system Processor Main memory I/O devices Software Hardware Figure 1.10 from CS:APP by: Uwe Kils CC BY-SA 3.0 Not pictured: numerous examples of libraries, firmware (for each hardware device, network switches), remote services

4 Systems code is everywhere Running networks 24/7 Cisco Catalyst network switch A Tour of Computer Systems Lab 0 Preview of next week

5 Systems code is everywhere Running networks 24/7 Linksys WRT54G switch/router A Tour of Computer Systems Lab 0 Preview of next week

6 Systems code is everywhere Feeding us!

7 Systems code is everywhere Feeding us!

8 Creating a programming language for systems Originally all systems level code written as assembly

9 Creating a programming language for systems Originally all systems level code written as assembly C developed by Dennis Ritchie : K&R book published 1988: 2nd edition (ANSI/ISO standard) 1 See Aside on p.4 of CS:APP

10 Creating a programming language for systems Originally all systems level code written as assembly C developed by Dennis Ritchie : K&R book published 1988: 2nd edition (ANSI/ISO standard) C89/C90 most common. Some updates since then: C99, C See Aside on p.4 of CS:APP 2 See Aside on p.35 of CS:APP

11 Creating a programming language for systems Originally all systems level code written as assembly C developed by Dennis Ritchie : K&R book published 1988: 2nd edition (ANSI/ISO standard) C89/C90 most common. Some updates since then: C99, C11 2 C was created for a specific purpose: to implement the UNIX OS (kernel, libraries, tools) 1 See Aside on p.4 of CS:APP 2 See Aside on p.35 of CS:APP

12 Creating a programming language for systems Originally all systems level code written as assembly C developed by Dennis Ritchie : K&R book published 1988: 2nd edition (ANSI/ISO standard) C89/C90 most common. Some updates since then: C99, C11 2 C was created for a specific purpose: to implement the UNIX OS (kernel, libraries, tools) Small, simple language. Easily ported to different hardware. 1 See Aside on p.4 of CS:APP 2 See Aside on p.35 of CS:APP

13 Creating a programming language for systems Originally all systems level code written as assembly C developed by Dennis Ritchie : K&R book published 1988: 2nd edition (ANSI/ISO standard) C89/C90 most common. Some updates since then: C99, C11 2 C was created for a specific purpose: to implement the UNIX OS (kernel, libraries, tools) Small, simple language. Easily ported to different hardware. C becomes de facto choice for systems (OS, drivers, tools) 1 See Aside on p.4 of CS:APP 2 See Aside on p.35 of CS:APP

14 Creating a programming language for systems Originally all systems level code written as assembly C developed by Dennis Ritchie : K&R book published 1988: 2nd edition (ANSI/ISO standard) C89/C90 most common. Some updates since then: C99, C11 2 C was created for a specific purpose: to implement the UNIX OS (kernel, libraries, tools) Small, simple language. Easily ported to different hardware. C becomes de facto choice for systems (OS, drivers, tools) C also becomes used in applications, but with some difficulty (pointers, fewer abstractions) 1 See Aside on p.4 of CS:APP 2 See Aside on p.35 of CS:APP

15 C: the Systems programming language

16 C: the Systems programming language No classes or objects

17 C: the Systems programming language No classes or objects (therefore no constructors or destructors)

18 C: the Systems programming language No classes or objects (therefore no constructors or destructors) No exceptions (no throw or catch)

19 C: the Systems programming language No classes or objects (therefore no constructors or destructors) No exceptions (no throw or catch) No templates/generics

20 C: the Systems programming language No classes or objects (therefore no constructors or destructors) No exceptions (no throw or catch) No templates/generics... C++ adds these things

21 C: the Systems programming language No classes or objects (therefore no constructors or destructors) No exceptions (no throw or catch) No templates/generics... C++ adds these things... making it very different! Don t confuse C and C++

22

23 hello.c (bits) #inclu de <st dio.h> int main() { print f("hel lo, wo rld\n" ); retur n 0;.}

24 hello.c (hexadecimal) e63 6c c f2e #include <stdio. 683e 0a0a 696e d61 696e a7b h>..int main().{ 0a e c. printf("hel 6c6f 2c20 776f 726c 645c 6e22 293b 0a20 lo, world\n"); e20 303b 0a7d 0a return 0;.}.

25 hello.c (hexadecimal)

26 hello.c (hexadecimal)

27 hello.c (hexadecimal)

28 hello.c (hexadecimal)

29 hello.c (hexadecimal)

30 hello.c (hexadecimal)

31 hello.c (hexadecimal)

32 hello.c (hexadecimal)

33 hello.c (hexadecimal)

34 hello.c (hexadecimal)

35 hello.c (hexadecimal)

36 hello compiled (hexadecimal)

37 hello compiled (hexadecimal)

38 Overview 1 A Tour of Computer Systems Systems Information is bits plus context Programs Are Translated by Other Programs into Different Forms Processors Read and Interpret Instructions Stored in Memory Caches & Memory Hierarchy The Operating System manages processes, memory, and more Wrapping up the intro 2 Lab 0 Prerequisites review Remote access with ssh The terminal Text editors Part 1 Part 2 Part 3 Handin notes 3 Preview of next week Reading Data surprises!

39 Steps in which hello.c gets converted to an executable printf.o hello.c Source program (text) Pre- processor (cpp) hello.i source program (text) Compiler (cc1) hello.s Assembly program (text) Assembler (as) hello.o Relocatable object programs (binary) Linker (ld) hello Executable object program (binary) Source: CS:APP, Bryant & O Hallaron

40 Layered view of systems Application programs Operating system Processor Main memory I/O devices Software Hardware Source: CS:APP, Bryant & O Hallaron

41 A diagram of the computer s hardware CPU PC ALU System bus Memory bus Bus interface I/O bridge Main memory USB Graphics adapter I/O bus Disk other devices such as network adapters Mouse Keyboard Disk hello executable stored on disk Source: CS:APP, Bryant & O Hallaron

42 Someone types in the hello command... CPU PC ALU System bus Memory bus Bus interface I/O bridge Main "hello" memory USB Graphics adapter I/O bus Disk controller Expansion slots for other devices such as network adapters Mouse Keyboard User types "hello" Display Disk Source: CS:APP, Bryant & O Hallaron

43 OS loads hello from the disk into memory CPU PC ALU System bus Memory bus Bus interface I/O bridge Main memory "hello,world\n" hello code USB Graphics adapter I/O bus Disk controller other devices such as network adapters Mouse Keyboard Disk hello executable stored on disk Source: CS:APP, Bryant & O Hallaron

44 How hello output reaches the display CPU PC ALU System bus Memory bus Bus interface I/O bridge Main "hello,world\n" memory hello code USB Graphics adapter I/O bus Disk other devices such as network adapters Mouse Keyboard "hello,world\n" Disk hello executable stored on disk Source: CS:APP, Bryant & O Hallaron

45 Caches on the Intel Core i7 CPU Processor package Core 0 Core 3 Regs Regs L1 d-cache L1 i-cache L1 d-cache L1 i-cache (shared by all cores) Main memory

46 Memory Hierarchy: many layers, from fast down to slow Smaller, faster, and costlier (per byte) storage devices Larger, slower, and cheaper (per byte) storage devices L5: L6: L4: L3: L2: L1: L0: Regs L1 cache (SRAM) L2 cache (SRAM) L3 cache (SRAM) Main memory (DRAM) Local secondary storage (local disks) Remote secondary storage CPU registers hold words retrieved from cache memory. L1 cache holds cache lines retrieved from the L2 cache. L2 cache holds cache lines retrieved from L3 cache L3 cache holds cache lines retrieved from memory. Main memory holds disk blocks retrieved from local disks. retrieved from disks on remote network servers.

47 OS Kernel handles switching between two processes Process A Process B Time read Disk interrupt Return from read CS:APP, Bryant & O Hallaron User code Kernel code User code Kernel code User code Context switch Context switch Source:

48 OS Kernel also manages Virtual Memory addressing Kernel virtual memory User stack (created at runtime) Memory invisible to user code Memory mapped region for shared libraries printf function Run-time heap (created by malloc) Program start 0 Read/write data Read-only code and data Loaded from the hello Source: CS:APP, Bryant & O Hallaron

49 Chapter 1: to sum up Systems skills give you a broad and deep insight into what happens when you write and/or run programs

50 Chapter 1: to sum up Systems skills give you a broad and deep insight into what happens when you write and/or run programs Potentially big impacts on speed, reliability, security

51 Chapter 1: to sum up Systems skills give you a broad and deep insight into what happens when you write and/or run programs Potentially big impacts on speed, reliability, security This is a brief preview of what is to come! You aren t expected to know all of this... yet

52 Chapter 1: to sum up Systems skills give you a broad and deep insight into what happens when you write and/or run programs Potentially big impacts on speed, reliability, security This is a brief preview of what is to come! You aren t expected to know all of this... yet Chapter 1 should be a quick read.

53 Chapter 1: to sum up Systems skills give you a broad and deep insight into what happens when you write and/or run programs Potentially big impacts on speed, reliability, security This is a brief preview of what is to come! You aren t expected to know all of this... yet Chapter 1 should be a quick read. Don t worry about memorizing Amdahl s law, it is a useful principle to keep in mind but I will not ask you to solve those math problems

54 2 Lab 0 Prerequisites review Remote access with ssh The terminal Text editors Part 1 Part 2 Part 3 Handin notes

55 Your first lab assignment

56 Prerequisites A few notes on this process: bash shell, text editors, permissions Also: remote access via ssh Be careful about usernames, and be aware there is a temporary block after a certain number of failed login attempts.

57 Using Sun Ray terminals Log in using UNO username, password.

58 Using Sun Ray terminals All labs are done using math209 (linux)... BUT you can access math209 on the Sun Ray in class OR remotely (on or off-campus) using ssh

59 Using git-bash

60 Using git-bash

61 Using PuTTY

62 Using PuTTY Pay attention to which prompt is which

63 Directories and permissions changed directory with cd command

64 Directories and permissions changed directory with cd command created a directory with mkdir command

65 Directories and permissions changed directory with cd command created a directory with mkdir command changed permissions with chmod command

66 Directories and permissions changed directory with cd command created a directory with mkdir command changed permissions with chmod command

67 Directories and permissions Source: Open Source GIS: A GRASS GIS Approach, First Edition, 2002

68 Directory hierarchy Starting at the root / (root of file system) bin etc home lib opt sbin tmp usr var

69 Directory hierarchy System files/programs / (root of file system) bin bash home usr bin... firefox vim lib share...

70 Directory hierarchy Our work happens in /home / (root of file system)... home... hmahdi matoups2... usr...

71 Directory hierarchy We each created a 2467 working directory in our own home directory / (root of file system)... home... hmahdi matoups Downloads usr

72 Directory hierarchy... and worked on files within that directory / (root of file system) home matoups introlab-handout.tar part1 part1.c student.c student.h part2 part3

73 Directory hierarchy There is also a separate class directory /home/csci2467 / (root of file system) home CSCI2467 handin labs lab0 introlab-handout.tar matoups part1 part1.c part1

74 Directory hierarchy / (root of file system) home CSCI2467 handin matoups2 labs lab0 introlab-handout.tar matoups2 2467

75 Directory hierarchy / (root of file system) home CSCI2467 handin... matoups2 labs matoups part1 part1.c part1

76 Terminal Shortcuts (bash shell) is an alias to your home directory: cp foo.txt ~

77 Terminal Shortcuts (bash shell) is an alias to your home directory: cp foo.txt ~. is an alias to your present directory: cp ~/foo.txt.

78 Terminal Shortcuts (bash shell) is an alias to your home directory: cp foo.txt ~. is an alias to your present directory: cp ~/foo.txt... is an alias to the parent directory: cp ~/foo.txt..

79 Terminal Shortcuts (bash shell) is an alias to your home directory: cp foo.txt ~. is an alias to your present directory: cp ~/foo.txt... is an alias to the parent directory: cp ~/foo.txt.. * will match as many characters as it can. cp ~/*.txt.

80 Terminal Shortcuts (bash shell) is an alias to your home directory: cp foo.txt ~. is an alias to your present directory: cp ~/foo.txt... is an alias to the parent directory: cp ~/foo.txt.. * will match as many characters as it can. cp ~/*.txt. Example: objdump -d *

81 Terminal Shortcuts (bash shell) is an alias to your home directory: cp foo.txt ~. is an alias to your present directory: cp ~/foo.txt... is an alias to the parent directory: cp ~/foo.txt.. * will match as many characters as it can. cp ~/*.txt. Example: objdump -d * Example: rm *.c (be very very very careful!!)

82 Terminal Shortcuts (bash shell) is an alias to your home directory: cp foo.txt ~. is an alias to your present directory: cp ~/foo.txt... is an alias to the parent directory: cp ~/foo.txt.. * will match as many characters as it can. cp ~/*.txt. Example: objdump -d * Example: rm *.c (be very very very careful!!) There is no trash with rm. It is gone.

83 More bash tricks Pressing tab will autocomplete filenames.

84 More bash tricks Pressing tab will autocomplete filenames. Use the up & down arrow keys to scroll through your previous commands.

85 More bash tricks Pressing tab will autocomplete filenames. Use the up & down arrow keys to scroll through your previous commands. Control+R lets you search your command history.

86 More bash tricks Pressing tab will autocomplete filenames. Use the up & down arrow keys to scroll through your previous commands. Control+R lets you search your command history. Control+C terminates your current program.

87 More bash tricks Pressing tab will autocomplete filenames. Use the up & down arrow keys to scroll through your previous commands. Control+R lets you search your command history. Control+C terminates your current program. Control+D (on a blank line) exits the terminal.

88 Text editors Source: xkcd.com

89 nano Note command-key functions described at bottom

90 vi / vim see vimtutor command stateful or modal editor: can be in command, insert or visual mode.

91 emacs Described in Section 4 of Unix programming tools (optional reading)

92 gedit Nice GUI you can use in the lab Disadvantage: doesn t work (easily) via ssh Previous examples were all text-mode (terminal-based, work the same over ssh)

93 Sublime Text

94 How to use Sublime Text for these labs Haydar can show you how to set up SFTP for sublime to keep files in sync with our server. (See resources section of class website)

95 Notepad++ Haydar can show you the steps to get notepad++ to sync with our server.

96 Notepad++ Haydar can show you the steps to get notepad++ to sync with our server.

97 Notepad++ Haydar can show you the steps to get notepad++ to sync with our server.

98 Part 1: Who are you?

99 Part 1: Who are you?

100 Part 1: Who are you?

101 Part 1: Compiling and running Save (in text editor)

102 Part 1: Compiling and running Save (in text editor) Compile with gcc as given in lab manual

103 Part 1: Compiling and running Save (in text editor) Compile with gcc as given in lab manual Did compilation succeed? (No errors means yes)

104 Part 1: Compiling and running Save (in text editor) Compile with gcc as given in lab manual Did compilation succeed? (No errors means yes) Run with./part1

105 Part 1: Compiling and running Save (in text editor) Compile with gcc as given in lab manual Did compilation succeed? (No errors means yes) Run with./part1 Turn in by copying into /home/csci2467/handin/<userid>

106 Part 2: lifecycle of a C program A slightly modified hello.c

107 Part 2: lifecycle of a C program printf.o hello.c Source program (text) Pre- hello.i processor (cpp) source program (text) Compiler (cc1) hello.s Assembly program (text) Assembler (as) hello.o Relocatable object programs (binary) Linker (ld) hello Executable object program (binary) Source: Bryant & O Hallaron (2003)

108 Part 3: How long does it take to copy an array? Source: Bryant & O Hallaron (2003)

109 Part 3: How long does it take to copy an array? Source: Bryant & O Hallaron (2003)

110 Part 3: Memory mountain Source: Bryant & O Hallaron (2003)

111 Part 3: Memory hierarchy from Chapter 1 Smaller, faster, and costlier (per byte) storage devices Larger, slower, and cheaper (per byte) storage devices L5: L6: O Hallaron (2003) L4: L3: L2: L1: L0: Regs L1 cache (SRAM) L2 cache (SRAM) L3 cache (SRAM) Main memory (DRAM) Local secondary storage (local disks) Remote secondary storage CPU registers hold words retrieved from cache memory. L1 cache holds cache lines retrieved from the L2 cache. L2 cache holds cache lines retrieved from L3 cache L3 cache holds cache lines retrieved from memory. Main memory holds disk blocks retrieved from local disks. retrieved from disks on remote network servers. Source: Bryant &

112 Wrapping up: Handing in your work To hand in your work (when finished), you will put the files into /home/csci2467/handin/<username> on math209 (not your own system) Programs must work when compiled with gcc on math209 You can work in the lab or remotely via ssh. Either way your program must compile and run on math209. (Compile and run before you hand in)

113 3 Preview of next week Reading Data surprises!

114 Chapter 2 is long, start today! Read 2.1 before Monday January 29 Section 2.2 for Wednesday January 31 Section 2.3 for Monday February 5

115 Source: xkcd.com

116 ints are not Integers Source: xkcd.com Z is infinitely large, computer memory is not. This is the fundamental challenge!