Computer Systems A Programmer s Perspective 1 (Beta Draft)

Transcription

1 Computer Systems A Programmer s Perspective 1 (Beta Draft) Randal E. Bryant David R. O Hallaron August 1, Copyright c 2001, R. E. Bryant, D. R. O Hallaron. All rights reserved.

2 2

3 Contents Preface i 1 Introduction Information is Bits in Context Programs are Translated by Other Programs into Different Forms It Pays to Understand How Compilation Systems Work Processors Read and Interpret Instructions Stored in Memory Hardware Organization of a System Running the hello Program Caches Matter Storage Devices Form a Hierarchy The Operating System Manages the Hardware Processes Threads Virtual Memory Files Systems Communicate With Other Systems Using Networks Summary I Program Structure and Execution 19 2 Representing and Manipulating Information Information Storage Hexadecimal Notation Words

4 4 CONTENTS Data Sizes Addressing and Byte Ordering Representing Strings Representing Code Boolean Algebras and Rings Bit-Level Operations in C Logical Operations in C Shift Operations in C Integer Representations Integral Data Types Unsigned and Two s Complement Encodings Conversions Between Signed and Unsigned Signed vs. Unsigned in C Expanding the Bit Representation of a Number Truncating Numbers Advice on Signed vs. Unsigned Integer Arithmetic Unsigned Addition Two s Complement Addition Two s Complement Negation Unsigned Multiplication Two s Complement Multiplication Multiplying by Powers of Two Dividing by Powers of Two Floating Point Fractional Binary Numbers IEEE Floating-Point Representation Example Numbers Rounding Floating-Point Operations Floating Point in C Summary

5 CONTENTS 5 3 Machine-Level Representation of C Programs A Historical Perspective Program Encodings Machine-Level Code Code Examples A Note on Formatting Data Formats Accessing Information Operand Specifiers Data Movement Instructions Data Movement Example Arithmetic and Logical Operations Load Effective Address Unary and Binary Operations Shift Operations Discussion Special Arithmetic Operations Control Condition Codes Accessing the Condition Codes Jump Instructions and their Encodings Translating Conditional Branches Loops Switch Statements Procedures Stack Frame Structure Transferring Control Register Usage Conventions Procedure Example Recursive Procedures Array Allocation and Access Basic Principles Pointer Arithmetic

6 6 CONTENTS Arrays and Loops Nested Arrays Fixed Size Arrays Dynamically Allocated Arrays Heterogeneous Data Structures Structures Unions Alignment Putting it Together: Understanding Pointers Life in the Real World: Using the GDB Debugger Out-of-Bounds Memory References and Buffer Overflow *Floating-Point Code Floating-Point Registers Extended-Precision Arithmetic Stack Evaluation of Expressions Floating-Point Data Movement and Conversion Operations Floating-Point Arithmetic Instructions Using Floating Point in Procedures Testing and Comparing Floating-Point Values *Embedding Assembly Code in C Programs Basic Inline Assembly Extended Form of asm Summary Processor Architecture Optimizing Program Performance Capabilities and Limitations of Optimizing Compilers Expressing Program Performance Program Example Eliminating Loop Inefficiencies Reducing Procedure Calls Eliminating Unneeded Memory References

7 CONTENTS Understanding Modern Processors Overall Operation Functional Unit Performance A Closer Look at Processor Operation Reducing Loop Overhead Converting to Pointer Code Enhancing Parallelism Loop Splitting Register Spilling Limits to Parallelism Putting it Together: Summary of Results for Optimizing Combining Code Floating-Point Performance Anomaly Changing Platforms Branch Prediction and Misprediction Penalties Understanding Memory Performance Load Latency Store Latency Life in the Real World: Performance Improvement Techniques Identifying and Eliminating Performance Bottlenecks Program Profiling Using a Profiler to Guide Optimization Amdahl s Law Summary The Memory Hierarchy Storage Technologies Random-Access Memory Disk Storage Storage Technology Trends Locality Locality of References to Program Data Locality of Instruction Fetches Summary of Locality

8 8 CONTENTS 6.3 The Memory Hierarchy Caching in the Memory Hierarchy Summary of Memory Hierarchy Concepts Cache Memories Generic Cache Memory Organization Direct-Mapped Caches Set Associative Caches Fully Associative Caches Issues with Writes Instruction Caches and Unified Caches Performance Impact of Cache Parameters Writing Cache-friendly Code Putting it Together: The Impact of Caches on Program Performance The Memory Mountain Rearranging Loops to Increase Spatial Locality Using Blocking to Increase Temporal Locality Summary II Running Programs on a System Linking Compiler Drivers Static Linking Object Files Relocatable Object Files Symbols and Symbol Tables Symbol Resolution How Linkers Resolve Multiply-Defined Global Symbols Linking with Static Libraries How Linkers Use Static Libraries to Resolve References Relocation Relocation Entries Relocating Symbol References

9 CONTENTS Executable Object Files Loading Executable Object Files Dynamic Linking with Shared Libraries Loading and Linking Shared Libraries from Applications *Position-Independent Code (PIC) Tools for Manipulating Object Files Summary Exceptional Control Flow Exceptions Exception Handling Classes of Exceptions Exceptions in Intel Processors Processes Logical Control Flow Private Address Space User and Kernel Modes Context Switches System Calls and Error Handling Process Control Obtaining Process ID s Creating and Terminating Processes Reaping Child Processes Putting Processes to Sleep Loading and Running Programs Using fork and execve to Run Programs Signals Signal Terminology Sending Signals Receiving Signals Signal Handling Issues Portable Signal Handling Nonlocal Jumps

10 10 CONTENTS 8.7 Tools for Manipulating Processes Summary Measuring Program Execution Time The Flow of Time on a Computer System Process Scheduling and Timer Interrupts Time from an Application Program s Perspective Measuring Time by Interval Counting Operation Reading the Process Timers Accuracy of Process Timers Cycle Counters IA32 Cycle Counters Measuring Program Execution Time with Cycle Counters The Effects of Context Switching Caching and Other Effects The -Best Measurement Scheme Time-of-Day Measurements Putting it Together: An Experimental Protocol Looking into the Future Life in the Real World: An Implementation of the -Best Measurement Scheme Summary Virtual Memory Physical and Virtual Addressing Address Spaces VM as a Tool for Caching DRAM Cache Organization Page Tables Page Hits Page Faults Allocating Pages Locality to the Rescue Again

11 CONTENTS VM as a Tool for Memory Management Simplifying Linking Simplifying Sharing Simplifying Memory Allocation Simplifying Loading VM as a Tool for Memory Protection Address Translation Integrating Caches and VM Speeding up Address Translation with a TLB Multi-level Page Tables Putting it Together: End-to-end Address Translation Case Study: The Pentium/Linux Memory System Pentium Address Translation Linux Virtual Memory System Memory Mapping Shared Objects Revisited The fork Function Revisited The execve Function Revisited User-level Memory Mapping with the mmap Function Dynamic Memory Allocation The malloc and free Functions Why Dynamic Memory Allocation? Allocator Requirements and Goals Fragmentation Implementation Issues Implicit Free Lists Placing Allocated Blocks Splitting Free Blocks Getting Additional Heap Memory Coalescing Free Blocks Coalescing with Boundary Tags Putting it Together: Implementing a Simple Allocator Explicit Free Lists

12 12 CONTENTS Segregated Free Lists Garbage Collection Garbage Collector Basics Mark&Sweep Garbage Collectors Conservative Mark&Sweep for C Programs Common Memory-related Bugs in C Programs Dereferencing Bad Pointers Reading Uninitialized Memory Allowing Stack Buffer Overflows Assuming that Pointers and the Objects they Point to Are the Same Size Making Off-by-one Errors Referencing a Pointer Instead of the Object it Points to Misunderstanding Pointer Arithmetic Referencing Non-existent Variables Referencing Data in Free Heap Blocks Introducing Memory Leaks Summary III Interaction and Communication Between Programs Concurrent Programming with Threads Basic Thread Concepts Thread Control Creating Threads Terminating Threads Reaping Terminated Threads Detaching Threads Shared Variables in Threaded Programs Threads Memory Model Mapping Variables to Memory Shared Variables Synchronizing Threads with Semaphores Sequential Consistency

13 CONTENTS Progress Graphs Protecting Shared Variables with Semaphores Posix Semaphores Signaling With Semaphores Synchronizing Threads with Mutex and Condition Variables Mutex Variables Condition Variables Barrier Synchronization Timeout Waiting Thread-safe and Reentrant Functions Reentrant Functions Thread-safe Library Functions Other Synchronization Errors Races Deadlocks Summary Network Programming Client-Server Programming Model Networks The Global IP Internet IP Addresses Internet Domain Names Internet Connections Unix file I/O The read and write Functions Robust File I/O With the readn and writen Functions Robust Input of Text Lines Using the readline Function The stat Function The dup2 Function The close Function Other Unix I/O Functions Unix I/O vs. Standard I/O

14 14 CONTENTS 12.5 The Sockets Interface Socket Address Structures The socket Function The connect Function The bind Function The listen Function The accept Function Example Echo Client and Server Concurrent Servers Concurrent Servers Based on Processes Concurrent Servers Based on Threads Web Servers Web Basics Web Content HTTP Transactions Serving Dynamic Content Putting it Together: The TINY Web Server Summary A Error handling 667 A.1 Introduction A.2 Error handling in Unix systems A.3 Error-handling wrappers A.4 The csapp.h header file A.5 The csapp.c source file B Solutions to Practice Problems 693 B.1 Intro B.2 Representing and Manipulating Information B.3 Machine Level Representation of C Programs B.4 Processor Architecture B.5 Optimizing Program Performance B.6 The Memory Hierarchy

15 CONTENTS 15 B.7 Linking B.8 Exceptional Control Flow B.9 Measuring Program Performance B.10 Virtual Memory B.11 Concurrent Programming with Threads B.12 Network Programming

16 16 CONTENTS