A Security Architecture for Microprocessors

Transcription

1 A Security Architecture for Microprocessors Doctoral Thesis Jörg Platte Genehmigte Dissertation zur Erlangung des akademischen Grades eines Doktors an der Fakultät für Elektrotechnik und Informationstechnik Technischen Universität Dortmund Abteilung Informationstechnik Institut für Roboterforschung

2 II Acknowledgements It is my pleasure to thank all the people who supported me to make this thesis possible. Prüfungskommission: Prof. Dr.-Ing. Christian Rehtanz (Vorsitzender) Prof. Dr.-Ing. Uwe Schwiegelshohn (Referent) Prof. Dr.-Ing. Christian Grimm (Korreferent) Dr.-Ing. Wolfgang Endemann

3 III Abstract The Security Architecture for Microprocessors (SAM ) is a lightweight and high-performance combined hard- and software security extension for microprocessors. SAM has been designed to provide a secure remote code execution environment. It can be used to implement effective copy-protection schemes and provides mechanisms to prevent data and algorithm disclosure. SAM provides protection even if an attacker has full access to both the operating system and hardware. SAM uses an enhanced processor core which can be used as a drop in replacement for a standard processor to provide transparent encryption and hashing of memory contents to prevent external tampering and sniffing attacks. Further internal security-related extensions support a secure operating system implementation. Both the hardware and software design are presented in this thesis.

4 IV

5 Contents List of Figures List of Tables List of Algorithms E F G 1 Introduction 1 I Software Protection Schemes 5 2 Computer and Operating System Architecture Processor Overview Instruction Set Protection Mechanisms Virtual Memory Memory Hierarchy Interrupts Operating Systems Memory Layout Interrupts Virtual Memory Handling Protection Mechanisms Multiuser Environment File System Process Management Administrator Access Sophisticated Access Control Security Issues Hardware Access Software-Based Attacks Side Channel Attacks Program Analysis Copy Protection Sandbox Security A

6 B CONTENTS 4 Cryptography Symmetric Cryptography Random Number Generation Algorithms Modes of Operation Data Integrity Asymmetric Cryptography Key Exchange Protocols Cryptographic Attacks Brute Force Known Plaintext Related-Key Attack Replay Attacks XOR Security Considerations Man-in-the-Middle Other Attacks Standard Protocols Security Architectures Memory Protection Schemes Hash Trees Memory Encryption Hiding Address Information Security Architectures Smart Cards Secure Co-Processors LaGrande Digital Rights Management X-Box XOM AEGIS II The Security Architecture for Microprocessors (SAM ) 35 6 SAM Design Goals Motivation Requirements for Secure Computing Hardware Requirements Software Requirements Processor Architecture Overview Cryptographic Keys Tamper Detection Unit RSA Unit and SAM Configurator Cryptographic Functions

7 CONTENTS C 7.6 Security-Aware Cache Memory Layout Memory Views Protected Operating System Protected TRAP Table Sparse Hash Tree SAM Instruction Set Speculative Execution Memory Decryption Attack Secure Speculative Execution Memory Protection Memory Integrity Verification Memory Encryption SAM Implementations SAM for SPARC Register Protection Instruction Set Context Switches TRAP Modifications Memory-Mapped Configuration Registers Cache Further Changes SAM for IA-32 Processors Register Protection Context Handling and Configuration Privilege Level Transitions Instruction Set Multiprocessor Support SAM Operating System Design Threats Protected Kernel Limitations Loading Protected Programs Hash Tree Handling User-Supervisor Mode Transitions Protected Compartment System Calls Multi Threading and Signal Handling Cache Architecture Comparison with other Caches L1 Data Cache L1 Instruction Cache L2 Cache

8 D CONTENTS TAG RAM Hit Logic TLB AES Unit Queues Cache Arbitration and Deadlock Prevention Speculative Execution Performance Optimizations Read-only Shared Memory Prefetching Application Design Compiler and Assembler SAM Linker Tasks SAM File Format Library Support Limited Execution Data Exchange Sample Application: Java Virtual Machine Design Goals Architecture Usage Scenario Performance Analysis Security Analysis Security Analysis Hashing Replay Attack Random Attack Pre-Image Attack Birthday Attack Cryptography Speculative Execution Processor Architecture SAM Operating System Implementation Comparison with other Architectures Evaluation VHDL Implementation Development Hardware Processor L2 Cache Cryptographic Units Simulation Model System Emulation Environment Cache Simulator

9 CONTENTS E Limitations Operating System L2 Prefetcher Cache-Memory Overhead Simulation Results L2 Cache with speculative Execution L2 Cache without speculative Execution Cache Size Variations Cryptography L2 Cache with Prefetching Queues Multitasking Benchmarks Conclusion and Future Work 137 A System Emulation Parameters 139 B Speedups for selected Configurations 140 Bibliography 167

10 List of Figures 7.1 Schematic SAM processor overview Memory Layout for each protected process Hash tree layout Hash value computation Memory decryption SPARC Register Windows Transitions Addresses and flags passed to caches Cache accesses and miss rates for selected cache configurations L1/L2 cache design Queue data flow and dependencies Context Dictionary Stride-Filtered Markov-Predictor SJVM Overview Placement of logical units on both FPGA s Cache simulation results Speculative execution Different cache sizes Write Through cache configuration Performance of cryptographic-related parts of the L2 cache Cache access prefetching Geometric average of speedup for chosen benchmarks (8-256) Different predictors Influence of stride predictors and Markov predictors Queue-related configurations Comparison between single-task and multitasking benchmarks Multitasking benchmarks F

11 List of Tables 7.1 SAM s processor flags Supervisor-mode-related security violations st immediate constant description SAM configuration address space Contents of CR Flags set and checked by the kernel TAG-RAM entries used by the L1 data cache TAG-RAM entries used by the L2 cache L2 hit logic Comparison of LEON and SAM -LEON LEON Configuration FPGA-RAM usage L2 cache LUT usage Size of cryptographic units Trace file information Cache properties Cache configurations Parameter ranges and descriptions used as a genome Cache-memory overhead (without prefetching capabilities) JavaEvA results A.1 Abbreviations A.2 QEMU configurations G

12 List of Algorithms 7.1 Function verifycacheline(address, CacheLine) Implementation of st Implementation of rprot Implementation of copybits H

13 Chapter 1 Introduction The digital revolution and the omnipresence of computers has changed our life and our sense of justice. With the mp3 compression format and the availability of small players digital music is now a part of everybody s life. The same applies for digital videos and movies as well as for computer games and software. One of the reasons why digital multimedia content is getting more and more successful is their availability. Everything can be downloaded from the Internet, regardless of copyright and distribution restrictions. Most people do not even feel that they are breaking laws when they are copying digital contents. One reason for this is that the ownership and distribution of physical goods can easily be defined, but this is not the case for digital data like programs or digital multimedia content. This can be illustrated by the following example: Digital data can be produced by companies, just like physical goods. When producing physical goods like cars, a lot of effort is required for both development and production. This is typically not the case for digital data. Here, the effort for development is much higher than for production. This is due to the fact that digital data can easily be copied or reproduced once it has been created. A car, for example, has to be physically moved to the customer, and the customer has to buy another car in case he needs another one. However, this does not apply to digital data. It can be copied without notable effort, and broadband Internet connections have replaced previous distribution channels requiring physical data mediums like compact discs. Due to the success of CD and DVD burners and the growth of hard disk capacities, it is possible to permanently store huge amounts of data. This includes copyrighted work like programs or multimedia data. The laws have been adjusted to meet the properties of digital data. But this is not sufficient, since people break the laws regularly, if it is easy (like driving faster than allowed) and hard to prove. Therefore, digital data requires other protection schemes than physical goods. While physical goods can easily be protected against theft, this is not possible for digital data, because digital data has to be copied all the time and it is not easy to distinguish between allowed and forbidden copying. Typically, data is copied from the hard disk or an optical medium to main memory and then copied into the processor. There are copy-protection schemes relying on additional verifications during runtime. For example, a program may request permission from a license server before it starts. But these kind of verifications can easily be broken just by removing the check from the program code. Since this is mostly impossible for inexperienced users, a lot of programs designed to remove these copy-protection checks exist. Protecting software against unauthorized distribution is one goal of the Security Architecture 1

14 2 CHAPTER 1. INTRODUCTION for Microprocessors (SAM ), but it is not the only one. In some cases the program itself has to be protected against external analysis. This is comparable to physical goods, where the production process is kept secret to prevent replicas of the same quality produced by competitors. For programs, all know-how is represented in the programs algorithms. If a competitor is able to analyze these algorithms, he is able to write programs of similar performance or quality. Especially multimedia content is distributed in encrypted form to prevent unauthorized copying. But the program used to display the content has to be able to decrypt the data. If it is now possible to analyze the program, an attacker may be able to analyze the decryption algorithm or to extract the encryption key. Another scenario where digital data protection is required is GRID computing. In GRID environments, simulation data and programs are spread all over the world to be executed on computing clusters. Typically, the submitter of these programs cannot ensure that the executing computers are well administered and the simulation programs with the computed data cannot be analyzed or modified by third parties. Today, this limitation prevents the commercial usage of GRID computing, due to the probability of spy attacks by competitors. There are other areas which would benefit from additional software protection, as for example in mobile agent scenarios, where programs (the mobile agents) are transferred and executed on several computer systems to perform tasks in behalf of the agent s owner. These agentbased systems can work reliably only if the agent can perform its tasks without any external tampering attempts. For example, it could be possible to modify the agent s collected results to spoof the agent s owner. There are solutions available for some of these problems [80], but there is no general approach. Software only protection schemes are limited, because they cannot prevent memory-based attacks like memory dumps performed by the administrator. Even a protected execution environment which prevents execution of unauthorized software cannot prevent memory attacks if hardware supported bus sniffers are used. This kind of attack has been noticed by the broad public when the copy-protection of the XBox [36] has been analyzed by bus sniffing. Likewise, a hardware-only approach has limitations in a multitasking environment as well, because programs interact with other programs and access data over the network or on the hard disk. Hence, in a multitasking environment any possible attack based on the (modified) operating system has to be prevented as well as hardware-based attacks. With a fully untrusted operating system, all tasks required to support secure multitasking would have to be implemented in hardware. This might be possible but requires more complex and expensive chip designs. Hence, hardware only protection schemes can be used only for small systems like smart cards or microcontrollers. But these approaches are restricted due to limitations in terms of available memory, computing power or network and file system access. Most new technologies require some time before they are accepted and widely used. Therefore, most new technologies are first developed as an optional add-on to existing technologies. Hence, the acceptance of a security extension for computer systems can be improved if it is implemented as an optional extension, which can be used but is not mandatory to be activated to raise their acceptance. Furthermore, more security should not result in less flexibility from the user s point of view. A secure system which limits the number of programs to be executed or requires only new and adjusted software without compatibility to existing software is not likely to be accepted. Hence, the Security Architecture for Microprocessors (SAM ) has been developed as a drop in replacement for a standard RISC processor by providing full downward compatibility. Other hardware parts remain unchanged, only the operating system has to be adjusted to take

15 advantage of the new protection mechanisms for a secure user environment. The processor has been extended to provide strong cryptographic functions used to transparently encrypt, decrypt and hash memory contents to protect all program-related data. In the following, the term protected program refers to a program designed to use the SAM protection mechanisms. An unprotected program is a normal program which uses none of SAM s protection parts. This work is structured as follows. Part I provides an overview of existing technologies that can be used to protect programs. It starts in chapter 2 with basic protection mechanisms on the processor level. Chapter 3 shows threats for current computer systems and software. A short overview of cryptographic functions is given in chapter 4, followed by security architectures based on cryptographic algorithms in chapter 5. Part II describes SAM in detail. At first the SAM design goals are presented in chapter 6. Chapters 7 and 8 describe the architecture and the memory protection mechanisms. The next chapters describe sample implementations of SAM. This includes two processor designs in chapter 9, an operating system design in chapter 10, a cache implementation in chapter 11 and a sample application design in chapter 12. Chapter 13 analyzes the security of SAM. The simulation environment and simulation results are presented in chapter 14. Finally, chapter 15 concludes this work. The appendices A and B list simulation results and configuration parameters. 3

16 4 CHAPTER 1. INTRODUCTION

17 Part I Software Protection Schemes 5

18 Chapter 2 Computer and Operating System Architecture from a Security Point of View This chapter gives a brief overview of modern microprocessors and operating system kernels. Beside a presentation of the basic functions of a processor and an operating system, this chapter provides an overview of the software protection mechanisms of current operating systems. 2.1 Processor Overview Today s processor architectures can be divided into CISC (Complex Instruction Set Computing) and RISC (Reduced Instruction Set Computing) architectures [33]. CISC architectures have a longer history and their instruction set provides a lot of functionality. The instructions have different sizes based on their information content. Since not all instructions have the same complexity, their execution time is different. Many CISC architectures provide a small number of general-purpose registers but allow memory contents as operands. In contrast to CISC processors, RISC processors have a simple instruction set. For example, the first versions of the SPARC [85] architecture did not provide multiplication or division instructions. These operations had to be emulated using a couple of more simple instructions. To simplify memory access all instructions have the same size, a mostly similar runtime and instruction format. These properties are then used to increase execution performance by using pipelining. Compared to CISC processors, RISC processors typically have a large number of general-purpose registers, but operands stored in memory cannot directly be addressed by instructions except for dedicated load and store instructions (Load/Store architecture). Along with the general-purpose registers most processors contain a number of special registers. Sometimes the special registers are accessible by dedicated instructions only or they are for internal use only. Examples for values stored in special register are: The Program Counter (PC) or Instruction Pointer (IP) which always points to the currently executed instruction. Interrupt-related registers to enable or disable interrupts. 6

19 2.1. PROCESSOR OVERVIEW 7 The current privilege level, because some instructions require special privileges to be executable. Most processors provide at least two levels: supervisor mode and user mode. Many modern processors cannot be classified as RISC or CISC, because they use a hybrid approach. Intel, for example, translates CISC instructions to one or more so-called µops [9]. These µops are less complex and can therefore be processed much faster Instruction Set Protection Mechanisms Most processor architectures provide protection mechanisms on the instruction set level by providing at least two modes of operation: user and supervisor mode. User mode defines a normal operation mode for programs without granting additional privileges like direct access to all instructions, memory and processor configuration registers used for memory management or interrupt handling (see below). These operations are limited to supervisor mode, and this mode is dedicated to the Operating System (OS) Virtual Memory A processor supporting virtual memory distinguishes between addressable and physically available memory. For 32-bit architectures the addressable memory region covers 2 32 bytes, but there may be more or less physical RAM (Random Access Memory) available. The virtual address space is typically divided in pages with a typical size of at least 4096 bytes, and they can be mapped to physical memory using a page table. Beside the mapping information the page table contains additional data for memory access permission like read, write or execute permissions. All of these functions are provided by the Memory Management Unit (MMU). To speed up mapping of virtual pages to physical pages, a Translation Lookaside Buffer (TLB) can be used. This buffer stores a certain number (for example 16) of the last accessed page mappings to prevent page table accesses when these mappings are needed again. Some architectures provide direct hardware support for several processes, called contexts, running at the same time. Read or write access can then be granted individually for each context and each page. For architectures supporting only one context a new page table has to be provided manually and the TLB has to be flushed. Each time a virtual page is not mapped to a physical page or on insufficient access permission an interrupt is raised. This mechanism can be used to further extend the physical memory by for example storing pages on hard disk and loading them to Random Access Memory (RAM) on access Memory Hierarchy All available memory in a computer is organized hierarchically, because the speed of today s processors is growing faster than the memory access speed. Sometimes even parts of the main memory are shared by the processor and the graphic card, thus further limiting the memory bandwidth. With SRAM there is a fast RAM technology available, but SRAM (Static Random Access Memory) is more expensive than the commonly used DRAM (Dynamic Random Access Memory) technology. Hence, memory is organized in layers, where each layer closer to the processor has a smaller size but provides a better performance than a farther layer. Typical modern processors contain at least one cache. A cache provides a small amount of fast memory used to buffer parts of the main memory for faster access. A cache is organized

20 8 CHAPTER 2. COMPUTER AND OPERATING SYSTEM ARCHITECTURE in small chunks of memory called blocks or lines. Cache lines can be placed in a cache based on the memory address (direct-mapped), freely (fully associative) or in a combination of both (x-way set associative) [33]. Set associative caches have another level of freedom, because they have to choose a cache line to replace within a given set. Typical algorithms for cache line selection are FIFO 1, LRU 2 and random selection. A cache line can be loaded into the cache on each read or write access (write allocate) or on read access only (no write allocate). Data written to the cache can be passed directly to memory (write-through) or on a cache line exchange (write-back). Each cache stores mainly user data but additionally a small amount of data used for cache management. The memory used to store this data is called TAG-RAM. It typically contains cache line identifiers, flags to mark lines with valid and modified data and cache line usage information used by the line replacement algorithm. A processor typically contains a cache hierarchy with a cache for data (L1 data cache) and another cache for instructions (L1 instruction cache) and another and much larger but slower combined data and instruction cache (L2 cache) Interrupts Interrupts, sometimes called TRAP or fault, can be divided into hardware and software interrupts. Hardware interrupts are raised by hardware, for example when new data is available from the hard disk or to report failures. Software interrupts are raised by dedicated instructions. The processor stores the base address of the interrupt table (TRAP table) either in a special register or at a given position in memory. The interrupt table typically contains the addresses of the corresponding interrupt handlers or a couple of instructions to be executed on each interrupt. An interrupt suspends the current program flow to execute the interrupt handler. To be able to continue execution at the interrupted position the processor stores at least the program counter on each interrupt of the interrupted instruction and based on the architecture additional information in registers or dedicated memory. After each interrupt further interrupts are temporarily disabled by the processor. When the state of the interrupted program has been saved successfully, the interrupt handler can reenable interrupts. In cases where priorized interrupts are supported only interrupts with a higher priority than the current one can be raised. Cases where an interrupt handler is interrupted by another interrupt are called nested interrupts. With each interrupt, the current privilege level is changed to a higher one. An interrupt handler is left with a dedicated instruction which adjusts the privilege level and resumes the interrupted program. The same mechanism is used to provide breakpoints to support program analysis. Breakpoints are markers which interrupt program execution if the marked instruction is to be executed. There are hardware and software breakpoints. For hardware breakpoints the instruction address is stored in a processor-specific register. Software breakpoints are realized by replacing the marked instruction by a software interrupt instruction. 1 First In, First Out. 2 Least Recently Used.

21 2.2. OPERATING SYSTEMS Operating Systems The task of an Operating System (OS) is to manage the hardware and software resources of a computer. This section gives a brief introduction based on UNIX-like multiuser and preemptive multitasking operating system kernels. These kernels are implemented using a monolithic design. In a monolithic kernel all core functions like memory and process management and drivers are executed with full access to all hardware and software resources. On the one hand this increases performance, but on the other hand the system stability may decrease, since a faulty driver may accidentally overwrite vital parts of the operating system Memory Layout A user process does not have full access to the whole virtual address space. A contiguous part, typically located at higher addresses is used by the kernel only and is shared between all processes. The corresponding pages are not directly accessible by user processes. This kernel address space contains the kernel and memory used to store kernel data like process and memory descriptors and kernel stacks. The remaining virtual address space can be used by programs. Each process has its own virtual address space but it is possible to share memory between two or more processes. Some of the actual addresses used by a program are not determined by the kernel but by the linker at program creation time. The task of the linker is to take one or more objects generated by compilers and assemble them into a single executable program [48]. The ELF (Executable and Linkable Format [8]) file format is presented as an example of a program file format. An ELF file contains among others the following sections: Initialization routines (.init section). The program code (.text section). Program constants (.data section). Zero initialized constants (.bss section). The definition of these sections is used only by the linker and debugger. For the kernel all sections are mapped to segments. Segments are contiguous page-aligned regions in an ELF file which can directly be mapped to memory. Each segment provides information about its start address and size. Programs can be statically or dynamically linked. For statically linked programs all subroutines provided by external libraries are stored with the program code, resulting in a larger program file. This is in contrast to dynamically linked programs. They consist of the program data only, and all used libraries have to be supplied at program start resulting in smaller program files and a smaller memory footprint. Dynamic program data can be stored at two different places: on the heap or on the stack. The stack is typically located below the kernel address space and grows to lower addresses. The heap is located between the executable and the stack and it grows to higher addresses. Memory on the heap has to be explicitly allocated during runtime by the program using mmap, malloc, new or a similar function. The stack is used to store function parameters, the return address of a subroutine and automatic variables, i.e., variables which are only valid in the current context of a function. Additionally, the stack is used as a temporary storage for

22 10 CHAPTER 2. COMPUTER AND OPERATING SYSTEM ARCHITECTURE register values, for example when a subroutine is called. The programmer typically does not have direct access to the stack; instead, the stack is managed by the instructions generated by the compiler. The stack is normally located below the kernel address space and is growing to lower addresses. Environment variables and command line parameters are stored at the beginning of the stack Interrupts The operating system handles software and hardware interrupts differently. Hardware interrupts are handled as described above. Interrupt handling is getting more complex if the kernel is preemptive, i.e., it can be interrupted as well. Then, the operating system has to distinguish if a user program or the kernel itself has been interrupted. Software interrupts are mostly used to implement system calls. The kernel provides different interfaces for process-kernel communication, but they can all be reduced to system calls. Since direct access of the kernel code is not possible, another method to access kernel functions is required and serves two purposes: Privilege expansion by activating supervisor mode. Without supervisor mode direct access to the kernel address space is not possible. A standardized interface used for access control and parameter verification. The supervisor mode can be activated by raising an interrupt. Hence, a system call can be invoked by raising a dedicated software interrupt. The parameters passed to the kernel can be stored in registers and on the stack. These parameters may contain references in cases where additional data is located in user memory. A system call basically performs the same actions as an interrupt, but additionally the user-supplied data is verified. Otherwise, for example, it would be possible for a user process to access kernel memory by passing references to kernel memory with a system call Virtual Memory Handling The operating system uses virtual memory for different purposes: It allows a better memory utilization, because a virtual contiguous memory area can be mapped to arbitrary non-contiguous physical pages. The available physical memory can be extended by storing currently unused pages on external memory like hard disks and loading them on access back to RAM. This is called paging. The operating system is notified by an interrupt each time a page not located in physical memory is accessed. The same mechanism can be used for programs to load the parts to be executed from the program file on demand. The memory access permissions can be used to provide shared read-only memory used for example by shared libraries. Then the same physical pages are mapped to different virtual pages to save physical memory.

23 2.3. PROTECTION MECHANISMS 11 Virtual memory provides Copy On Write (COW) functionality. This is used for example when a program requests a large memory region initialized with zeros. Then the same physical page containing zeros can be mapped to several virtual pages with no write permission. On write access to one of the virtual pages the operating system allocates a physical page, copies the contents of the read-only page to the newly allocated one and maps it to the accesses virtual page with read and write permission. With execute permissions parts of the virtual address space can be explicitly reserved for program code. Memory regions containing data and marked non-executable like the stack or the heap can then not be used to execute injected program code using buffer overflows (see section 3.2). Physical memory consumption can be lowered by storing libraries used by different dynamically linked programs only one time in physical memory and mapping them into the virtual address space of each process. 2.3 Protection Mechanisms This section discusses protection mechanisms provided by modern UNIX-like 3 operating systems which do not require additional cryptographic units. Other operating systems provide similar protection mechanisms Multiuser Environment A multiuser operating system can assign different processes to different users where each user may have different permissions. Users can be assigned to groups, and group membership may give additional permissions. Permissions can be granted on the file system level or directly on the process level. Each file and each process belongs to at least one user, and this user is called the owner of the file or process File System File system permissions include read, write and execute access to files as well as permissions to create or delete files or directories. A user is not allowed to change access permissions of files if the owner of the file does not explicitly allow this. Therefore, file system permissions can be used to protect sensitive information from unauthorized user access Process Management The operating system ensures that each process running on a computer is separated from other processes on the same computer. Each process has its own virtual address space and has no access to memory belonging to other processes. Processes can typically be altered by the owner of a process but not by other users. This greatly improves system stability, because a failure in one process cannot affect other independently running processes on the same computer. System security is improved as well, because sensitive information like passwords stored in memory by a process cannot be read by another process. 3 UNIX is a multiuser operating system written by the Bell Laboratories.

24 12 CHAPTER 2. COMPUTER AND OPERATING SYSTEM ARCHITECTURE Administrator Access Most operating systems provide a fine-grained permission design, but there is at least one user which has full access to the system or which can grant itself full access. In the following, this user is called administrator. Due to its permissions, each administrator of a multiuser system has to be trustworthy, because he is able to bypass all operating system protection mechanisms Sophisticated Access Control The NSA (National Security Agency), for example, has developed a security extension for Linux, called SELinux 4. This extension adds mandatory access control mechanisms to Linux and therefore can be used to provide a better separation of information on Linux systems. However, SELinux is only a tool which has to be set up properly to provide enhanced security. Hence, sophisticated usage of access permissions can be an effective protection against unauthorized access as long as the administrator is trustworthy and there are no known weaknesses to bypass the operating system security. Possible weaknesses are described in the following chapter. 4

25 Chapter 3 Security Issues This chapter gives a brief overview of attacks to computer systems. Weaknesses in computer systems can be classified in architectural and implementation weaknesses. Architectural weaknesses are the worst case, because they cannot easily be fixed without losing compatibility to existing implementations. For example, the wireless encryption standard WEP (Wired Equivalent Privacy) has several vulnerabilities. The last discovered one allows sending data over a WEP protected link without knowing the secret key [6] or can be used to directly compute the key within 60 seconds [91]. WEP s vulnerabilities cannot be circumvented without loosing compatibility to existing hardware. Fortunately, these kinds of problems are rare compared to implementation errors. Therefore, implementation errors are the main target for adversaries, and some of them will be discussed below. 3.1 Hardware Access All operating system protection mechanisms can be bypassed if an adversary has direct physical hardware access. This kind of attack is not based on a real vulnerability, because most computer systems are intentionally not designed to prevent this kind of attack. For example, it is possible to bypass file system protection by directly accessing the hard disk by another operating system or by using direct block access. The easiest protection against this attack is a physical protection of the particular hardware. In cases where this is not possible other protection mechanisms using special hardware or cryptography can be used (please refer to chapter 5 for further information). When using cryptography to encrypt all sensitive data, other problems arise. Encrypted data can be accessed only after decrypting it with the corresponding key. Therefore, this key should not be stored on the same medium. Hence, encrypting data to prevent hardware-based attacks is effective, but mostly requires user interaction for decryption. In cases where sensitive data has to be transferred over busses hardware-based sniffers can be used. This technique has been used to break the security of the XBox [36] by capturing data transferred over a bus using an FPGA (Field Programmable Gate Array) to pre-process the bus signals. 13

26 14 CHAPTER 3. SECURITY ISSUES 3.2 Software-Based Attacks In cases where direct hardware access is not possible or too complicated software-based attacks are used. These attacks are mainly used remotely to break into computer systems and to increase privileges in cases where unprivileged access is already available. For this kind of attacks the attacker mainly tries to execute his code with the unprivileged user s permission to access user data or to exploit errors in system software to get administrator access. In the simplest case the adversary manages to let the user execute a malicious program. In cases where this is not possible, the adversary may try to exploit software implementation errors. Common attacks are: Buffer overflow: Here the attacker exploits errors where the programmer has forgotten to check if the length of the input data does not exceed the size of the buffer located on the stack which is intended to hold the data. In these cases the adversary tries to replace the return address stored on the stack with an address pointing to the overflowed buffer containing its own injected code. For example, this code can then be used to copy passwords or other sensitive information to an adversary. Several techniques exist to prevent buffer overflows [97, 99]. Even when buffer overflows cannot be prevented, the execution of the injected code can be prevented on some processors by marking pages containing the stack or the heap non-executable. Another approach is based on programming languages providing runtime size checks of buffers. Other programing languages like Java [30] or C# [15] do not even allow direct stack access. Heap overflow: These attacks are based on exceeding buffer sizes, too, but this time the buffers are located on the heap. The heap does not contain return addresses, but in most implementations the actual buffer and additional administrative data like the size of the buffer and pointers to the next and the previous buffer are stored in the same chunk of memory. The attacker can then try to overwrite one of the pointers to point to the return address on the stack. This overwrites the return address the next time the buffer is freed. There are several approaches to prevent this attack, like storing the buffer and administrative data at different positions [98]. Other approaches are based on stack and heap randomization where the beginning of the stack and heap is selected randomly on program start-up and runtime-allocated memory is selected randomly as well. Both techniques are, for example, part of the Grsecurity Linux kernel extension 1 and have been analyzed in [96]. They can help to defeat some attacks where the knowledge of absolute addresses is required for the attack to succeed. Double free vulnerability: This attack is similar to heap overflows, but this time the adversary exploits a request to free heap memory which already has been freed, thus overwriting administrative data structures. This attack can for example be prevented by using execution filtering approaches [65]. Format string attack: The function printf is used to print data based on a format string. The format string may contain wildcards which are evaluated during runtime to 1

27 3.3. SIDE CHANNEL ATTACKS 15 read or write to variables located on the stack. If an attacker is able to alter the format string, he may be able to overwrite stack contents to execute his own code. This error can be detected at compile time and the programmer can be informed. Temporary file vulnerability: Many programs create temporary files to write data. If the program does not choose an unpredictable file name, the attacker may try to place a symbolic link with the expected file name to force the program to write to an existing file the attacker cannot access but the attacked program has write access to. This attack can for example be used to modify the password database to get administrator access. Brute-force attacks: Most password-based authentication schemes are vulnerable to brute-force attacks. Here the adversary simply tries a huge number of user name and password combinations. Most current authentication schemes try to slow down brute force attacks by putting additional latencies in the authentication process and by disabling an account after a given number of unsuccessful authentication attempts. This countermeasure can be circumvented by an attacker by using the same password but different user names. This attack is applicable in cases of a large user base, because inexperienced users tend to use simple passwords. In cases where the attacker tries common words as passwords the attack is called dictionary attack. Brute force attacks can be used as well to find overflowed buffers or possible format string vulnerabilities by passing random or specially crafted data to programs until the program crashes. 3.3 Side Channel Attacks For this class of attack properties of a computer system or program are analyzed to get additional information. Therefore, this attack is not based on theoretical weaknesses. For example, this kind of attack can be used to distinguish illegal user names and illegal passwords in authentication systems if both cases result in a different time until an error is reported. The Secure Shell (SSH) was vulnerable to this kind of attack (see CVE ) 2. Side channel attacks based on cryptographic algorithms are described in section Program Analysis The attacks described above can be used to get access to computer systems or to extract cryptographic keys. Another kind of attack is used to analyze program code to be able to extract algorithms or to modify the program. This attack can be used to remove restrictions from programs like a copy-protection. In other cases software is analyzed to reverse engineer algorithms and protocols. Typical tools to analyze programs are debuggers and decompilers. A debugger is used to stop the program at arbitrary places to read or modify program variables and instructions and to analyze the program flow. However, when debugging a program without additional debug information, only the disassembled instructions without variable and function names are available, which makes further analysis difficult. Programs can further be analyzed by 2 Common Vulnerability Exposure (CVE) ID,