CMU MSP 36601: Computer Hardware and Data Representation

Transcription

1 CMU MSP 36601: Computer Hardware and Data Representation H. Seltman, September 6, Warm-up in R a. > == 0.3 [1] FALSE b. > 1e15+1e32 == 1e32 [1] TRUE c. > 1e-324 == 0 [1] TRUE d. > 1e-324 / 1e-325 [1] NaN e. > as.integer(c(2e9, 3e9)) [1] NA Warning message: NAs introduced by coercion to integer range f. > == [1] TRUE g. > exp(709) [1] e+307 h. > exp(710) [1] Inf 2. Hardware a. Roughly, your computer consists of: i. A clock whose ticks coordinate everything else. The speed of the computer is rated by the clock speed, e.g, 1.6GHz is 1.6 x 10 9 cycles per second. ii. A Central Processing Unit that interprets instructions (machine language) and performs basic data manipulation and calculation. The CPU consists of: 1. A Control Unit which controls everything else 2. Registers (on-cpu memory) that hold numbers and memory addresses (the size of the registers is the size of the CPU, e.g., 32 vs. 64 bits) 3. An Arithmetic-Logic Unit interprets instructions and performs integer and bit-wise operations 4. A Floating-Point-Unit which holds non-integer values in floating point registers and performs mathematical operations on them iii. Busses (more like highways) that carry the flow of data between parts of the computer iv. Non-volatile Read-Only Memory containing the Basic Input Output System which is the small, fixed, initial set of instructions the computer needs to load in the full operating system (usually from a disk). v. Volatile Random-Access Memory that is the working memory of the computer (often supplemented by cache memory more closely associated with the CPU) vi. Non-volatile user-writable storage (hard drive or solid-state storage) for storing programs and data when the computer is powered off or the RAM capacity is exceeded. Programs/data must be moved to RAM to be run/used. vii. Input/output devices: keyboard, mouse, video display, network adaptor viii. Optional additional hardware, e.g., USB port, expansion cards (such as a TV tuner) 1

2 b. Useful facts based on hardware knowledge i. Based on the CPU make and model, very different machine language may be supported. Therefore, programs compiled on one machine may not run on another. ii. The most basic operations in the CPU are: interpretation of machine language instructions in order; conditional branching to alternate instruction points; transfer of data into and out of registers; and performance of logical and mathematical operations on data in registers. iii. Modern machine language instruction sets have special hardware capabilities to perform repeated tasks on contiguous streams of data. Therefore, depending on how matrix data is stored, there may be a large difference in speed between row vs. column operations. iv. Data in RAM is individually addressable and very fast. (Cache memory is significantly faster.) Data from hard drives or solid-state storage are addressed in blocks, which are transferred to RAM before being used. Hard drive access is typically 10 to 50 times slower than RAM access. Programs that intelligently store data that will be used soon in cache are faster than those that do not. v. Operating systems usually implement virtual memory to allow more and bigger programs to run than would be possible with the available RAM. This is based on mapping some hard drive space as if it were RAM. When the fake RAM is needed, it is swapped with some real RAM, causing a large delay. When this happens frequently, it is called thrashing. 3. Software a. The Operating System provides services and coordination (via interfaces) i. Program execution ii. Multi-tasking and interrupts iii. Memory management iv. Disk management (file system) v. Device use and sharing (e.g., keyboard and display) vi. Security vii. Networking viii. The interface is a GUI (e.g., Windows), command line, and/or interrupts b. Programs must request services from the operating system, and this is relatively slow. c. Programs writing i. Programs may be written in machine language and then directly executed ii. Practically, very low-level programs are written in assembly language, which adds tools such as address names, instruction mnemonics, and macros to the basic machine language. iii. Compilers are programs that input their own language and output machine language (including calls to the operating system where needed) iv. Interpreters are programs that input their own language and output machine language that will be executed immediately or an interpreter-specific intermediate bytecode that can be run immediately or later using a virtual machine specific to that interpreter on your or a different computer. 2

3 4. Data Representation a. It is true that everything is bits! Your RAM or hard drive is best thought of as a single long string of bits (e.g., 16GB of RAM is 16*10 9 *8 bits). Depending on the device, bits are physically represented as electrical or magnetic patterns with two states. All further discussion need only focus on the arbitrary names 0 and 1. b. Bits can be thought of in aggregation: i. Nibble: 4 bits ii. Byte: 8 bits iii. Word: usually 16, 32, or 64 bytes c. Math review of numeric bases i. In base b, digits go from 0 to b-1 (b of them), columns from right to left are 1 s, b s, b 2 s, b 3 s, etc., and 10 b always represents b, i.e., 1 b s. ii. Examples Base Digits ABCDEF A 10 2B d. For convenience we can represent a bit pattern in several ways other than 0/1: i. Decimal: ( ) = 93 ( or 64+3*8+5 or 5*16+13) ii. Group by 3s is octal: = 135 octal (1, 2+1, 4+1) [strtoi("135",8) in R] iii. Group by 4s is hexadecimal: = 5D (4+1, 8+4+1) [0x5D in R 5Dx elsewhere] e. Nerd joke: Why do computer nerds confuse Halloween with Christmas? (see next page for answer) f. Central fact: n bits can represent 2 n things g. **** What a set of bits represents depends entirely on context **** h. Context: Machine language instructions are arbitrary mappings with a variable number of bytes A small made-up example of a set of machine language instructions: : stop to : increment registers A through D to : decrement registers A through D : A <- A + B : A <- A B : A <- A * B : A <- A / B aaaa aaaa aaaa aaaa: load A from aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa: store A in to aaaa aaaa aaaa aaaa aaaa aaaa: jump ahead aaaa aaaa instructions if A== aaaa aaaa aaaa aaaa: jump to aaaa aaaa aaaa aaaa if A==0 Etc. 3

4 i. Nerd joke answer: 31OCT = 25DEC j. Context: English text coded as ASCII (0b bbb bbb, written in octal) A : 101 a : 141 : 40 0 : 30,-. : LineFeed: 10 CarriageReturn: 13 Abde 4321 Baa. : k. Context: Text as UTF-8 Unicode 1. Format: 0bbbbbbb or 110bbbbb 10bbbbbb or 1110bbbb 10bbbbbb 10bbbbbb or 11110bbb 10bbbbbb 10bbbbbb 10bbbbbb 2. Possible values: 127 single-byte codes two-byte codes three-byte codes four-byte codes (135,268,479 possible characters and symbols) 3. Backward compatible with 7-bit ASCII and self-synchronizing 4. Allows accents, Hangul alphabet, Chinese ideographs (~80K), math symbols, etc. 5. Example: Abstractly: 2*β": ASCII hex 22, 2A plus UTF-8 U+03B2 In R: "2*\u3B2" [3=_011, B=1011, 2=0010] b bb/bb 10bb/ bbbb l. Context: Integers 1. Short (2 bytes) representing 2 16 values i. (INT) Signed: -32,768 to +32,767 using two s complement ii. (UINT) Unsigned: 0 to 65, Long (4 [or 8] bytes) representing 2 32 [or 2 64 ] values i. (LONG) Signed: -2,147,483,649 to +2,147,483,648 ii. (ULONG) Unsigned: 0 to 4,294,967, long long (twice the length of long) 2 10 = *10 18 m. Context: memory address (pointer) generally 32 (4GB RAM) or 64 bits (4 million TB) n. Context: Simplified bit-mapped 256 level grey-scale image (max 65,535 x 65,535) UINT rows; UINT cols; BYTES for row 0; ; BYTES for last row o. Context: Highly simplified file system header UINT number of files; repeat for each file{short flags (executable, write protected, etc.); LONG location of file on disk; BYTES*32 file name in ASCII} p. Context: Excel document Complex proprietary format, partially decodable q. Context: R objects (rough idea) TYPE (NULL vs. symbol vs. pairlist vs. function vs. environment ) LENGTH FLAGS for debug, garbage collection, etc. POINTER TO ATTRIBUTE NAME/VALUE pairs POINTER TO ACTUAL DATA 4

5 5. Miscellany r. Math review of floating point represents 6* * * * * * Of course, irrational numbers cannot be represented exactly in finite digits 3. 1/3=0.333 is rational, but cannot be represented exactly in finite digits in base 10, but is exactly equal to 1*3-1 in base Any irreducible fraction whose denominator is not factorable by 2 and 5 cannot be represented in finite digits in base is 1*2 2 +1*2 0 +1*2-2 +1*2-3 = 4+1+1/4+1/8 = 5 3/8 = Any irreducible fraction whose denominator is not factorable by 2 cannot be represented in finite digits in base 2. E.g., 1/5 = Scientific notation in base 2: e.g., = * 2 3 (note that this mantissa only requires 5, not 6, bits of storage). s. Context: IEEE Standard #754 Floating Point numbers 1. most common version is binary64 (double precision) 2. one bit is reserved for the sign 3. 52(53) bits for the mantissa gives significant decimal digits bits for the exponent gives a range of to ± special patterns are reserved for +, -, and Not A Number a. Typically, programs have access to RAM in two forms: stacks and heaps b. Stacks are fixed size, First-In-Last-Out mechanisms for storage. Data goes in and out via push and pop. Stacks play a key role in running functions. i. When one function transfers control to another function and expects to regain control later, the calling function pushes its next code address onto the stack before setting the program pointer to the start of the called function. When any function completes by reaching its return instruction, it transfers control to the address on the top of the stack, which automatically functions as return to caller, no matter which function called the current function and no matter how deep. ii. In programs like C and R, the calling function pushes all of the arguments for the function being called onto the stack after pushing the return code address onto the stack. In this system, all functions start by popping their arguments off of the stack, leaving the return address of the calling function on top. One language choice is to place (a copy of) the value of the argument on the stack ( call by value ) in which case the calling function cannot change the original value. An alternate choice is to place the address of the argument on the stack ( call by reference ), in which case the calling function does have the opportunity to change the original value. iii. Functions designed to return values push those values on the stack before returning so that the calling function can access them. 5

6 c. Heaps are blocks (really linear segments) of RAM that the program asks for. If the request is successful, the operating system return the starting address to the program. i. The process is usually called memory allocation. The program asks for a block of a certain size, and the operating system assigns the block to the program or reports that a block of that size is not available. ii. A program may explicitly release ( free ) memory that it no longer needs back to the operating system. Programs that do a lot of allocation and freeing may leave the available memory fragmented (analogous to disk fragmentation), and some mechanism for garbage collection may be required periodically, when a large enough block is not available and/or when the program explicitly requests it. This may involve massive movements of data and be very slow. iii. Also note that if, e.g., R asks for a numeric vector of size 1024, and then you run vec = c(vec, 2.5), R may need to ask the OS to allocate a larger block, copy all 1024 bytes to the new memory block, copy the 2.5 on to the end, and then free the original block. This may be very slow. Therefore, whenever possible allocate the full vector length you need up front. d. Typically, the stack and the heap grow towards each other resulting in a combined limit, but the OS may allow separate limits. Exceeding the stack size gives a stack overflow error. Exceeding the heap gives an error message such as out of memory. e. In R, memory.size() shows MB of heap allocated, memory.limit() shows total MB of heap available, and object.size(a) shows the size of object a. See limit in Unix. 6. Concluding remarks a. On the vast majority of computer systems, anything other than integer or FP vectors is non-fundamental. NA is also non-fundamental. b. Testing equality of FP values may be dangerous. In R, all.equal() appropriately tests for approximate FP equality (but returns TRUE or a string detailing differences). > x=0; for (i in 1:10) {x=x+0.1; if (x==0.3) print(x)} > x=0; for (i in 1:10) {x=x+0.1; if (istrue(all.equal(x, 0.3))) print(x)} [1] 0.3 c. Pay attention to heap memory allocation to improve efficiency. d. You should now understand the difference between human-readable and non-human readable files. You may be able to use published file format information or deduction to read non-human readable files. (Start with tools like FileAlyzer in Windows or hexdump -C" in Linux.) e. Beware of numeric overflow and underflow. f. Realize that compiled programs tend to be faster than interpreted ones. g. In a human readable file, is 10 bytes long; when read into R, SAS, etc. it will programmatically be converted to its bit FP pattern (always 8 bytes). 6