Coding Theory. Networks and Embedded Software. Digital Circuits. by Wolfgang Neff

Transcription

1 Coding Theory Networks and Embedded Software Digital Circuits by Wolfgang Neff

2 Coding (1) Basic concepts Information Knowledge about something Abstract concept (just in mind, can not be touched) Data Representation of information Signals on a medium (characters on paper) Coding Write down information Tree Coding Theory, W. Neff 2

3 Coding (2) Coding and communication are related Message Sender Encoding Channel Decoding Receiver Signal via Medium (e. g. Speech) Coding Theory, W. Neff 3

4 Coding (3) Coding and computer science are related IPO Model Input Encoding Processing Decoding Output Storage Input Encoding Processing Decoding Output Coding Theory, W. Neff 4

5 Coding (4) Coding maps the elements of two sets A := {,, } B:= {Car, Tree, House} Car Tree House Information Representation Coding Theory, W. Neff 5

6 Coding (5) Computers represent information by numbers Information Number Coding Theory, W. Neff 6

7 Coding (6) Example A door has the three states open, closed, locked The house has four doors We will code the state of the doors of the house Binary coding of the states of a door Hot one coding of the state of a door Binary coding of the state of the house I IV II III Coding Theory, W. Neff 7

8 Coding (7) Binary coding of the states of a door We need two bits to encode the state of the door State First bit Second Bit Open 0 0 Close 0 1 Locked 1 0 Unused 1 1 We need eight bits for the state of the house Door 1 Door 2 Door 3 Door Coding Theory, W. Neff 8

9 Coding (8) Hot one coding of the state of a door We use one bit per state of the door State First bit Second Bit Third Bit Open Close Locked We need twelve bits for the state of the house Door 1 Door 2 Door 3 Door Coding Theory, W. Neff 9

10 Coding (9) Direct coding A door has three states O: open C: closed L: locked Our house has 3 4 = 81 states We need seven bits to encode all states 2 7 > 81 There are many don tcare terms n Door I Door II Door III Door IV 0 O O O O 1 O O O C 2 O O O L 3 O O C O 4 O O C C 5 O O C L 6 O O L O 7 O O L C 8 O O L L 9 O C O O Coding Theory, W. Neff 10

11 Numeral Systems (1) Numeral systems encode numbers IIIII IIIII IIIII IIIII IIIII IIIII IIIII IIIII Information Decimal Representation XXV V X 五 二十五 十 Roman Representation Chinese Representation Coding Theory, W. Neff 11

12 Numeral Systems (2) Example: Decimal System Base: 10 (number of digits) Digits: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 The value of a digit depends on its position Example 2012 dec = dec = dec = 2012 val (value) Coding Theory, W. Neff 12

13 Numeral Systems (3) Values of numeral representation in general a n a n 1 a 1 a 0 = n i=0 a i b i a i : the digit at position i b: the number of digits Example dec = i=0 a i 10 i = Coding Theory, W. Neff 13

14 Numeral Systems (4) Basic concepts Base: number of digits 10 (we use the decimal system as example) Digits: representation of the digits 0,1,2,3,4,5,6,7,8,9 Value of a representation 2012 dec = = 2012 val Range: number of possible values = blength of number 4 digits (0 9999) = 10 4 different values Coding Theory, W. Neff 14

15 Numeral Systems (5) Basic concepts (continued) Representation 2012 val = = 201 remainder = 20 remainder = 2 remainder = 0 remainder 2 Reading Direction The algorithm ends here A value of 2012 is represented by 2012 dec Coding Theory, W. Neff 15

16 Numeral Systems (6) The hexadecimal system Base: 16 Digits: 0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F Value 2012 hex = val Range 4 digits (0 FFFF hex ) = 16 4 (65536) different values Coding Theory, W. Neff 16

17 Numeral Systems (7) The hexadecimal system (continued) Representation 2012 val = = 125 remainder 12 (C) = 7 remainder 13 (D) 7 16 = 0 remainder 7 A value of 2012 is represented by 7DC hex Coding Theory, W. Neff 17

18 Numeral Systems (8) The binary system Base: 2 Digits: 0,1 Value 1011 bin = val Range 4 digits ( bin ) = 2 4 (16) different values Coding Theory, W. Neff 18

19 Numeral Systems (9) The binary system (continued) Representation 57 val = 57 2 = 28 remainder = 14 remainder = 7 remainder = 3 remainder = 1 remainder = 0 remainder 1 A value of 57 is represented by bin Coding Theory, W. Neff 19

20 Binary Numbers (1) Computers use the binary system Because they are digital circuits Therefore they are based on Boolean algebra The binary digits are called bits Some groups of bits have special names Nibble (4 bits), byte (8 bits), word (16 bits or more) Data is encoded as binary numbers Numbers, text, images etc Coding Theory, W. Neff 20

21 Binary Numbers (2) Fixed-length binary numbers The length of binary numbers is limited The first and the last digit have a name Rightmost bit: LSB (Least Significant Bit 2 0 =1) Leftmost bit: MSB (Most Significant Bit 2 7 =128) Bit Value Number MSB LSB Coding Theory, W. Neff 21

22 Binary Numbers (3) Use exponential calculus to get the range 8 Bits: 2 8 = 256 values Range is Bits: 2 16 = = * = Range is Bits: 2 32 = = *1000*1000*4 4,000,000, = 4,294,967, Coding Theory, W. Neff 22

23 Binary Numbers (4) Nibbles (4 bits) can easily converted to hex Examples C D A 1110 E B 1111 F bin 00 hex bin BC hex bin 64 hex bin FF hex Coding Theory, W. Neff 23

24 Binary Numbers (5) Base64 coding uses 64 digits Used in internet communication Three bytes are encoded in four ASCII chars 1 st Byte 2 nd Byte 3 rd Byte st Character 2 nd Character 3 rd Character 4 th Character Coding Theory, W. Neff 24

25 Binary Numbers (6) The coding table of Base64 00 A 08 I 10 Q 18 Y 20 g 28 o 30 w B 09 J 11 R 19 Z 21 h 29 p 31 x C 0A K 12 S 1A a 22 i 2A q 32 y 3A 6 03 D 0B L 13 T 1B b 23 j 2B r 33 z 3B 7 04 E 0C M 14 U 1C c 24 k 2C s C 8 05 F 0D N 15 V 1D d 25 l 2D t D 9 06 G 0E O 16 W 1E e 26 m 2E u E + 07 H 0F P 17 X 1F f 27 n 2F v F / Coding Theory, W. Neff 25

26 Binary Numbers (7) Rapid conversion Bin Dec Write 1 above the LSB. Double until MSB is reached. Add all values where the digit is 1. Example: bin = = 179 dec Coding Theory, W. Neff 26

27 Binary Numbers (8) Rapid Conversion Dec Bin Double 1 until it is greater than the number to convert. Try to subtract these values. If it is possible the digit is 1 otherwise it is 0. Example: 179 dec = bin Coding Theory, W. Neff 27

28 Calculus (1) Binary Addition Same method as decimal addition Noteworthy facts: Leading Zeros Carry Overflow 1 st Number nd Number Carry Result Carry Leading Zero No Overflow Coding Theory, W. Neff 28

29 Calculus (2) Binary Addition Example Leading Zero Fixed bit length Bit Flag st Number nd Number Carry Result Carry An overflow is stored in the carry flag Coding Theory, W. Neff 29

30 Calculus (3) Binary Addition We encounter the following situations 1 st Bit nd Bit Carry Result dec bin Carry Result Coding Theory, W. Neff 30

31 Calculus (4) Binary Subtraction Subtraction works with borrowing Borrowed from next bit 1 st Bit bin dec nd Bit Borrow Result Coding Theory, W. Neff 31

32 Calculus (5) Binary Multiplication Same method as decimal multiplication Example: 13*5 = 65 = bin * Coding Theory, W. Neff 32

33 Calculus (6) Shifting A left shift doubles the value of the number *2 = 10 Left Shift A right shift halves the value of the number 10 / 2 = Right Shift Coding Theory, W. Neff 33

34 Calculus (6) Binary Division Same method as decimal division. Sometimes replaced by shifting because of speed. Example: 2,54*15 = 254*15/100 = 38 Multiply with an appropriate power of two (2 7 =128) 2,54 = 2,54*2 7 / 2 7 = 325 / 2 7 Multiply with numerator and then shift 2,54*15 = 325*15 / 2 7 = 4875 / 2 7 = 4875 >> 7 = 38 This is only an approximation but may be very fast» 254/100 = 2,540 vs. 325/128 = 2,539» E. g. AVR: with division 250 cycles vs. 16 cycles with shift Coding Theory, W. Neff 34

35 Signed Numbers (1) Fixed-length binary numbers have a range. If we exceed the range we restart from zero. Therefore the number ray becomes a circle. length = 3 bits Coding Theory, W. Neff 35

36 Signed Numbers (2) We get negative numbers if we go beyond zero. 2 n numbers 2 n-1 numbers 2 n-1 numbers Coding Theory, W. Neff 36

37 Signed Numbers (3) We have to sacrifice positive numbers to get negative numbers. Negative numbers have its MSB is set. The range of negative number is Positive number range: n-1-1 Negative number range: n-1 n: bit length of number Coding Theory, W. Neff 37

38 Signed Numbers (4) Two s complement Used to negate a number Complement all bits Add a value of one Number Complement Carry Result Coding Theory, W. Neff 38

39 Signed Numbers (5) Rapid conversion Bin signed Dec Negate the MSB Add the rest Example: B3 hex = -77 dec bin = = -77 dec Coding Theory, W. Neff 39

40 Signed Numbers (6) Rapid conversion negative Dec Bin Add number to range Convert result to binary Example: -77 dec = B3 hex Bit length = 8 Range: 2 8 = = dec = bin = B3 hex Coding Theory, W. Neff 40

41 Fixed-Point Numbers (1) Fixed-point numbers are numeral systems with a negative index: a n a n 1 a 1 a 0. a 1 a m = n i= m a i b i There are m positions after the decimal point There are n+1 positions before the decimal point Coding Theory, W. Neff 41

42 Fixed-Point Numbers (2) Example: a base 16 fixed-point number Number: hex Value: hex = / / You have to round to preserve the number of positions after the decimal point Coding Theory, W. Neff 42

43 Fixed-Point Numbers (2) Example: a base 16 fixed-point number Representation = = 20 hex Division by 1 / = = 1 remainder = 1 remainder = 14 remainder = 11 remainder 0.52 Stop when you have got enough positions The value of is 20.11EB hex Round-off errors are almost inevitable Reading Direction Coding Theory, W. Neff 43

44 Fixed-Point Numbers (3) Binary fixed-point numbers Notation of binary fractional numbers s(n.m): signed binary fractional - n bits before the decimal point and m bits after. MSB is sign bit. u(n.m): unsigned binary fractional n bits before the decimal point and m bits after it. No sign bit. Binary fractional calculus d(n.m) + d(n.m) = d(n+1.m) = d(n.m) and carry flag d(n 1.m 1 ) d(n 2.m 2 ) = d(n 1 +n 2.m 1 +m 2 ) Coding Theory, W. Neff 44

45 Fixed-Point Numbers (4) d(1.n) binary fractional numbers often used in digital signal processing (DSP) Multiplication modifies the number format d(1.n) d(1.n) = d(2.2 n) A left shift is necessary to obtain d(1.2 n+1) An overflow might occur indicated by the carry flag Some microcontroller have special instructions AVR: FMUL Fractional Multiply Unsigned s(1.7) s(1.7) s(1.15) Coding Theory, W. Neff 45

46 Fixed-Point Numbers (5) Rapid conversion d(n,m) Dec Start with 1 at the decimal point. Double it until you reach the MSB. Half it until you reach the LSB. Example: s(1.7) = B3 s(1.7) -1 1 / 2 1 / 4 1 / 8 1 / 16 1 / 32 1 / 64 1 / B3 s(1.7) = / / / / 128 = dec Coding Theory, W. Neff 46

47 Fixed-Point Numbers (6) Rapid Conversion Dec Bin Choose a binary fractional format and write its values above the positions. Try to subtract these values. If it is possible then the digit is 1 otherwise it is 0. Example: dec = s(1.7) / 2 1 / 4 1 / 8 1 / 16 1 / 32 1 / 64 1 / Coding Theory, W. Neff 47

48 Fixed-Point Numbers (7) Example: sin(0 2π) in s(1.7) fractional binaries n x 0, sin(x) s(1.7) 00 hex 30 hex 5A hex 76 hex 7F hex 76 hex 5A hex 30 hex 1.0 is not in the range of s(1.7). We use 7E instead. This is called saturation. n x sin(x) s(1.7) 00 hex CF hex A5 hex 89 hex 80 hex 89 hex A5 hex CF hex Coding Theory, W. Neff 48

49 Floating-Point Numbers (1) Do not have a fixed decimal point position. Are expressed by a mantissa and an exponent. Are usually normalized: Only one digit before the decimal point dec dec 10 1 (normalized) Floating-point numbers are standardized: IEEE 754: Standard for Floating-Point Arithmetic Commonly used: single and double precision Coding Theory, W. Neff 49

50 Floating-Point Numbers (2) IEEE 754 floating-point binary numbers Format of a half precision floating-point 1 st Byte 2 nd Byte S C C C C C F F F F F F F F F F Sign Character Fraction Value of a half precision floating-point = (-1) S Mantissa 2 Exponent Bias = 15 Exponent = Character Bias ( ) Character and reserved for ± and NaN (Not a Number) Mantissa = 1.Fraction bin Coding Theory, W. Neff 50

51 Floating-Point Numbers (3) Example: value of half precision B248 hex Value: 1 st Byte 2 nd Byte S C C C C C F F F F F F F F F F Exponent = Character - Bias = = 1 Mantissa = 1. + Fraction = = 1, Value = (-1) S Mantissa 2 Exponent = -1 1, Half precision B248 hex = dec Coding Theory, W. Neff 51

52 Floating-Point Numbers (4) Example: π as a half precision binary number Representation = bin bin 2 1 Character = Exponent + Bias = = 16 = bin Fraction = bin 1 st Byte 2 nd Byte S C C C C C F F F F F F F F F F π = 4248 hex represented as a half precision binary Coding Theory, W. Neff 52

53 Floating-Point Numbers (5) Characteristics of IEEE 754 floating-points Characteristic Half precision Single precision Double precision Size 16 bit 32 bit 64 bit Character 5 bit 8 bit 11 bit Fraction 10 bit 23 bit 52 bit Exponent Bias Coding Theory, W. Neff 53

54 Characters (1) Characters are mapped to codes and fonts Coding C A Ä ð B A B C Ä A B Fonts Fonts describe the graphic appearance of characters Character C Coding Theory, W. Neff 54

55 Characters (2) Characters are encoded by numbers. Characters are displayed by fonts. The mapping need not be total. There can be characters without code There can be characters not present in a font There are many different fonts. Times New Roman Harlow Solid Italic Coding Theory, W. Neff 55

56 Characters (3) ASCII: the ancestor American Standard Code for Information Interchange Published by ASA in 1963 Uses 7 bits per character Contains Control characters Letters and Numbers Punctuation and special characters Coding Theory, W. Neff 56

57 Characters (4) The ASCII code table ASCII Lower Hex Digit A B C D E F 0 NUL SOH STX ETX EOF ENQ ACK BEL BS HT LF VT FF CR SO SI Higher Hex Digit 1 DLE DC1 DC2 DC3 DC4 NAK SYN ETB CAN EM SUB ESC FS GS RS US 2! " # $ % & ' ( ) * +, -. / : ; < = >? A B C D E F G H I J K L M N O 5 P Q R S T U V W X Y Z [ \ ] ^ _ 6 ` a b c d e f g h i j k l m n o 7 p q r s t u v w x y z { } ~ DEL Coding Theory, W. Neff 57

58 Characters (5) Latin-1: One nation one character set Deficiencies of ASCII Suitable for English, only Other languages use additional symbols Only 7 bits of a byte used ISO 8859 Uses all 8 bits of a byte for coding 128 additional symbols One standard per region -1: Western European -2: Central European -7: Latin/Greek -11: Latin/Thai Coding Theory, W. Neff 58

59 Characters (6) The ISO code table: Latin 1 Latin Higher Hex Digit 8 9 Lower Hex Digit A B C D E F same as ASCII unused A NBSP ª «B ± ² ³ µ ¹ º» ¼ ½ ¾ C À Á Â Ã Ä Å Æ Ç È É Ê Ë Ì Í Î Ï D Ð Ñ Ò Ó Ô Õ Ö Ø Ù Ú Û Ü Ý Þ ß E à á â ã ä å æ ç è é ê ë ì í î ï F ð ñ ò ó ô õ ö ø ù ú û ü ý þ ÿ Coding Theory, W. Neff 59

60 Characters (7) Unicode: One code for all nations Consistent encoding of all writing system Comprises modern and historic scripts Consists of more than characters Uses 32 bits for coding Standardized as ISO/IEC Special formats for storage (UTF) Coding Theory, W. Neff 60

61 Characters (8) Unicode is or organized in planes. Diacritics and Ligatures à, å, ä, æ Non-Latin writing systems العربية Arabic: Logographic writing system Chinese: 汉语 Special purpose characters Mathematics:,,,, ±,, Coding Theory, W. Neff 61

62 Characters (9) Example: Greek and Coptic (U+0370-U+03FF) Code ΐ Α Β Γ Δ Ε Ζ Η U+0390: ΐ Greek small litter iota with U+0391: Α Greek capital letter alpha U+0392: Β Greek capital letter beta U+0393: Γ Greek capital letter gamma U+0394: Δ Greek capital letter delta Coding Theory, W. Neff 62

63 Characters (10) Unicode Transformation Format Used to store and process Unicode characters UTF-16 Used in Windows, OS X, Java,.Net UTF-8 Used to store Unicode characters in files Aims to reduce the size of Unicode files Variable length 8 bit code ASCII files are valid UTF-8 files Coding Theory, W. Neff 63

64 Characters (11) UTF-8 Coding procedure Unicode Character U+ xxxx xyyy yzzz zzzz If each x and each y is 0 The code is 0zzz zzzz (n. b. this is the ASCI code) If each x is 0 The code is 110y yyyz 10zz zzzz Otherwise The code is 1110 xxxx 10xy yyyz 10zz zzzz Coding Theory, W. Neff 64

65 Characters (12) UTF-8 Coding examples y U Each x and each y is 0 y ä U+00E Each x is 0 ä U+20AC Coding Theory, W. Neff 65

66 Important Expressions (1) numbering system digit range decimal system binary system hexadecimal system bit length MSB / LSB Zahlensystem Ziffer Wertebereich Dezimalsystem Binärsystem Hexadezimalsystem Bitbreite MSB / LSB Coding Theory, W. Neff 66

67 Important Expressions (2) carry leading zero overflow two's Complement floating point number character encoding scheme internationalization (i18n) Übertrag Führende Null Überlauf Zweikomplement Gleitkommazahl Zeichen Kodierungsschema Internationalisierung Coding Theory, W. Neff 67