Process model formulation and solution, 3E4 Computer software tutorial - Tutorial 3

Size: px
Start display at page:

Download "Process model formulation and solution, 3E4 Computer software tutorial - Tutorial 3"

Transcription

1 Process model formulation and solution, 3E4 Computer software tutorial - Tutorial 3 Kevin Dunn, dunnkg@mcmaster.ca October 2 Tutorial solutions: Elliot Cameron. Tutorial objectives Understand computer representation of decimal numbers and chemical engineering data. Question [] Convert from binary to decimal: (.) 2 Convert from decimal to binary: (32.625) Convert from decimal to binary: (.2). (.) 2 Therefore 2. (32.625) x2 }. x2 x2 2 {{ x2 3 x2 4 x2 5 } = (.) 2 = (.65625) Start to the left of the decimal point (i.e. Representing 32 in binary) log 2 (32) = 5 This helps us determine how many binary digits are required Rounding 5 up to 5 (obviously) Therefore 2 5 = 32 (32 32 = ) x2 5 + x2 4 + x2 3 + x2 2 + x2 + x2 = () 2 = (32) Moving to the right of the decimal place : 2 =.5 ( =.25) 2 2 =.25 (.25 <.2) : therefore we move to the next digit 2 3 =.25 ( =.) x2 5 +x2 4 +x2 3 +x2 2 +x2 +x2 +x2 +x2 2 +x2 3 = (.) 2 = (32.625)

2 3. (.2) Start to the left of the decimal point (i.e. Representing in binary) () = () 2 Moving to the right of the decimal place 2 =.5 (.2 <.5) : therefore we move to the next digit 2 2 =.25 (.2 <.25) : therefore we move to the next digit 2 3 =.25 (.2.25 =.75) 2 4 =.625 ( =.25) 2 5 =.325 (.25 <.325) : therefore we move to the next digit 2 6 =.5625 (.25 <.5625) : therefore we move to the next digit 2 7 =.7825 ( =.46875) 2 8 = ( =.7825) 2 9 = (.7825 <.95325) :move to the next digit 2 = (.7825 < ) : move to the next digit 2 = ( = ) 2 2 = ( = )... x2 + x2 + x2 2 + x2 3 + x2 4 + x2 5 + x2 6 + x2 7 + x2 8 + x2 9 + x2 + x2 + x2 2 (.) 2 = (.2) What this questions aims to show you is, just as in the decimal system, there are numbers that cannot be represented finitely in binary (think /3 in decimal notation). Question 2 []. Write, in binary form, the representation for the most negative floating point number that can be stored in an 8-bit word, using bit for the sign, 3 bits for the signed exponent and the remainder for the significand (in normalized form). 2. What is the decimal equivalent of this number? 3. What is the machine number next to this one (both in binary and in decimal please)? 4. Calculate the maximal interval-to-value ratio around these two values. Does it agree with the limit for theoretical machine precision? 5. As which machine number would be stored if the machine used (a) chopping, or (b) rounding?. We start by recalling that normalized scientific notation assumes that the significand starts with an implied leading digit of (i.e. it falls between the following bounds): b m < 2

3 Where m = significand b = base e = exponent Therefore the most negative 8-bit binary number with bit for the sign, 3 bits for the signed exponent, and 4 bits for the normalized significand is: sign exponent ( x2 + x2 2 + x2 3 + x2 4) x2 (x2 +x2 ) =.9375x2 3 significand 2. sign exponent =.9375x2 3 =.9375x8 = 7.5 significand 3. The next closest machine number is the next physically representable number in the 8-bit floating point system. Since no more space exists above the current significand we must go down in magnitude. Therefore: sign exponent =.875x2 3 = 7. significand 4. We start by estimating the theoretical machine precision: t = 4 : number of significant digits in the significand β = 2 : base of number system ɛ mach = β t = 2 4 =.25 Next we test the maximal interval-to-ratio value on either side of the values from part (a)/(b) and (c) x x = ( 7.) ( 7.5) ( 7.) = ɛ mach x = ( 7.) ( 7.5) ( 7.5) = ɛ mach It is easy to see above that the maximal interval-to-value ratios agree with the upper ɛ mach limit 5. As observed in parts (a) - (c) the two closest machine numbers to are -7. and Therefore the effect of chopping (rounding towards ) and rounding (rounding to the closest avaiable machine number) would result in the same floating point value: -7.. Question 3 [.5]. Increasingly we are seeing cameras being used in chemical processes to monitor and control the process, especially systems that deal with foods and solid products. How much space, in kilobytes, is required to store a digital photo with 64 rows and 48 columns of pixels and 3 layers (red, green and blue) using uint8 integer representation? 2. Computer systems are used to archive data from each electronic measurement, such as temperature, pressure, flow measurements, etc. Each measurement is called a tag. At your plant, you wish to store 6,525 tags, recorded once per second and stored in double precision. How much space would be required on the company s server, in terabytes, to store a single copy of year of data? What difference does it make to store the data in single precision? 3

4 3. How many data points can you store in double precision in 5 megabytes of RAM? (For example, MATLAB on a 32-bit Windows Vista machine cannot create arrays greater than 428 megabytes.) Use the fact that: 24 bytes = kilobyte 24 kilobytes = megabyte 24 megabytes = gigabyte 24 gigabyte = terabyte = 2 4 bytes. Recall that the uint8 integer representation refers to an unsigned integer with no exponential term. Therefore it refers to an integer that can store a value from As such this question is simply asking us how much memory would be required to store a 64x48 pixel image using the 256/256/256 RGB colour notation. Now, if each pixel in each of the RGB matrices is represented by a uint8 then each value will logically take up 8 bits of memory. If each of the three colour matrices contains 64x48 values and we assume a standard 8 bit byte then the total space required to store the photo would be: Space = 3 64 }{{ 48 } 8 }{{ bits } = bits byte kilobyte = 9 kilobytes 8 bit 24 byte rgb pixels uint8 2. Recall that a standard floating point double precision number takes up 64 bits (8 bytes) of memory. We start this problem by calculating the number of values to be stored in one year. Values = 6, 525 values s 6 s 6 min 24 hr 365 day = 52, 32, 4, values min hr day year year Therefore the memory requirement is: Memory = 52, 32, 4, values year 8 bytes value kilobyte 24 byte megabyte 24 kilobyte gigabyte 24 megabyte terabyte = 3.79 terabytes 24 gigabyte A floating point single precision value takes up exactly half as much space as a floating point double precision number (i.e. 32 bits = 4 bytes). Therefore storing the same tags in single precision would take up.9 T B. 3. If we take the absolute value of RAM available (i.e. 5 MB) and the standard definition of a floating point double precision number (i.e. 64 bits = 8 bytes), then, 5 MB 24 KB MB 24 B kb =, 572, 864, bytes Therefore the number of double precision values that could be stored would be. Number =, 572, 864, bytes value 8 bytes = 96, 68, values If we use the maximum MATLAB array size then., 428 MB 24 KB MB 24 bytes kb =, 497, 366, 528 bytes Therefore the number of double precision values that could be stored in MATLAB would be. Number =, 497, 366, 528 bytes value 8 bytes = 87, 7, 86 values 4

5 Question 4 [] Consider the following system of linear algebraic equations: 2x 2 + 4x 3 = x 3 + 4x 3 = 3x + 5x 3 =. Use Gauss elimination (forward elimination and backward substitution) to solve these equations for (x,, x 3 ). 2. Validate your solution in either Python or MATLAB.. We are asked to solve this system of equations using Gauss Elimination (without partial pivoting). 2x 2 +4x 3 = x 3 +4x 3 = x = 3x +5x 3 = 3 5 x 3 Forward elimination Divide row by element (,) x x 3 = Subtract row from row 2 and 3 times row from row 3 to eliminate all elements in column below the diagonal element 2 x 2 2 = 2 x 3 Divide row 2 by element (2,2) 2 2 x x 3 =.5 Subtract 2 times row 2 from row 3 to eliminate all elements in column 2 below the diagonal element 2 x =.5 x 3 Backwards substitution 5

6 Subtract - times row 3 from row 2 and 2 times row 3 from row to eliminate all elements in column 3 above the diagonal element x = 2.5 x 3 Subtract - times row 2 from row to eliminate all elements in column 2 above the diagonal element x =.5.5 x 3 2. Checking this solution in MATLAB: EDU>> A = [2,-2,4;,-3,4; 3,-,5]; EDU>> b = [;-;]; EDU>> x = A\b x = Checking this solution in Python: In []: A = np.array([[2,-2,4],[,-3,4],[3,-,5]]) In [2]: b = np.array([[],[-],[]]) In [3]: x = np.linalg.solve(a,b) In [4]: print(x) [[.5] [-.5] [-. ]] 6

Roundoff Errors and Computer Arithmetic

Roundoff Errors and Computer Arithmetic Jim Lambers Math 105A Summer Session I 2003-04 Lecture 2 Notes These notes correspond to Section 1.2 in the text. Roundoff Errors and Computer Arithmetic In computing the solution to any mathematical problem,

More information

Math 340 Fall 2014, Victor Matveev. Binary system, round-off errors, loss of significance, and double precision accuracy.

Math 340 Fall 2014, Victor Matveev. Binary system, round-off errors, loss of significance, and double precision accuracy. Math 340 Fall 2014, Victor Matveev Binary system, round-off errors, loss of significance, and double precision accuracy. 1. Bits and the binary number system A bit is one digit in a binary representation

More information

The type of all data used in a C (or C++) program must be specified

The type of all data used in a C (or C++) program must be specified The type of all data used in a C (or C++) program must be specified A data type is a description of the data being represented That is, a set of possible values and a set of operations on those values

More information

COSC 243. Data Representation 3. Lecture 3 - Data Representation 3 1. COSC 243 (Computer Architecture)

COSC 243. Data Representation 3. Lecture 3 - Data Representation 3 1. COSC 243 (Computer Architecture) COSC 243 Data Representation 3 Lecture 3 - Data Representation 3 1 Data Representation Test Material Lectures 1, 2, and 3 Tutorials 1b, 2a, and 2b During Tutorial a Next Week 12 th and 13 th March If you

More information

Module 1: Information Representation I -- Number Systems

Module 1: Information Representation I -- Number Systems Unit 1: Computer Systems, pages 1 of 7 - Department of Computer and Mathematical Sciences CS 1305 Intro to Computer Technology 1 Module 1: Information Representation I -- Number Systems Objectives: Learn

More information

CS321 Introduction To Numerical Methods

CS321 Introduction To Numerical Methods CS3 Introduction To Numerical Methods Fuhua (Frank) Cheng Department of Computer Science University of Kentucky Lexington KY 456-46 - - Table of Contents Errors and Number Representations 3 Error Types

More information

fractional quantities are typically represented in computers using floating point format this approach is very much similar to scientific notation

fractional quantities are typically represented in computers using floating point format this approach is very much similar to scientific notation Floating Point Arithmetic fractional quantities are typically represented in computers using floating point format this approach is very much similar to scientific notation for example, fixed point number

More information

IT 1204 Section 2.0. Data Representation and Arithmetic. 2009, University of Colombo School of Computing 1

IT 1204 Section 2.0. Data Representation and Arithmetic. 2009, University of Colombo School of Computing 1 IT 1204 Section 2.0 Data Representation and Arithmetic 2009, University of Colombo School of Computing 1 What is Analog and Digital The interpretation of an analog signal would correspond to a signal whose

More information

Bindel, Fall 2016 Matrix Computations (CS 6210) Notes for

Bindel, Fall 2016 Matrix Computations (CS 6210) Notes for 1 Logistics Notes for 2016-09-07 1. We are still at 50. If you are still waiting and are not interested in knowing if a slot frees up, let me know. 2. There is a correction to HW 1, problem 4; the condition

More information

Floating Point. The World is Not Just Integers. Programming languages support numbers with fraction

Floating Point. The World is Not Just Integers. Programming languages support numbers with fraction 1 Floating Point The World is Not Just Integers Programming languages support numbers with fraction Called floating-point numbers Examples: 3.14159265 (π) 2.71828 (e) 0.000000001 or 1.0 10 9 (seconds in

More information

EE 109 Unit 19. IEEE 754 Floating Point Representation Floating Point Arithmetic

EE 109 Unit 19. IEEE 754 Floating Point Representation Floating Point Arithmetic 1 EE 109 Unit 19 IEEE 754 Floating Point Representation Floating Point Arithmetic 2 Floating Point Used to represent very small numbers (fractions) and very large numbers Avogadro s Number: +6.0247 * 10

More information

Chapter 3: Arithmetic for Computers

Chapter 3: Arithmetic for Computers Chapter 3: Arithmetic for Computers Objectives Signed and Unsigned Numbers Addition and Subtraction Multiplication and Division Floating Point Computer Architecture CS 35101-002 2 The Binary Numbering

More information

The type of all data used in a C++ program must be specified

The type of all data used in a C++ program must be specified The type of all data used in a C++ program must be specified A data type is a description of the data being represented That is, a set of possible values and a set of operations on those values There are

More information

Review of Calculus, cont d

Review of Calculus, cont d Jim Lambers MAT 460/560 Fall Semester 2009-10 Lecture 4 Notes These notes correspond to Sections 1.1 1.2 in the text. Review of Calculus, cont d Taylor s Theorem, cont d We conclude our discussion of Taylor

More information

Computer (Literacy) Skills. Number representations and memory. Lubomír Bulej KDSS MFF UK

Computer (Literacy) Skills. Number representations and memory. Lubomír Bulej KDSS MFF UK Computer (Literacy Skills Number representations and memory Lubomír Bulej KDSS MFF UK Number representations? What for? Recall: computer works with binary numbers Groups of zeroes and ones 8 bits (byte,

More information

Module 2: Computer Arithmetic

Module 2: Computer Arithmetic Module 2: Computer Arithmetic 1 B O O K : C O M P U T E R O R G A N I Z A T I O N A N D D E S I G N, 3 E D, D A V I D L. P A T T E R S O N A N D J O H N L. H A N N E S S Y, M O R G A N K A U F M A N N

More information

Data Representation in Computer Memory

Data Representation in Computer Memory Data Representation in Computer Memory Data Representation in Computer Memory Digital computer stores the data in the form of binary bit sequences. Binary number system has two symbols: 0 and 1, called

More information

Binary floating point encodings

Binary floating point encodings Week 1: Wednesday, Jan 25 Binary floating point encodings Binary floating point arithmetic is essentially scientific notation. Where in decimal scientific notation we write in floating point, we write

More information

Floating-Point Arithmetic

Floating-Point Arithmetic Floating-Point Arithmetic 1 Numerical Analysis a definition sources of error 2 Floating-Point Numbers floating-point representation of a real number machine precision 3 Floating-Point Arithmetic adding

More information

UNIT 7A Data Representation: Numbers and Text. Digital Data

UNIT 7A Data Representation: Numbers and Text. Digital Data UNIT 7A Data Representation: Numbers and Text 1 Digital Data 10010101011110101010110101001110 What does this binary sequence represent? It could be: an integer a floating point number text encoded with

More information

COMP2611: Computer Organization. Data Representation

COMP2611: Computer Organization. Data Representation COMP2611: Computer Organization Comp2611 Fall 2015 2 1. Binary numbers and 2 s Complement Numbers 3 Bits: are the basis for binary number representation in digital computers What you will learn here: How

More information

What we need to know about error: Class Outline. Computational Methods CMSC/AMSC/MAPL 460. Errors in data and computation

What we need to know about error: Class Outline. Computational Methods CMSC/AMSC/MAPL 460. Errors in data and computation Class Outline Computational Methods CMSC/AMSC/MAPL 460 Errors in data and computation Representing numbers in floating point Ramani Duraiswami, Dept. of Computer Science Computations should be as accurate

More information

Hani Mehrpouyan, California State University, Bakersfield. Signals and Systems

Hani Mehrpouyan, California State University, Bakersfield. Signals and Systems Hani Mehrpouyan, Department of Electrical and Computer Engineering, California State University, Bakersfield Lecture 3 (Error and Computer Arithmetic) April 8 th, 2013 The material in these lectures is

More information

Experimental Methods I

Experimental Methods I Experimental Methods I Computing: Data types and binary representation M.P. Vaughan Learning objectives Understanding data types for digital computers binary representation of different data types: Integers

More information

Real Numbers finite subset real numbers floating point numbers Scientific Notation fixed point numbers

Real Numbers finite subset real numbers floating point numbers Scientific Notation fixed point numbers Real Numbers We have been studying integer arithmetic up to this point. We have discovered that a standard computer can represent a finite subset of the infinite set of integers. The range is determined

More information

Computational Methods CMSC/AMSC/MAPL 460. Representing numbers in floating point and associated issues. Ramani Duraiswami, Dept. of Computer Science

Computational Methods CMSC/AMSC/MAPL 460. Representing numbers in floating point and associated issues. Ramani Duraiswami, Dept. of Computer Science Computational Methods CMSC/AMSC/MAPL 460 Representing numbers in floating point and associated issues Ramani Duraiswami, Dept. of Computer Science Class Outline Computations should be as accurate and as

More information

ME 261: Numerical Analysis. ME 261: Numerical Analysis

ME 261: Numerical Analysis. ME 261: Numerical Analysis ME 261: Numerical Analysis 3. credit hours Prereq.: ME 163/ME 171 Course content Approximations and error types Roots of polynomials and transcendental equations Determinants and matrices Solution of linear

More information

Real Numbers finite subset real numbers floating point numbers Scientific Notation fixed point numbers

Real Numbers finite subset real numbers floating point numbers Scientific Notation fixed point numbers Real Numbers We have been studying integer arithmetic up to this point. We have discovered that a standard computer can represent a finite subset of the infinite set of integers. The range is determined

More information

Lecture 1: What is a computer?

Lecture 1: What is a computer? 02-201, Fall 2015, Carl Kingsford Lecture 1: What is a computer? 0. Today's Topics Basic computer architecture How the computer represents data 1. What is a computer? A modern computer is a collection

More information

ECE232: Hardware Organization and Design

ECE232: Hardware Organization and Design ECE232: Hardware Organization and Design Lecture 11: Floating Point & Floating Point Addition Adapted from Computer Organization and Design, Patterson & Hennessy, UCB Last time: Single Precision Format

More information

Objectives. Connecting with Computer Science 2

Objectives. Connecting with Computer Science 2 Objectives Learn why numbering systems are important to understand Refresh your knowledge of powers of numbers Learn how numbering systems are used to count Understand the significance of positional value

More information

Data Representation 1

Data Representation 1 1 Data Representation Outline Binary Numbers Adding Binary Numbers Negative Integers Other Operations with Binary Numbers Floating Point Numbers Character Representation Image Representation Sound Representation

More information

Introduction to Computers and Programming. Numeric Values

Introduction to Computers and Programming. Numeric Values Introduction to Computers and Programming Prof. I. K. Lundqvist Lecture 5 Reading: B pp. 47-71 Sept 1 003 Numeric Values Storing the value of 5 10 using ASCII: 00110010 00110101 Binary notation: 00000000

More information

Chapter 4 Section 2 Operations on Decimals

Chapter 4 Section 2 Operations on Decimals Chapter 4 Section 2 Operations on Decimals Addition and subtraction of decimals To add decimals, write the numbers so that the decimal points are on a vertical line. Add as you would with whole numbers.

More information

Numeric Encodings Prof. James L. Frankel Harvard University

Numeric Encodings Prof. James L. Frankel Harvard University Numeric Encodings Prof. James L. Frankel Harvard University Version of 10:19 PM 12-Sep-2017 Copyright 2017, 2016 James L. Frankel. All rights reserved. Representation of Positive & Negative Integral and

More information

Electronic Data and Instructions

Electronic Data and Instructions Lecture 2 - The information Layer Binary Values and Number Systems, Data Representation. Know the different types of numbers Describe positional notation Convert numbers in other bases to base 10 Convert

More information

Review Questions 26 CHAPTER 1. SCIENTIFIC COMPUTING

Review Questions 26 CHAPTER 1. SCIENTIFIC COMPUTING 26 CHAPTER 1. SCIENTIFIC COMPUTING amples. The IEEE floating-point standard can be found in [131]. A useful tutorial on floating-point arithmetic and the IEEE standard is [97]. Although it is no substitute

More information

MACHINE LEVEL REPRESENTATION OF DATA

MACHINE LEVEL REPRESENTATION OF DATA MACHINE LEVEL REPRESENTATION OF DATA CHAPTER 2 1 Objectives Understand how integers and fractional numbers are represented in binary Explore the relationship between decimal number system and number systems

More information

Scientific Computing. Error Analysis

Scientific Computing. Error Analysis ECE257 Numerical Methods and Scientific Computing Error Analysis Today s s class: Introduction to error analysis Approximations Round-Off Errors Introduction Error is the difference between the exact solution

More information

3 Data Storage 3.1. Foundations of Computer Science Cengage Learning

3 Data Storage 3.1. Foundations of Computer Science Cengage Learning 3 Data Storage 3.1 Foundations of Computer Science Cengage Learning Objectives After studying this chapter, the student should be able to: List five different data types used in a computer. Describe how

More information

Computer Architecture and Organization. Chapter 2 Data Representation!

Computer Architecture and Organization. Chapter 2 Data Representation! 2-1! Chapter 2 - Data Representation! Computer Architecture and Organization Miles Murdocca and Vincent Heuring! Chapter 2 Data Representation! 2-2! Chapter 2 - Data Representation! Chapter Contents! 2.1

More information

Computational Methods CMSC/AMSC/MAPL 460. Representing numbers in floating point Error Analysis. Ramani Duraiswami, Dept. of Computer Science

Computational Methods CMSC/AMSC/MAPL 460. Representing numbers in floating point Error Analysis. Ramani Duraiswami, Dept. of Computer Science Computational Methods CMSC/AMSC/MAPL 460 Representing numbers in floating point Error Analysis Ramani Duraiswami, Dept. of Computer Science Class Outline Recap of floating point representation Matlab demos

More information

data within a computer system are stored in one of 2 physical states (hence the use of binary digits)

data within a computer system are stored in one of 2 physical states (hence the use of binary digits) Binary Digits (bits) data within a computer system are stored in one of 2 physical states (hence the use of binary digits) 0V and 5V charge / NO charge on a transistor gate ferrite core magnetised clockwise

More information

Chapter 03: Computer Arithmetic. Lesson 09: Arithmetic using floating point numbers

Chapter 03: Computer Arithmetic. Lesson 09: Arithmetic using floating point numbers Chapter 03: Computer Arithmetic Lesson 09: Arithmetic using floating point numbers Objective To understand arithmetic operations in case of floating point numbers 2 Multiplication of Floating Point Numbers

More information

15213 Recitation 2: Floating Point

15213 Recitation 2: Floating Point 15213 Recitation 2: Floating Point 1 Introduction This handout will introduce and test your knowledge of the floating point representation of real numbers, as defined by the IEEE standard. This information

More information

Accuracy versus precision

Accuracy versus precision Accuracy versus precision Accuracy is a consistent error from the true value, but not necessarily a good or precise error Precision is a consistent result within a small error, but not necessarily anywhere

More information

Computational Methods CMSC/AMSC/MAPL 460. Representing numbers in floating point and associated issues. Ramani Duraiswami, Dept. of Computer Science

Computational Methods CMSC/AMSC/MAPL 460. Representing numbers in floating point and associated issues. Ramani Duraiswami, Dept. of Computer Science Computational Methods CMSC/AMSC/MAPL 460 Representing numbers in floating point and associated issues Ramani Duraiswami, Dept. of Computer Science Class Outline Computations should be as accurate and as

More information

Mathematical preliminaries and error analysis

Mathematical preliminaries and error analysis Mathematical preliminaries and error analysis Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan August 28, 2011 Outline 1 Round-off errors and computer arithmetic IEEE

More information

Lecture Objectives. Structured Programming & an Introduction to Error. Review the basic good habits of programming

Lecture Objectives. Structured Programming & an Introduction to Error. Review the basic good habits of programming Structured Programming & an Introduction to Error Lecture Objectives Review the basic good habits of programming To understand basic concepts of error and error estimation as it applies to Numerical Methods

More information

Chapter Three. Arithmetic

Chapter Three. Arithmetic Chapter Three 1 Arithmetic Where we've been: Performance (seconds, cycles, instructions) Abstractions: Instruction Set Architecture Assembly Language and Machine Language What's up ahead: Implementing

More information

MAT128A: Numerical Analysis Lecture Two: Finite Precision Arithmetic

MAT128A: Numerical Analysis Lecture Two: Finite Precision Arithmetic MAT128A: Numerical Analysis Lecture Two: Finite Precision Arithmetic September 28, 2018 Lecture 1 September 28, 2018 1 / 25 Floating point arithmetic Computers use finite strings of binary digits to represent

More information

Divide: Paper & Pencil

Divide: Paper & Pencil Divide: Paper & Pencil 1001 Quotient Divisor 1000 1001010 Dividend -1000 10 101 1010 1000 10 Remainder See how big a number can be subtracted, creating quotient bit on each step Binary => 1 * divisor or

More information

Positional notation Ch Conversions between Decimal and Binary. /continued. Binary to Decimal

Positional notation Ch Conversions between Decimal and Binary. /continued. Binary to Decimal Positional notation Ch.. /continued Conversions between Decimal and Binary Binary to Decimal - use the definition of a number in a positional number system with base - evaluate the definition formula using

More information

2 Computation with Floating-Point Numbers

2 Computation with Floating-Point Numbers 2 Computation with Floating-Point Numbers 2.1 Floating-Point Representation The notion of real numbers in mathematics is convenient for hand computations and formula manipulations. However, real numbers

More information

Computational Economics and Finance

Computational Economics and Finance Computational Economics and Finance Part I: Elementary Concepts of Numerical Analysis Spring 2015 Outline Computer arithmetic Error analysis: Sources of error Error propagation Controlling the error Rates

More information

Algebraically Speaking Chalkdust Algebra 1 Fall Semester

Algebraically Speaking Chalkdust Algebra 1 Fall Semester Algebraically Speaking Chalkdust Algebra 1 Fall Semester Homework Assignments: Chapter 1 The Real Number System: Lesson 1.1 - Real Numbers: Order and Absolute Value Do the following problems: # 1 9 Odd,

More information

Floating Point Arithmetic

Floating Point Arithmetic Floating Point Arithmetic CS 365 Floating-Point What can be represented in N bits? Unsigned 0 to 2 N 2s Complement -2 N-1 to 2 N-1-1 But, what about? very large numbers? 9,349,398,989,787,762,244,859,087,678

More information

Adding Integers pp

Adding Integers pp LESSON 2-1 Adding Integers pp. 60 61 Vocabulary integers (p. 60) opposites (p. 60) absolute value (p. 60) Additional Examples Example 1 Use a number line to find the sum. (6) 2 6 5 4 3 2 1 0 1 2 3 4 5

More information

Numerical computing. How computers store real numbers and the problems that result

Numerical computing. How computers store real numbers and the problems that result Numerical computing How computers store real numbers and the problems that result The scientific method Theory: Mathematical equations provide a description or model Experiment Inference from data Test

More information

1. NUMBER SYSTEMS USED IN COMPUTING: THE BINARY NUMBER SYSTEM

1. NUMBER SYSTEMS USED IN COMPUTING: THE BINARY NUMBER SYSTEM 1. NUMBER SYSTEMS USED IN COMPUTING: THE BINARY NUMBER SYSTEM 1.1 Introduction Given that digital logic and memory devices are based on two electrical states (on and off), it is natural to use a number

More information

Floating-Point Numbers in Digital Computers

Floating-Point Numbers in Digital Computers POLYTECHNIC UNIVERSITY Department of Computer and Information Science Floating-Point Numbers in Digital Computers K. Ming Leung Abstract: We explain how floating-point numbers are represented and stored

More information

Floating-Point Numbers in Digital Computers

Floating-Point Numbers in Digital Computers POLYTECHNIC UNIVERSITY Department of Computer and Information Science Floating-Point Numbers in Digital Computers K. Ming Leung Abstract: We explain how floating-point numbers are represented and stored

More information

AM205: lecture 2. 1 These have been shifted to MD 323 for the rest of the semester.

AM205: lecture 2. 1 These have been shifted to MD 323 for the rest of the semester. AM205: lecture 2 Luna and Gary will hold a Python tutorial on Wednesday in 60 Oxford Street, Room 330 Assignment 1 will be posted this week Chris will hold office hours on Thursday (1:30pm 3:30pm, Pierce

More information

Floating-Point Arithmetic

Floating-Point Arithmetic Floating-Point Arithmetic Raymond J. Spiteri Lecture Notes for CMPT 898: Numerical Software University of Saskatchewan January 9, 2013 Objectives Floating-point numbers Floating-point arithmetic Analysis

More information

9/3/2015. Data Representation II. 2.4 Signed Integer Representation. 2.4 Signed Integer Representation

9/3/2015. Data Representation II. 2.4 Signed Integer Representation. 2.4 Signed Integer Representation Data Representation II CMSC 313 Sections 01, 02 The conversions we have so far presented have involved only unsigned numbers. To represent signed integers, computer systems allocate the high-order bit

More information

ROUNDING ERRORS LAB 1. OBJECTIVE 2. INTRODUCTION

ROUNDING ERRORS LAB 1. OBJECTIVE 2. INTRODUCTION ROUNDING ERRORS LAB Imagine you are traveling in Italy, and you are trying to convert $27.00 into Euros. You go to the bank teller, who gives you 20.19. Your friend is with you, and she is converting $2,700.00.

More information

Benchmarks Addressed. Quizzes. Strand IV, S 1-2, 1-4, 2-1, 2-2, 2-3, 3-1. Journal writing. Projects. Worksheets

Benchmarks Addressed. Quizzes. Strand IV, S 1-2, 1-4, 2-1, 2-2, 2-3, 3-1. Journal writing. Projects. Worksheets August/September The Decimal System Translating English words into Decimal Notation What is the decimal system? Str IV, S 1-2, 1-4, 2-1, 2-2, 2-3, 3-1 Estimating Decimals What is a number line? Changing

More information

Floating-Point Arithmetic

Floating-Point Arithmetic Floating-Point Arithmetic ECS30 Winter 207 January 27, 207 Floating point numbers Floating-point representation of numbers (scientific notation) has four components, for example, 3.46 0 sign significand

More information

b A bit is the basic unit for storing electronic data, for example an MP3 file. The term bit is a

b A bit is the basic unit for storing electronic data, for example an MP3 file. The term bit is a Digital download and file storage Syllabus: FSCo2 Focus Study: Mathematics and Communication Digital Storage b A bit is the basic unit for storing electronic data, for example an MP3 file. The term bit

More information

Floating Point. EE 109 Unit 20. Floating Point Representation. Fixed Point

Floating Point. EE 109 Unit 20. Floating Point Representation. Fixed Point 2.1 Floating Point 2.2 EE 19 Unit 2 IEEE 754 Floating Point Representation Floating Point Arithmetic Used to represent very numbers (fractions) and very numbers Avogadro s Number: +6.247 * 1 23 Planck

More information

CS 265. Computer Architecture. Wei Lu, Ph.D., P.Eng.

CS 265. Computer Architecture. Wei Lu, Ph.D., P.Eng. CS 265 Computer Architecture Wei Lu, Ph.D., P.Eng. 1 Part 1: Data Representation Our goal: revisit and re-establish fundamental of mathematics for the computer architecture course Overview: what are bits

More information

Number Systems and Binary Arithmetic. Quantitative Analysis II Professor Bob Orr

Number Systems and Binary Arithmetic. Quantitative Analysis II Professor Bob Orr Number Systems and Binary Arithmetic Quantitative Analysis II Professor Bob Orr Introduction to Numbering Systems We are all familiar with the decimal number system (Base 10). Some other number systems

More information

Chapter 2 Bits, Data Types, and Operations

Chapter 2 Bits, Data Types, and Operations Chapter Bits, Data Types, and Operations How do we represent data in a computer? At the lowest level, a computer is an electronic machine. works by controlling the flow of electrons Easy to recognize two

More information

MATH 353 Engineering mathematics III

MATH 353 Engineering mathematics III MATH 353 Engineering mathematics III Instructor: Francisco-Javier Pancho Sayas Spring 2014 University of Delaware Instructor: Francisco-Javier Pancho Sayas MATH 353 1 / 20 MEET YOUR COMPUTER Instructor:

More information

Intermediate Mathematics League of Eastern Massachusetts

Intermediate Mathematics League of Eastern Massachusetts Meet # January 010 Intermediate Mathematics League of Eastern Massachusetts Meet # January 010 Category 1 - Mystery Meet #, January 010 1. Of all the number pairs whose sum equals their product, what is

More information

Finite arithmetic and error analysis

Finite arithmetic and error analysis Finite arithmetic and error analysis Escuela de Ingeniería Informática de Oviedo (Dpto de Matemáticas-UniOvi) Numerical Computation Finite arithmetic and error analysis 1 / 45 Outline 1 Number representation:

More information

Error in Numerical Methods

Error in Numerical Methods Error in Numerical Methods By Brian D. Storey This section will describe two types of error that are common in numerical calculations: roundoff and truncation error. Roundoff error is due to the fact that

More information

1.3 Floating Point Form

1.3 Floating Point Form Section 1.3 Floating Point Form 29 1.3 Floating Point Form Floating point numbers are used by computers to approximate real numbers. On the surface, the question is a simple one. There are an infinite

More information

EE 109 Unit 20. IEEE 754 Floating Point Representation Floating Point Arithmetic

EE 109 Unit 20. IEEE 754 Floating Point Representation Floating Point Arithmetic 1 EE 109 Unit 20 IEEE 754 Floating Point Representation Floating Point Arithmetic 2 Floating Point Used to represent very small numbers (fractions) and very large numbers Avogadro s Number: +6.0247 * 10

More information

3.1 DATA REPRESENTATION (PART C)

3.1 DATA REPRESENTATION (PART C) 3.1 DATA REPRESENTATION (PART C) 3.1.3 REAL NUMBERS AND NORMALISED FLOATING-POINT REPRESENTATION In decimal notation, the number 23.456 can be written as 0.23456 x 10 2. This means that in decimal notation,

More information

Floating point numbers in Scilab

Floating point numbers in Scilab Floating point numbers in Scilab Michaël Baudin May 2011 Abstract This document is a small introduction to floating point numbers in Scilab. In the first part, we describe the theory of floating point

More information

Homework 1 graded and returned in class today. Solutions posted online. Request regrades by next class period. Question 10 treated as extra credit

Homework 1 graded and returned in class today. Solutions posted online. Request regrades by next class period. Question 10 treated as extra credit Announcements Homework 1 graded and returned in class today. Solutions posted online. Request regrades by next class period. Question 10 treated as extra credit Quiz 2 Monday on Number System Conversions

More information

Approximations and Errors

Approximations and Errors The Islamic University of Gaza Faculty of Engineering Civil Engineering Department Numerical Analysis ECIV 3306 Chapter 3 Approximations and Errors Associate Prof. Mazen Abualtayef Civil Engineering Department,

More information

Number Systems Standard positional representation of numbers: An unsigned number with whole and fraction portions is represented as:

Number Systems Standard positional representation of numbers: An unsigned number with whole and fraction portions is represented as: N Number Systems Standard positional representation of numbers: An unsigned number with whole and fraction portions is represented as: a n a a a The value of this number is given by: = a n Ka a a a a a

More information

1.2 Round-off Errors and Computer Arithmetic

1.2 Round-off Errors and Computer Arithmetic 1.2 Round-off Errors and Computer Arithmetic 1 In a computer model, a memory storage unit word is used to store a number. A word has only a finite number of bits. These facts imply: 1. Only a small set

More information

Chapter 2 Bits, Data Types, and Operations

Chapter 2 Bits, Data Types, and Operations Chapter 2 Bits, Data Types, and Operations How do we represent data in a computer? At the lowest level, a computer is an electronic machine. works by controlling the flow of electrons Easy to recognize

More information

Floating-point representation

Floating-point representation Lecture 3-4: Floating-point representation and arithmetic Floating-point representation The notion of real numbers in mathematics is convenient for hand computations and formula manipulations. However,

More information

Computational Methods. Sources of Errors

Computational Methods. Sources of Errors Computational Methods Sources of Errors Manfred Huber 2011 1 Numerical Analysis / Scientific Computing Many problems in Science and Engineering can not be solved analytically on a computer Numeric solutions

More information

Computational Economics and Finance

Computational Economics and Finance Computational Economics and Finance Part I: Elementary Concepts of Numerical Analysis Spring 2016 Outline Computer arithmetic Error analysis: Sources of error Error propagation Controlling the error Rates

More information

Signed Binary Numbers

Signed Binary Numbers Signed Binary Numbers Unsigned Binary Numbers We write numbers with as many digits as we need: 0, 99, 65536, 15000, 1979, However, memory locations and CPU registers always hold a constant, fixed number

More information

2 Computation with Floating-Point Numbers

2 Computation with Floating-Point Numbers 2 Computation with Floating-Point Numbers 2.1 Floating-Point Representation The notion of real numbers in mathematics is convenient for hand computations and formula manipulations. However, real numbers

More information

Section 1.4 Mathematics on the Computer: Floating Point Arithmetic

Section 1.4 Mathematics on the Computer: Floating Point Arithmetic Section 1.4 Mathematics on the Computer: Floating Point Arithmetic Key terms Floating point arithmetic IEE Standard Mantissa Exponent Roundoff error Pitfalls of floating point arithmetic Structuring computations

More information

Course Number 432/433 Title Algebra II (A & B) H Grade # of Days 120

Course Number 432/433 Title Algebra II (A & B) H Grade # of Days 120 Whitman-Hanson Regional High School provides all students with a high- quality education in order to develop reflective, concerned citizens and contributing members of the global community. Course Number

More information

Final Labs and Tutors

Final Labs and Tutors ICT106 Fundamentals of Computer Systems - Topic 2 REPRESENTATION AND STORAGE OF INFORMATION Reading: Linux Assembly Programming Language, Ch 2.4-2.9 and 3.6-3.8 Final Labs and Tutors Venue and time South

More information

KNOWLEDGE OF NUMBER SENSE, CONCEPTS, AND OPERATIONS

KNOWLEDGE OF NUMBER SENSE, CONCEPTS, AND OPERATIONS DOMAIN I. COMPETENCY 1.0 MATHEMATICS KNOWLEDGE OF NUMBER SENSE, CONCEPTS, AND OPERATIONS Skill 1.1 Compare the relative value of real numbers (e.g., integers, fractions, decimals, percents, irrational

More information

IEEE Standard for Floating-Point Arithmetic: 754

IEEE Standard for Floating-Point Arithmetic: 754 IEEE Standard for Floating-Point Arithmetic: 754 G.E. Antoniou G.E. Antoniou () IEEE Standard for Floating-Point Arithmetic: 754 1 / 34 Floating Point Standard: IEEE 754 1985/2008 Established in 1985 (2008)

More information

2.1.1 Fixed-Point (or Integer) Arithmetic

2.1.1 Fixed-Point (or Integer) Arithmetic x = approximation to true value x error = x x, relative error = x x. x 2.1.1 Fixed-Point (or Integer) Arithmetic A base 2 (base 10) fixed-point number has a fixed number of binary (decimal) places. 1.

More information

Exponential Numbers ID1050 Quantitative & Qualitative Reasoning

Exponential Numbers ID1050 Quantitative & Qualitative Reasoning Exponential Numbers ID1050 Quantitative & Qualitative Reasoning In what ways can you have $2000? Just like fractions, you can have a number in some denomination Number Denomination Mantissa Power of 10

More information

Binary Codes. Dr. Mudathir A. Fagiri

Binary Codes. Dr. Mudathir A. Fagiri Binary Codes Dr. Mudathir A. Fagiri Binary System The following are some of the technical terms used in binary system: Bit: It is the smallest unit of information used in a computer system. It can either

More information

CSCI 402: Computer Architectures. Arithmetic for Computers (3) Fengguang Song Department of Computer & Information Science IUPUI.

CSCI 402: Computer Architectures. Arithmetic for Computers (3) Fengguang Song Department of Computer & Information Science IUPUI. CSCI 402: Computer Architectures Arithmetic for Computers (3) Fengguang Song Department of Computer & Information Science IUPUI 3.5 Today s Contents Floating point numbers: 2.5, 10.1, 100.2, etc.. How

More information