Numerical Precision. Or, why my numbers aren t numbering right. 1 of 15
|
|
- Gilbert Long
- 6 years ago
- Views:
Transcription
1 Numerical Precision Or, why my numbers aren t numbering right 1 of 15
2 What s the deal? Maybe you ve seen this #include <stdio.h> int main() { float val = 3.6f; printf( %.20f \n, val); Print a float with value 3.6 to 20 decimal places } cslab6c 123sh>./myAmazingProgram cslab6c 123sh> of 15
3 What s the deal? We trust computers to do complicated arithmetic, using both integers and floating point numbers Like 6 41 or Sometimes, computers let us down :( Numerical overflow (integers above max wrap around to min and vice versa) Numerical underflow (very small floats become 0) Just plain incorrect results (like the example of 3.6 we just saw) Why do errors happen? 3 of 15
4 Some intuition Conceptually, in mathematics / the universe / life, there are Infinitely many whole numbers Infinitely many fractional numbers What if we constrain the range? For example, what happens when we only count values from 1-5? Finite amount of whole numbers: 1, 2, 3, 4, and 5 Still infinitely many fractional numbers 4 of 15
5 Some background All numbers are stored on the computer in binary Binary is a base 2 number system: each digit can be one of two values, 0 or 1 You probably write numbers in base 10, where digits go from 0-9 Ex. you type 552 computer stores Both binary and decimal can represent any number, given enough digits If theoretically this should work, then the problem must lie in implementation! Take CS33 or CS22 to learn more about number representations and binary! On CS department machines, data types have these specific, finite sizes* int 32 bits float 32 bits double 64 bits * Not guaranteed on all platforms! You can check a type s size in bytes using sizeof(float), etc. 5 of 15
6 Computers have finite resources Computers have limited hard drive space, and data points stored on a computer are size-limited by their type (as seen on previous slide) We simply cannot store infinite data Thankfully, within these limits integer representations (and arithmetic) work perfectly But floating point numbers run into issues... Big problem: If we want to compute precisely with floats, how can we represent infinitely many floating point numbers in limited space? We can t :( We have to cheat 6 of 15
7 Computers have finite resources IEEE standard for floating point numbers you do not need to memorize this 32 bits: (1) (8) (23) Sign bit: is it positive or negative? Fraction bits: what is the actual value? Exponent bits: what order of magnitude is it? What does this mean? A float stores numbers a lot like scientific notation: Fraction (base num) exponent Ex. decimal: Ex. floating point: Note about fraction bits in binary: since we omit leading zeros in the fraction (part of conversion to normalized form), every fraction will start with a 1, which means we can omit this bit and gain an extra bit of precision for free! 7 of 15
8 Computers have finite resources IEEE floats are actually pretty great we can represent a lot of numbers with just 32 bits! When we can t represent a number, we take the closest number that we can represent This leads to unexpected results sometimes Let s look at a (simplified) example of this format to see what s going on We ll use base 10 (because only Cylons think in binary) We ll constrain the size of our data 3 possible values for the exponent: {-1, 0, 1} 1 decimal digit for the fraction 8 of 15
9 Example: floating point imprecision Fraction Exponent = = = = = = = = = = = = = = = 40 What s this? In scientific / floating point notation, we re trying every fraction and every exponent = = = = = = = = = = = = 80 This will compute every value we can represent in our example scheme. Next, let s plot a picture! = = = 90 9 of 15
10 Example: floating point imprecision (A) Exponents = {-1} (B) Exponents = {-1, 0} (C) Exponents = {-1, 0, 1} In this example, we can already see 2 big issues 1) We can only represent samples of the entire continuum 2) The samples are not distributed uniformly! The closer we are to zero, the more numbers we can represent With an adequate amount of fraction and exponent bits, the distribution is less extreme, but still not uniform 10 of 15
11 Example: floating point imprecision Yikes. I don t want to compute with that system. Can we make our numbers more precise? Yes. We can get higher precision by adding more digits to the fraction We can get a wider range by adding more digits to the exponent In fact, plain floats are often big enough to store anything you d need in CS123 Our output is graphical and will be viewed by the human eye, which cannot detect these small errors Little value in using double-precision data: more work, small ROI Precision will always be an issue to some degree: with any kind of similar representation, you will always have some margin of error 11 of 15
12 How do we counter imprecision? In cases of comparison, allow some margin of acceptable error Never ever do something like this: if (mybigcomputation == 0.123)... BAD! May fail unexpectedly. Instead, do something like this: if (abs(mybigcomputation - targetvalue) < epsilon)... GOOD! Will catch computer error. abs() is absolute value; try picking epsilon to be near 1e-4 (0.0001) *but your mileage may vary!* 12 of 15
13 Imprecision in the wild Example from your Ray project: floating point imprecision can lead to speckling on objects. 13 of 15
14 Warning: error propagation Be careful of accumulators and repeated error! Small errors can build up if used over and over Ex. plotting points around a circle using a for loop float angle = 0; float inc = 1.0/20 * 360; for (int i=0; i<20; i++) { plotpointat(angle); angle += inc; } Will accumulate error over time. for (int i=0; i<20; i++) { float angle = i/20.0 * 360; plotpointat(angle); } Will not drift from true values. Yikes! On the left, an error of could become 0.02, 2, 200, or worse It can be difficult to debug where large errors come from, as they may seem unrelated to the minor errors ignored earlier! Try to avoid accumulating error by computing absolute position each time OR testing values for error before accumulating them 14 of 15
15 Takeaway Computers are less precise than true mathematics We encourage you to use floats in your code But be cautious! Testing for equality between floats is dangerous, because there is no guarantee that either value can be represented by the computer. Be careful of propagating errors in your code! Don t let floats accumulate too much error! 15 of 15
Introduction to Scientific Computing Lecture 1
Introduction to Scientific Computing Lecture 1 Professor Hanno Rein Last updated: September 10, 2017 1 Number Representations In this lecture, we will cover two concept that are important to understand
More informationWhat Every Programmer Should Know About Floating-Point Arithmetic
What Every Programmer Should Know About Floating-Point Arithmetic Last updated: October 15, 2015 Contents 1 Why don t my numbers add up? 3 2 Basic Answers 3 2.1 Why don t my numbers, like 0.1 + 0.2 add
More informationFloating Point Arithmetic
Floating Point Arithmetic CS 365 Floating-Point What can be represented in N bits? Unsigned 0 to 2 N 2s Complement -2 N-1 to 2 N-1-1 But, what about? very large numbers? 9,349,398,989,787,762,244,859,087,678
More informationMore about Binary 9/6/2016
More about Binary 9/6/2016 Unsigned vs. Two s Complement 8-bit example: 1 1 0 0 0 0 1 1 2 7 +2 6 + 2 1 +2 0 = 128+64+2+1 = 195-2 7 +2 6 + 2 1 +2 0 = -128+64+2+1 = -61 Why does two s complement work this
More informationC++ Data Types. 1 Simple C++ Data Types 2. 3 Numeric Types Integers (whole numbers) Decimal Numbers... 5
C++ Data Types Contents 1 Simple C++ Data Types 2 2 Quick Note About Representations 3 3 Numeric Types 4 3.1 Integers (whole numbers)............................................ 4 3.2 Decimal Numbers.................................................
More informationChapter 3: Arithmetic for Computers
Chapter 3: Arithmetic for Computers Objectives Signed and Unsigned Numbers Addition and Subtraction Multiplication and Division Floating Point Computer Architecture CS 35101-002 2 The Binary Numbering
More informationCS 265. Computer Architecture. Wei Lu, Ph.D., P.Eng.
CS 265 Computer Architecture Wei Lu, Ph.D., P.Eng. 1 Part 1: Data Representation Our goal: revisit and re-establish fundamental of mathematics for the computer architecture course Overview: what are bits
More information3.1 DATA REPRESENTATION (PART C)
3.1 DATA REPRESENTATION (PART C) 3.1.3 REAL NUMBERS AND NORMALISED FLOATING-POINT REPRESENTATION In decimal notation, the number 23.456 can be written as 0.23456 x 10 2. This means that in decimal notation,
More informationBasic Definition INTEGER DATA. Unsigned Binary and Binary-Coded Decimal. BCD: Binary-Coded Decimal
Basic Definition REPRESENTING INTEGER DATA Englander Ch. 4 An integer is a number which has no fractional part. Examples: -2022-213 0 1 514 323434565232 Unsigned and -Coded Decimal BCD: -Coded Decimal
More informationFloating Point Numbers
Floating Point Numbers Summer 8 Fractional numbers Fractional numbers fixed point Floating point numbers the IEEE 7 floating point standard Floating point operations Rounding modes CMPE Summer 8 Slides
More informationSigned umbers. Sign/Magnitude otation
Signed umbers So far we have discussed unsigned number representations. In particular, we have looked at the binary number system and shorthand methods in representing binary codes. With m binary digits,
More informationCS 261 Fall Floating-Point Numbers. Mike Lam, Professor. https://xkcd.com/217/
CS 261 Fall 2017 Mike Lam, Professor https://xkcd.com/217/ Floating-Point Numbers Floating-point Topics Binary fractions Floating-point representation Conversions and rounding error Binary fractions Now
More informationCOMP2611: Computer Organization. Data Representation
COMP2611: Computer Organization Comp2611 Fall 2015 2 1. Binary numbers and 2 s Complement Numbers 3 Bits: are the basis for binary number representation in digital computers What you will learn here: How
More informationMAT128A: Numerical Analysis Lecture Two: Finite Precision Arithmetic
MAT128A: Numerical Analysis Lecture Two: Finite Precision Arithmetic September 28, 2018 Lecture 1 September 28, 2018 1 / 25 Floating point arithmetic Computers use finite strings of binary digits to represent
More informationCS 261 Fall Floating-Point Numbers. Mike Lam, Professor.
CS 261 Fall 2018 Mike Lam, Professor https://xkcd.com/217/ Floating-Point Numbers Floating-point Topics Binary fractions Floating-point representation Conversions and rounding error Binary fractions Now
More informationModule 2: Computer Arithmetic
Module 2: Computer Arithmetic 1 B O O K : C O M P U T E R O R G A N I Z A T I O N A N D D E S I G N, 3 E D, D A V I D L. P A T T E R S O N A N D J O H N L. H A N N E S S Y, M O R G A N K A U F M A N N
More informationChapter 2. Data Representation in Computer Systems
Chapter 2 Data Representation in Computer Systems Chapter 2 Objectives Understand the fundamentals of numerical data representation and manipulation in digital computers. Master the skill of converting
More informationNumber Systems. Decimal numbers. Binary numbers. Chapter 1 <1> 8's column. 1000's column. 2's column. 4's column
1's column 10's column 100's column 1000's column 1's column 2's column 4's column 8's column Number Systems Decimal numbers 5374 10 = Binary numbers 1101 2 = Chapter 1 1's column 10's column 100's
More informationThe type of all data used in a C++ program must be specified
The type of all data used in a C++ program must be specified A data type is a description of the data being represented That is, a set of possible values and a set of operations on those values There are
More informationCSCI 402: Computer Architectures. Arithmetic for Computers (3) Fengguang Song Department of Computer & Information Science IUPUI.
CSCI 402: Computer Architectures Arithmetic for Computers (3) Fengguang Song Department of Computer & Information Science IUPUI 3.5 Today s Contents Floating point numbers: 2.5, 10.1, 100.2, etc.. How
More informationRepresentation of Non Negative Integers
Representation of Non Negative Integers In each of one s complement and two s complement arithmetic, no special steps are required to represent a non negative integer. All conversions to the complement
More informationNumbers and Computers. Debdeep Mukhopadhyay Assistant Professor Dept of Computer Sc and Engg IIT Madras
Numbers and Computers Debdeep Mukhopadhyay Assistant Professor Dept of Computer Sc and Engg IIT Madras 1 Think of a number between 1 and 15 8 9 10 11 12 13 14 15 4 5 6 7 12 13 14 15 2 3 6 7 10 11 14 15
More informationIEEE Floating Point Numbers Overview
COMP 40: Machine Structure and Assembly Language Programming (Fall 2015) IEEE Floating Point Numbers Overview Noah Mendelsohn Tufts University Email: noah@cs.tufts.edu Web: http://www.cs.tufts.edu/~noah
More informationUNIVERSITY OF CALIFORNIA, SANTA CRUZ BOARD OF STUDIES IN COMPUTER ENGINEERING
UNIVERSITY OF CALIFORNIA, SANTA CRUZ BOARD OF STUDIES IN COMPUTER ENGINEERING CMPE13/L: INTRODUCTION TO PROGRAMMING IN C SPRING 2012 Lab 3 Matrix Math Introduction Reading In this lab you will write a
More informationThe type of all data used in a C (or C++) program must be specified
The type of all data used in a C (or C++) program must be specified A data type is a description of the data being represented That is, a set of possible values and a set of operations on those values
More informationA Level Computing. Contents. For the Exam:
A Level Computing Contents For the Exam:... 1 Revision of Binary... 2 Computing Mathematics Revision... 2 Binary Addition/Subtraction revision... 3 BCD... 3 Sign and Magnitude... 4 2 s Compliment... 4
More informationCS 31: Introduction to Computer Systems. 03: Binary Arithmetic January 29
CS 31: Introduction to Computer Systems 03: Binary Arithmetic January 29 WiCS! Swarthmore Women in Computer Science Slide 2 Today Binary Arithmetic Unsigned addition Subtraction Representation Signed magnitude
More informationNumerical computing. How computers store real numbers and the problems that result
Numerical computing How computers store real numbers and the problems that result The scientific method Theory: Mathematical equations provide a description or model Experiment Inference from data Test
More informationunused unused unused unused unused unused
BCD numbers. In some applications, such as in the financial industry, the errors that can creep in due to converting numbers back and forth between decimal and binary is unacceptable. For these applications
More informationVariables and Data Representation
You will recall that a computer program is a set of instructions that tell a computer how to transform a given set of input into a specific output. Any program, procedural, event driven or object oriented
More informationTable : IEEE Single Format ± a a 2 a 3 :::a 8 b b 2 b 3 :::b 23 If exponent bitstring a :::a 8 is Then numerical value represented is ( ) 2 = (
Floating Point Numbers in Java by Michael L. Overton Virtually all modern computers follow the IEEE 2 floating point standard in their representation of floating point numbers. The Java programming language
More informationComputer System and programming in C
1 Basic Data Types Integral Types Integers are stored in various sizes. They can be signed or unsigned. Example Suppose an integer is represented by a byte (8 bits). Leftmost bit is sign bit. If the sign
More informationFloating Point. CSE 351 Autumn Instructor: Justin Hsia
Floating Point CSE 351 Autumn 2017 Instructor: Justin Hsia Teaching Assistants: Lucas Wotton Michael Zhang Parker DeWilde Ryan Wong Sam Gehman Sam Wolfson Savanna Yee Vinny Palaniappan Administrivia Lab
More informationUNIT 7A Data Representation: Numbers and Text. Digital Data
UNIT 7A Data Representation: Numbers and Text 1 Digital Data 10010101011110101010110101001110 What does this binary sequence represent? It could be: an integer a floating point number text encoded with
More informationMATH 353 Engineering mathematics III
MATH 353 Engineering mathematics III Instructor: Francisco-Javier Pancho Sayas Spring 2014 University of Delaware Instructor: Francisco-Javier Pancho Sayas MATH 353 1 / 20 MEET YOUR COMPUTER Instructor:
More informationRepresenting Information. Bit Juggling. - Representing information using bits - Number representations. - Some other bits - Chapters 1 and 2.3,2.
Representing Information 0 1 0 Bit Juggling 1 1 - Representing information using bits - Number representations 1 - Some other bits 0 0 - Chapters 1 and 2.3,2.4 Motivations Computers Process Information
More informationCS 101: Computer Programming and Utilization
CS 101: Computer Programming and Utilization Jul-Nov 2017 Umesh Bellur (cs101@cse.iitb.ac.in) Lecture 3: Number Representa.ons Representing Numbers Digital Circuits can store and manipulate 0 s and 1 s.
More informationObjects and Types. COMS W1007 Introduction to Computer Science. Christopher Conway 29 May 2003
Objects and Types COMS W1007 Introduction to Computer Science Christopher Conway 29 May 2003 Java Programs A Java program contains at least one class definition. public class Hello { public static void
More informationA Java program contains at least one class definition.
Java Programs Identifiers Objects and Types COMS W1007 Introduction to Computer Science Christopher Conway 29 May 2003 A Java program contains at least one class definition. public class Hello { public
More informationprintf( Please enter another number: ); scanf( %d, &num2);
CIT 593 Intro to Computer Systems Lecture #13 (11/1/12) Now that we've looked at how an assembly language program runs on a computer, we're ready to move up a level and start working with more powerful
More informationMA 1128: Lecture 02 1/22/2018
MA 1128: Lecture 02 1/22/2018 Exponents Scientific Notation 1 Exponents Exponents are used to indicate how many copies of a number are to be multiplied together. For example, I like to deal with the signs
More informationData Representation 1
1 Data Representation Outline Binary Numbers Adding Binary Numbers Negative Integers Other Operations with Binary Numbers Floating Point Numbers Character Representation Image Representation Sound Representation
More informationROUNDING ERRORS LAB 1. OBJECTIVE 2. INTRODUCTION
ROUNDING ERRORS LAB Imagine you are traveling in Italy, and you are trying to convert $27.00 into Euros. You go to the bank teller, who gives you 20.19. Your friend is with you, and she is converting $2,700.00.
More information3.5 Floating Point: Overview
3.5 Floating Point: Overview Floating point (FP) numbers Scientific notation Decimal scientific notation Binary scientific notation IEEE 754 FP Standard Floating point representation inside a computer
More informationDr. Chuck Cartledge. 3 June 2015
Miscellanea 8224 Revisited Break 1.5 1.6 Conclusion References Backup slides CSC-205 Computer Organization Lecture #002 Section 1.5 Dr. Chuck Cartledge 3 June 2015 1/30 Table of contents I 5 1.6 1 Miscellanea
More informationFinite arithmetic and error analysis
Finite arithmetic and error analysis Escuela de Ingeniería Informática de Oviedo (Dpto de Matemáticas-UniOvi) Numerical Computation Finite arithmetic and error analysis 1 / 45 Outline 1 Number representation:
More informationFloating Point. The World is Not Just Integers. Programming languages support numbers with fraction
1 Floating Point The World is Not Just Integers Programming languages support numbers with fraction Called floating-point numbers Examples: 3.14159265 (π) 2.71828 (e) 0.000000001 or 1.0 10 9 (seconds in
More informationRoundoff Errors and Computer Arithmetic
Jim Lambers Math 105A Summer Session I 2003-04 Lecture 2 Notes These notes correspond to Section 1.2 in the text. Roundoff Errors and Computer Arithmetic In computing the solution to any mathematical problem,
More informationCSCI 1100L: Topics in Computing Lab Lab 11: Programming with Scratch
CSCI 1100L: Topics in Computing Lab Lab 11: Programming with Scratch Purpose: We will take a look at programming this week using a language called Scratch. Scratch is a programming language that was developed
More informationFloating Point. CSE 351 Autumn Instructor: Justin Hsia
Floating Point CSE 351 Autumn 2017 Instructor: Justin Hsia Teaching Assistants: Lucas Wotton Michael Zhang Parker DeWilde Ryan Wong Sam Gehman Sam Wolfson Savanna Yee Vinny Palaniappan http://xkcd.com/571/
More information9/3/2015. Data Representation II. 2.4 Signed Integer Representation. 2.4 Signed Integer Representation
Data Representation II CMSC 313 Sections 01, 02 The conversions we have so far presented have involved only unsigned numbers. To represent signed integers, computer systems allocate the high-order bit
More informationDiscussion 11. Streams
Discussion 11 Streams A stream is an element and a promise to evaluate the rest of the stream. You ve already seen multiple examples of this and its syntax in lecture and in the books, so I will not dwell
More informationUnder the Hood: Data Representation. Computer Science 104 Lecture 2
Under the Hood: Data Representation Computer Science 104 Lecture 2 Admin Piazza, Sakai Up Everyone should have access Homework 1 Posted Due Feb 6 PDF or Plain Text Only: No Word or RTF Recommended: Learn
More informationFloating Point. CSC207 Fall 2017
Floating Point CSC207 Fall 2017 Ariane 5 Rocket Launch Ariane 5 rocket explosion In 1996, the European Space Agency s Ariane 5 rocket exploded 40 seconds after launch. During conversion of a 64-bit to
More informationFloating point. Today! IEEE Floating Point Standard! Rounding! Floating Point Operations! Mathematical properties. Next time. !
Floating point Today! IEEE Floating Point Standard! Rounding! Floating Point Operations! Mathematical properties Next time! The machine model Chris Riesbeck, Fall 2011 Checkpoint IEEE Floating point Floating
More informationCOSC 243. Data Representation 3. Lecture 3 - Data Representation 3 1. COSC 243 (Computer Architecture)
COSC 243 Data Representation 3 Lecture 3 - Data Representation 3 1 Data Representation Test Material Lectures 1, 2, and 3 Tutorials 1b, 2a, and 2b During Tutorial a Next Week 12 th and 13 th March If you
More informationNumber Systems. The Computer Works in Binary, or how I learned to think like a computer. The computer s natural number system is binary not decimal.
PROGRAMMING CONCEPTS Number Systems The Computer Works in Binary, or how I learned to think like a computer Copyright 2013 Dan McElroy The computer s natural number system is binary not decimal. For example,
More informationThese are notes for the third lecture; if statements and loops.
These are notes for the third lecture; if statements and loops. 1 Yeah, this is going to be the second slide in a lot of lectures. 2 - Dominant language for desktop application development - Most modern
More informationDivide: Paper & Pencil
Divide: Paper & Pencil 1001 Quotient Divisor 1000 1001010 Dividend -1000 10 101 1010 1000 10 Remainder See how big a number can be subtracted, creating quotient bit on each step Binary => 1 * divisor or
More informationMore Programming Constructs -- Introduction
More Programming Constructs -- Introduction We can now examine some additional programming concepts and constructs Chapter 5 focuses on: internal data representation conversions between one data type and
More informationThese are reserved words of the C language. For example int, float, if, else, for, while etc.
Tokens in C Keywords These are reserved words of the C language. For example int, float, if, else, for, while etc. Identifiers An Identifier is a sequence of letters and digits, but must start with a letter.
More informationNumber Systems. Both numbers are positive
Number Systems Range of Numbers and Overflow When arithmetic operation such as Addition, Subtraction, Multiplication and Division are performed on numbers the results generated may exceed the range of
More informationFloating Point. CSE 351 Autumn Instructor: Justin Hsia
Floating Point CSE 351 Autumn 2016 Instructor: Justin Hsia Teaching Assistants: Chris Ma Hunter Zahn John Kaltenbach Kevin Bi Sachin Mehta Suraj Bhat Thomas Neuman Waylon Huang Xi Liu Yufang Sun http://xkcd.com/899/
More informationWhat we need to know about error: Class Outline. Computational Methods CMSC/AMSC/MAPL 460. Errors in data and computation
Class Outline Computational Methods CMSC/AMSC/MAPL 460 Errors in data and computation Representing numbers in floating point Ramani Duraiswami, Dept. of Computer Science Computations should be as accurate
More informationCS102: Variables and Expressions
CS102: Variables and Expressions The topic of variables is one of the most important in C or any other high-level programming language. We will start with a simple example: int x; printf("the value of
More informationComputer Systems C S Cynthia Lee
Computer Systems C S 1 0 7 Cynthia Lee 2 Today s Topics LECTURE: Floating point! Real Numbers and Approximation MATH TIME! Some preliminary observations on approximation We know that some non-integer numbers
More informationThe Perils of Floating Point
The Perils of Floating Point by Bruce M. Bush Copyright (c) 1996 Lahey Computer Systems, Inc. Permission to copy is granted with acknowledgement of the source. Many great engineering and scientific advances
More information(Refer Slide Time: 02:59)
Numerical Methods and Programming P. B. Sunil Kumar Department of Physics Indian Institute of Technology, Madras Lecture - 7 Error propagation and stability Last class we discussed about the representation
More informationAs stated earlier, the declaration
The int data type As stated earlier, the declaration int a; is an instruction to the compiler to reserve a certain amount of memory to hold the values of the variable a. How much memory? Two bytes (usually,
More informationCS 261 Fall Mike Lam, Professor Integer Encodings
CS 261 Fall 2018 Mike Lam, Professor https://xkcd.com/571/ Integer Encodings Integers Topics C integer data types Unsigned encoding Signed encodings Conversions Integer data types in C99 1 byte 2 bytes
More informationSlide Set 1. for ENEL 339 Fall 2014 Lecture Section 02. Steve Norman, PhD, PEng
Slide Set 1 for ENEL 339 Fall 2014 Lecture Section 02 Steve Norman, PhD, PEng Electrical & Computer Engineering Schulich School of Engineering University of Calgary Fall Term, 2014 ENEL 353 F14 Section
More informationLearning from Math Library Testng for C Marcel Beemster Solid Sands
Learning from Math Library Testng for C Marcel Beemster Solid Sands Introduction In the process of improving SuperTest, I recently dived into its math library testing. Turns out there were some interesting
More informationCS321 Introduction To Numerical Methods
CS3 Introduction To Numerical Methods Fuhua (Frank) Cheng Department of Computer Science University of Kentucky Lexington KY 456-46 - - Table of Contents Errors and Number Representations 3 Error Types
More informationComputational Methods CMSC/AMSC/MAPL 460. Representing numbers in floating point and associated issues. Ramani Duraiswami, Dept. of Computer Science
Computational Methods CMSC/AMSC/MAPL 460 Representing numbers in floating point and associated issues Ramani Duraiswami, Dept. of Computer Science Class Outline Computations should be as accurate and as
More informationDivisibility Rules and Their Explanations
Divisibility Rules and Their Explanations Increase Your Number Sense These divisibility rules apply to determining the divisibility of a positive integer (1, 2, 3, ) by another positive integer or 0 (although
More informationFloating-Point Numbers in Digital Computers
POLYTECHNIC UNIVERSITY Department of Computer and Information Science Floating-Point Numbers in Digital Computers K. Ming Leung Abstract: We explain how floating-point numbers are represented and stored
More information2.1.1 Fixed-Point (or Integer) Arithmetic
x = approximation to true value x error = x x, relative error = x x. x 2.1.1 Fixed-Point (or Integer) Arithmetic A base 2 (base 10) fixed-point number has a fixed number of binary (decimal) places. 1.
More information1.3 Floating Point Form
Section 1.3 Floating Point Form 29 1.3 Floating Point Form Floating point numbers are used by computers to approximate real numbers. On the surface, the question is a simple one. There are an infinite
More informationFloating-Point Numbers in Digital Computers
POLYTECHNIC UNIVERSITY Department of Computer and Information Science Floating-Point Numbers in Digital Computers K. Ming Leung Abstract: We explain how floating-point numbers are represented and stored
More informationIntroduction to Computers and Programming. Numeric Values
Introduction to Computers and Programming Prof. I. K. Lundqvist Lecture 5 Reading: B pp. 47-71 Sept 1 003 Numeric Values Storing the value of 5 10 using ASCII: 00110010 00110101 Binary notation: 00000000
More informationFloating-point representation
Lecture 3-4: Floating-point representation and arithmetic Floating-point representation The notion of real numbers in mathematics is convenient for hand computations and formula manipulations. However,
More informationCITS 1401 Problem Solving & Programming
CITS 1401 Problem Solving & Programming Tim French Lecture 03 Numeric Data Processing (These slides are based on John Zelle s powerpoint slides for lectures accompanied with the text book) Objectives of
More informationFOR Loops. Last Modified: 01 June / 1
FOR Loops http://people.sc.fsu.edu/ jburkardt/isc/week04 lecture 08.pdf... ISC3313: Introduction to Scientific Computing with C++ Summer Semester 2011... John Burkardt Department of Scientific Computing
More informationFloating-Point Arithmetic
Floating-Point Arithmetic Raymond J. Spiteri Lecture Notes for CMPT 898: Numerical Software University of Saskatchewan January 9, 2013 Objectives Floating-point numbers Floating-point arithmetic Analysis
More informationScientific Computing. Error Analysis
ECE257 Numerical Methods and Scientific Computing Error Analysis Today s s class: Introduction to error analysis Approximations Round-Off Errors Introduction Error is the difference between the exact solution
More information8/30/2016. In Binary, We Have A Binary Point. ECE 120: Introduction to Computing. Fixed-Point Representations Support Fractions
University of Illinois at Urbana-Champaign Dept. of Electrical and Computer Engineering ECE 120: Introduction to Computing Fixed- and Floating-Point Representations In Binary, We Have A Binary Point Let
More informationComputer Architecture Set Four. Arithmetic
Computer Architecture Set Four Arithmetic Arithmetic Where we ve been: Performance (seconds, cycles, instructions) Abstractions: Instruction Set Architecture Assembly Language and Machine Language What
More informationIEEE Standard 754 Floating Point Numbers
IEEE Standard 754 Floating Point Numbers Steve Hollasch / Last update 2005-Feb-24 IEEE Standard 754 floating point is the most common representation today for real numbers on computers, including Intel-based
More informationData Types. Data Types. Integer Types. Signed Integers
Data Types Data Types Dr. TGI Fernando 1 2 The fundamental building blocks of any programming language. What is a data type? A data type is a set of values and a set of operations define on these values.
More informationB ABC is mapped into A'B'C'
h. 00 Transformations Sec. 1 Mappings & ongruence Mappings Moving a figure around a plane is called mapping. In the figure below, was moved (mapped) to a new position in the plane and the new triangle
More informationTopic Notes: Bits and Bytes and Numbers
Computer Science 220 Assembly Language & Comp Architecture Siena College Fall 2010 Topic Notes: Bits and Bytes and Numbers Binary Basics At least some of this will be review, but we will go over it for
More informationPython Programming: An Introduction to Computer Science
Python Programming: An Introduction to Computer Science Chapter 3 Computing with Numbers Python Programming, 2/e 1 Objectives n To understand the concept of data types. n To be familiar with the basic
More informationLecture Notes: Floating-Point Numbers
Lecture Notes: Floating-Point Numbers CS227-Scientific Computing September 8, 2010 What this Lecture is About How computers represent numbers How this affects the accuracy of computation Positional Number
More informationFloating Point Arithmetic
Floating Point Arithmetic Computer Systems, Section 2.4 Abstraction Anything that is not an integer can be thought of as . e.g. 391.1356 Or can be thought of as + /
More informationAMTH142 Lecture 10. Scilab Graphs Floating Point Arithmetic
AMTH142 Lecture 1 Scilab Graphs Floating Point Arithmetic April 2, 27 Contents 1.1 Graphs in Scilab......................... 2 1.1.1 Simple Graphs...................... 2 1.1.2 Line Styles........................
More informationSlide Set 1. for ENEL 353 Fall Steve Norman, PhD, PEng. Electrical & Computer Engineering Schulich School of Engineering University of Calgary
Slide Set 1 for ENEL 353 Fall 2017 Steve Norman, PhD, PEng Electrical & Computer Engineering Schulich School of Engineering University of Calgary Fall Term, 2017 SN s ENEL 353 Fall 2017 Slide Set 1 slide
More informationComputer Architecture and Organization
3-1 Chapter 3 - Arithmetic Computer Architecture and Organization Miles Murdocca and Vincent Heuring Chapter 3 Arithmetic 3-2 Chapter 3 - Arithmetic Chapter Contents 3.1 Fixed Point Addition and Subtraction
More informationFloating Point Numbers
Floating Point Numbers Computer Systems Organization (Spring 2016) CSCI-UA 201, Section 2 Instructor: Joanna Klukowska Slides adapted from Randal E. Bryant and David R. O Hallaron (CMU) Mohamed Zahran
More informationFloating Point Numbers
Floating Point Numbers Computer Systems Organization (Spring 2016) CSCI-UA 201, Section 2 Fractions in Binary Instructor: Joanna Klukowska Slides adapted from Randal E. Bryant and David R. O Hallaron (CMU)
More informationCMPSCI 145 MIDTERM #1 Solution Key. SPRING 2017 March 3, 2017 Professor William T. Verts
CMPSCI 145 MIDTERM #1 Solution Key NAME SPRING 2017 March 3, 2017 PROBLEM SCORE POINTS 1 10 2 10 3 15 4 15 5 20 6 12 7 8 8 10 TOTAL 100 10 Points Examine the following diagram of two systems, one involving
More information