Parallel Systems. Part 7: Evaluation of Computers and Programs. foils by Yang-Suk Kee, X. Sun, T. Fahringer
|
|
- Grant Anthony
- 6 years ago
- Views:
Transcription
1 Parallel Systems Part 7: Evaluation of Computers and Programs foils by Yang-Suk Kee, X. Sun, T. Fahringer
2 How To Evaluate Computers and Programs? Learning objectives: Predict performance of parallel programs on parallel computers Understand barriers to higher performance Simulation-based evaluation Accurate simulators are costly to develop and verify Simulation is time-consuming Sometimes this is done for machines not yet existing. Quantitative evaluation A grounded engineering discipline Standard benchmarks Understanding of parallel programs as workloads is critical!
3 Workload Classification Serial type : increase throughput Single application runs serially Batch processing number of jobs per time unit I/O issue: network bandwidth >= aggregate I/O requirements Workload management Interactive Transaction processing Multi-user logins Multi-job serial Parametric computation
4 Workload Classification (Cont d) Parallel type : turn-around time or response time Single application run on multiple nodes Workloads Workload with large effort Workload with minimum effort
5 Workload with Large Efforts Grand Challenge Problems PetaFLOP levels of computation Fundamental problems in science and engineering with broad application (ex) computational fluid dynamics for weather forecasting Academic research thesis making Heavily used programs Databases, OLTP servers, Internet servers, Online Games, Stocks prediction, etc. Aggressive parallelization effort should be justified
6 Workload with Minimum Efforts Commercial Transaction Processing Systems Inter-transaction parallelism: multiple transactions at the same time Intra-application parallelism: parallelism within a single database operation: learn how to express queries for best parallel execution
7 Performance Improvement (Speedup) When work is fixed speedup ( p) = performance( p) performance(1) speedup ( p) = time(1) time( p) Basic measures of multiprocessor performance efficiency( p) = speedup( p p)
8 Scaling Problem (Small Work) Appropriate for small machine Parallelism overheads begin to dominate benefits for larger machines Load imbalance Communication to computation ratio May even achieve slowdowns
9 Scaling Problem (Large Work) Appropriate for big machine Difficult to measure improvement May not fit for small machine Can t run Thrashing to disk Working set doesn t fit in cache Fits at some p, leading to superlinear speedup
10 Demonstrating Scaling Problems Small Ocean problem On SGI Origin2000 Big equation solver problem On SGI Origin2000 parallelism overhead superlinear User want to scale problems as machines grow!
11 Definitions Scaling a machine Make a machine more powerful Machine size <processor, memory, communication, I/O> Scaling a machine in parallel processing Add more identical nodes Problem size Input configuration data set size: the amount of storage required to run it on a single processor memory usage: the amount of memory used by the program
12 Two Key Issues in Problem Scaling Under what constraints should the problem be scaled? Some properties must be fixed as the machine scales How should the problem be scaled? Which parameters? How?
13 Constraints To Scale Two types of constraints User-oriented Easy to think about change e.g. particles, rows, transactions Resource-oriented e.g. Memory, time
14 Resource-Oriented Constraints Problem constrained (PC) Problem size fixed Memory constrained (MC) Memory size fixed Time constrained (TC) Execution time fixed
15 Some Definitions t s : Processing time of the serial part of a program (using 1 processor). t p (1) : Processing time of the parallel part of the program using 1 processor. t p (P) : Processing time of the parallel part of the program using P processors. T(1) : Total processing time of the program including both the serial and the parallel parts using 1 processor = t s + t p (1) T(P) : Total processing time of the program including both the serial and the parallel parts using P processor = t s + t p (P)
16 Problem Constrained Scaling: Amdahl s law The main objective is to produce the results as soon as possible (turnaround time) (ex) video compression, computer graphics, VLSI routing, etc main usage of Amdahl s and Gustafson s laws: estimate speedup as a measure to determine a program s potential for parallelism Implications Upper-bound is 1/α Decrease serial part as much as possible Optimize the common case Modified Amdahl s law for fixed problem size including the overhead
17 Fixed-Size Speedup (Amdahl Law, 67) Amount of Work W 1 W 1 W 1 W 1 W 1 Elapsed Time T 1 T 1 T1 W p W p W p W p W p T p T 1 T1 T p Tp T p T p Number of Processors (p) Number of Processors (p)
18 Limitations of Amdahl s Law Ignores performance overhead (e.g. communication, load imbalance) Overestimates speedup achievable
19 Enhanced Amdahl s Law The overhead includes parallelism and interaction overheads Speedup PC = T as (1 α) T T p 1 overhead αt Toverhead α + p T Amdahl s law: argument against massively parallel systems
20 Speedup PC Amdahl Effect = T as (1 α) T T p 1 overhead αt Toverhead α + p T Typically T overhead has lower complexity than (1-α)T 1 /p As problem size n increases (1-α)T 1 /p dominates T overhead As problem size n increases, speedup increases
21 Illustration of Amdahl Effect Speedup n = 10,000 n = 1,000 Processors n = 100
22 Review of Amdahl s Law Treats problem size as a constant Shows how execution time decreases as number of processors increases
23 Another Perspective We often use faster computers to solve larger problem instances Let s treat time as a constant and allow problem size to increase with number of processors
24 Time Constrained Scaling: Gustafson s Law User wants more accurate results within a time limit. Execution time is fixed as system scales (ex) FEM for structural analysis, FDM for fluid dynamics Properties of a work metric Easy to measure Architecture independent Easy to model with an analytical expression The measure of work should scale linearly with sequential time complexity of the algorithm
25 Gustafson s Law (Without Overhead) α 1-α time α = t t ( p) p (1-α)p s + p t s Speedup TC = Work( p) Work(1) = αw + (1 α ) pw W = α + (1 α) p p is the nr. of processors
26 Fixed-Time Speedup (Gustafson) W 1 Amount of Work W 1 W 1 Elapsed Time W 1 W 1 Wp W p W p W p T 1 T 1 T 1 T 1 T 1 W p T p T p T p T p T p Number of Processors (p) Number of Processors (p)
27 Converting α s between Amdahl s and Gustafon s laws αa = 1+ 1 (1 αg ). n Based on this observation, Amdahl s and Gustafon s laws are identical. αg αa ( n n 1) + 1 = αg + (1 αg ) n
28 Memory Constrained Scaling: Sun and Ni s Law Scale the largest possible solution limited by the memory space. Or, fix memory usage per processor e.g. N-body problem problem size is scaled from W to W* W* is the work executed under memory limitation of a parallel computer * T(1, W Speedup MC = T(P, W * * ) )
29 Memory-Boundary Speedup (Sun & Ni) Work executed under memory limitation Hierarchical memory W 1 Amount of Work W 1 W 1 Elapsed Time W 1 Wp W p W p W 1 W p W p T 1 T p T 1 T 1 T p T p T 1 T 1 T p T p Number of Processors (p) Number of Processors (p)
30 Parallel Performance Metrics (Run-time is the dominant metric) Run-Time (Execution Time) Speed: mflops, mips Speedup Efficiency: E = Scalability Speedup Number of Processors
31 Scalability The Need for New Metrics Comparison of performances with different workload Availability of massively parallel processing Definition: Scalability Ability to maintain parallel processing gain when both problem size and machine size increase
32 Ideally Scalable T(m p, m W) = T(p, W) T: execution time W: work executed p: number of processors used m: scale up m times work: flop count based on the best practical serial algorithm Fact: T(m p, m W) = T(p, W) if and only if the average unit speed is fixed
33 Definition (average unit speed): The average unit speed is the achieved (work) divided by the number of processors Definition (Isospeed Scalability): An algorithm-machine combination is scalable if the achieved average unit speed can remain constant with increasing numbers of processors, provided the problem size is increased proportionally
34 Issoefficiency Parallel system: parallel program executing on a parallel computer Scalability of a parallel system: measure of its ability to increase performance as the number of processors increases A scalable system maintains efficiency as processors are added Isoefficiency: way to measure scalability
35 Isospeed Scalability (Sun & Rover, 91) W: work executed when p processors are employed W': work executed when p' > p processors are employed to maintain the average speed Ideal case p' W W ' = p Scalabilit y =ψ ( p, p') = Scalability in terms of time ψ ( p, p' ) T = T p p', W p W ' p' = ψ ( p, p') = 1 p' W p W ' ( W ) timefor workw on p processors = ( W ') timefor workw 'on p' processors
36 The Relation of Scalability and Time More scalable leads to smaller time Better initial run-time and higher scalability lead to superior run-time Same initial run-time and same scalability lead to same scaled performance Superior initial performance may not last long if scalability is low
37 Summary (1/3) Performance terms Speedup Efficiency Model of speedup Serial component Parallel component overhead component
38 Summary (2/3) What prevents linear speedup? Serial operations Communication operations Process start-up Imbalanced workloads Architectural limitations
39 Summary (3/3) Analyzing parallel performance Amdahl s Law Gustafson s Law Sun-Ni s Law Isoefficiency metric
Outline. Speedup & Efficiency Amdahl s Law Gustafson s Law Sun & Ni s Law. Khoa Khoa học và Kỹ thuật Máy tính - ĐHBK TP.HCM
Speedup Thoai am Outline Speedup & Efficiency Amdahl s Law Gustafson s Law Sun & i s Law Speedup & Efficiency Speedup: S = Time(the most efficient sequential Efficiency: E = S / algorithm) / Time(parallel
More informationOutline. Speedup & Efficiency Amdahl s Law Gustafson s Law Sun & Ni s Law. Khoa Khoa học và Kỹ thuật Máy tính - ĐHBK TP.HCM
Speedup Thoai am Outline Speedup & Efficiency Amdahl s Law Gustafson s Law Sun & i s Law Speedup & Efficiency Speedup: S = T seq T par - T seq : Time(the most efficient sequential algorithm) - T par :
More informationECE 669 Parallel Computer Architecture
ECE 669 Parallel Computer Architecture Lecture 9 Workload Evaluation Outline Evaluation of applications is important Simulation of sample data sets provides important information Working sets indicate
More informationCSC630/CSC730 Parallel & Distributed Computing
CSC630/CSC730 Parallel & Distributed Computing Analytical Modeling of Parallel Programs Chapter 5 1 Contents Sources of Parallel Overhead Performance Metrics Granularity and Data Mapping Scalability 2
More informationParallel Programming with MPI and OpenMP
Parallel Programming with MPI and OpenMP Michael J. Quinn (revised by L.M. Liebrock) Chapter 7 Performance Analysis Learning Objectives Predict performance of parallel programs Understand barriers to higher
More informationAdvanced Topics UNIT 2 PERFORMANCE EVALUATIONS
Advanced Topics UNIT 2 PERFORMANCE EVALUATIONS Structure Page Nos. 2.0 Introduction 4 2. Objectives 5 2.2 Metrics for Performance Evaluation 5 2.2. Running Time 2.2.2 Speed Up 2.2.3 Efficiency 2.3 Factors
More informationAnalytical Modeling of Parallel Systems. To accompany the text ``Introduction to Parallel Computing'', Addison Wesley, 2003.
Analytical Modeling of Parallel Systems To accompany the text ``Introduction to Parallel Computing'', Addison Wesley, 2003. Topic Overview Sources of Overhead in Parallel Programs Performance Metrics for
More informationOutline. CSC 447: Parallel Programming for Multi- Core and Cluster Systems
CSC 447: Parallel Programming for Multi- Core and Cluster Systems Performance Analysis Instructor: Haidar M. Harmanani Spring 2018 Outline Performance scalability Analytical performance measures Amdahl
More informationParallel Programming with MPI and OpenMP
Parallel Programming with MPI and OpenMP Michael J. Quinn Chapter 7 Performance Analysis Learning Objectives n Predict performance of parallel programs n Understand barriers to higher performance Outline
More informationThe typical speedup curve - fixed problem size
Performance analysis Goals are 1. to be able to understand better why your program has the performance it has, and 2. what could be preventing its performance from being better. The typical speedup curve
More informationScalability of Heterogeneous Computing
Scalability of Heterogeneous Computing Xian-He Sun, Yong Chen, Ming u Department of Computer Science Illinois Institute of Technology {sun, chenyon1, wuming}@iit.edu Abstract Scalability is a key factor
More informationDesign of Parallel Algorithms. Course Introduction
+ Design of Parallel Algorithms Course Introduction + CSE 4163/6163 Parallel Algorithm Analysis & Design! Course Web Site: http://www.cse.msstate.edu/~luke/courses/fl17/cse4163! Instructor: Ed Luke! Office:
More informationAnalytical Modeling of Parallel Programs
Analytical Modeling of Parallel Programs Alexandre David Introduction to Parallel Computing 1 Topic overview Sources of overhead in parallel programs. Performance metrics for parallel systems. Effect of
More informationCSE5351: Parallel Processing Part III
CSE5351: Parallel Processing Part III -1- Performance Metrics and Benchmarks How should one characterize the performance of applications and systems? What are user s requirements in performance and cost?
More informationAnalytical Modeling of Parallel Programs
2014 IJEDR Volume 2, Issue 1 ISSN: 2321-9939 Analytical Modeling of Parallel Programs Hardik K. Molia Master of Computer Engineering, Department of Computer Engineering Atmiya Institute of Technology &
More informationChapter 5: Analytical Modelling of Parallel Programs
Chapter 5: Analytical Modelling of Parallel Programs Introduction to Parallel Computing, Second Edition By Ananth Grama, Anshul Gupta, George Karypis, Vipin Kumar Contents 1. Sources of Overhead in Parallel
More informationCSE 613: Parallel Programming. Lecture 2 ( Analytical Modeling of Parallel Algorithms )
CSE 613: Parallel Programming Lecture 2 ( Analytical Modeling of Parallel Algorithms ) Rezaul A. Chowdhury Department of Computer Science SUNY Stony Brook Spring 2017 Parallel Execution Time & Overhead
More informationCMPSCI 691AD General Purpose Computation on the GPU
CMPSCI 691AD General Purpose Computation on the GPU Spring 2009 Lecture 5: Quantitative Analysis of Parallel Algorithms Rui Wang (cont. from last lecture) Device Management Context Management Module Management
More informationWhat is Good Performance. Benchmark at Home and Office. Benchmark at Home and Office. Program with 2 threads Home program.
Performance COMP375 Computer Architecture and dorganization What is Good Performance Which is the best performing jet? Airplane Passengers Range (mi) Speed (mph) Boeing 737-100 101 630 598 Boeing 747 470
More informationUnderstanding Parallelism and the Limitations of Parallel Computing
Understanding Parallelism and the Limitations of Parallel omputing Understanding Parallelism: Sequential work After 16 time steps: 4 cars Scalability Laws 2 Understanding Parallelism: Parallel work After
More informationParallel DBMS. Parallel Database Systems. PDBS vs Distributed DBS. Types of Parallelism. Goals and Metrics Speedup. Types of Parallelism
Parallel DBMS Parallel Database Systems CS5225 Parallel DB 1 Uniprocessor technology has reached its limit Difficult to build machines powerful enough to meet the CPU and I/O demands of DBMS serving large
More informationCOURSE 12. Parallel DBMS
COURSE 12 Parallel DBMS 1 Parallel DBMS Most DB research focused on specialized hardware CCD Memory: Non-volatile memory like, but slower than flash memory Bubble Memory: Non-volatile memory like, but
More informationLecture 10: Performance Metrics. Shantanu Dutt ECE Dept. UIC
Lecture 10: Performance Metrics Shantanu Dutt ECE Dept. UIC Acknowledgement Adapted from Chapter 5 slides of the text, by A. Grama w/ a few changes, augmentations and corrections in colored text by Shantanu
More informationSlides compliment of Yong Chen and Xian-He Sun From paper Reevaluating Amdahl's Law in the Multicore Era. 11/16/2011 Many-Core Computing 2
Slides compliment of Yong Chen and Xian-He Sun From paper Reevaluating Amdahl's Law in the Multicore Era 11/16/2011 Many-Core Computing 2 Gene M. Amdahl, Validity of the Single-Processor Approach to Achieving
More information18-447: Computer Architecture Lecture 30B: Multiprocessors. Prof. Onur Mutlu Carnegie Mellon University Spring 2013, 4/22/2013
18-447: Computer Architecture Lecture 30B: Multiprocessors Prof. Onur Mutlu Carnegie Mellon University Spring 2013, 4/22/2013 Readings: Multiprocessing Required Amdahl, Validity of the single processor
More informationNOW Handout Page 1. Recap. Protocol Design Space of Snooping Cache Coherent Multiprocessors. Sequential Consistency.
Recap Protocol Design Space of Snooping Cache Coherent ultiprocessors CS 28, Spring 99 David E. Culler Computer Science Division U.C. Berkeley Snooping cache coherence solve difficult problem by applying
More informationLecture 7: Parallel Processing
Lecture 7: Parallel Processing Introduction and motivation Architecture classification Performance evaluation Interconnection network Zebo Peng, IDA, LiTH 1 Performance Improvement Reduction of instruction
More informationParallel Computing. Hwansoo Han (SKKU)
Parallel Computing Hwansoo Han (SKKU) Unicore Limitations Performance scaling stopped due to Power consumption Wire delay DRAM latency Limitation in ILP 10000 SPEC CINT2000 2 cores/chip Xeon 3.0GHz Core2duo
More informationPerformance analysis. Performance analysis p. 1
Performance analysis Performance analysis p. 1 An example of time measurements Dark grey: time spent on computation, decreasing with p White: time spent on communication, increasing with p Performance
More informationHomework # 2 Due: October 6. Programming Multiprocessors: Parallelism, Communication, and Synchronization
ECE669: Parallel Computer Architecture Fall 2 Handout #2 Homework # 2 Due: October 6 Programming Multiprocessors: Parallelism, Communication, and Synchronization 1 Introduction When developing multiprocessor
More informationReview: Creating a Parallel Program. Programming for Performance
Review: Creating a Parallel Program Can be done by programmer, compiler, run-time system or OS Steps for creating parallel program Decomposition Assignment of tasks to processes Orchestration Mapping (C)
More informationIntroduction to Modeling. Lecture Overview
Lecture Overview What is a Model? Uses of Modeling The Modeling Process Pose the Question Define the Abstractions Create the Model Analyze the Data Model Representations * Queuing Models * Petri Nets *
More informationSCALABILITY ANALYSIS
SCALABILITY ANALYSIS PERFORMANCE AND SCALABILITY OF PARALLEL SYSTEMS Evaluation Sequential: runtime (execution time) Ts =T (InputSize) Parallel: runtime (start-->last PE ends) Tp =T (InputSize,p,architecture)
More informationΠαράλληλη Επεξεργασία
Παράλληλη Επεξεργασία Μέτρηση και σύγκριση Παράλληλης Απόδοσης Γιάννος Σαζεϊδης Εαρινό Εξάμηνο 2013 HW 1. Homework #3 due on cuda (summary of Tesla paper on web page) Slides based on Lin and Snyder textbook
More informationSuperlinear Speedup in Parallel Computation
Superlinear Speedup in Parallel Computation Jing Shan jshan@ccs.neu.edu 1 Introduction to The Problem Because of its good speedup, parallel computing becomes more and more important in scientific computations,
More informationHigh Performance Computing Systems
High Performance Computing Systems Shared Memory Doug Shook Shared Memory Bottlenecks Trips to memory Cache coherence 2 Why Multicore? Shared memory systems used to be purely the domain of HPC... What
More informationPage 1. Program Performance Metrics. Program Performance Metrics. Amdahl s Law. 1 seq seq 1
Program Performance Metrics The parallel run time (Tpar) is the time from the moment when computation starts to the moment when the last processor finished his execution The speedup (S) is defined as the
More information1 Introduction to Parallel Computing
1 Introduction to Parallel Computing 1.1 Goals of Parallel vs Distributed Computing Distributed computing, commonly represented by distributed services such as the world wide web, is one form of computational
More informationComputer Architecture: Parallel Processing Basics. Prof. Onur Mutlu Carnegie Mellon University
Computer Architecture: Parallel Processing Basics Prof. Onur Mutlu Carnegie Mellon University Readings Required Hill, Jouppi, Sohi, Multiprocessors and Multicomputers, pp. 551-560 in Readings in Computer
More informationComputer Architecture Lecture 27: Multiprocessors. Prof. Onur Mutlu Carnegie Mellon University Spring 2015, 4/6/2015
18-447 Computer Architecture Lecture 27: Multiprocessors Prof. Onur Mutlu Carnegie Mellon University Spring 2015, 4/6/2015 Assignments Lab 7 out Due April 17 HW 6 Due Friday (April 10) Midterm II April
More informationECE Spring 2017 Exam 2
ECE 56300 Spring 2017 Exam 2 All questions are worth 5 points. For isoefficiency questions, do not worry about breaking costs down to t c, t w and t s. Question 1. Innovative Big Machines has developed
More informationOverview of High Performance Computing
Overview of High Performance Computing Timothy H. Kaiser, PH.D. tkaiser@mines.edu http://inside.mines.edu/~tkaiser/csci580fall13/ 1 Near Term Overview HPC computing in a nutshell? Basic MPI - run an example
More informationUnit 9 : Fundamentals of Parallel Processing
Unit 9 : Fundamentals of Parallel Processing Lesson 1 : Types of Parallel Processing 1.1. Learning Objectives On completion of this lesson you will be able to : classify different types of parallel processing
More informationCourse web site: teaching/courses/car. Piazza discussion forum:
Announcements Course web site: http://www.inf.ed.ac.uk/ teaching/courses/car Lecture slides Tutorial problems Courseworks Piazza discussion forum: http://piazza.com/ed.ac.uk/spring2018/car Tutorials start
More informationIndex. ADEPT (tool for modelling proposed systerns),
Index A, see Arrivals Abstraction in modelling, 20-22, 217 Accumulated time in system ( w), 42 Accuracy of models, 14, 16, see also Separable models, robustness Active customer (memory constrained system),
More informationParallel SimOS: Scalability and Performance for Large System Simulation
Parallel SimOS: Scalability and Performance for Large System Simulation Ph.D. Oral Defense Robert E. Lantz Computer Systems Laboratory Stanford University 1 Overview This work develops methods to simulate
More informationComputer and Information Sciences College / Computer Science Department CS 207 D. Computer Architecture
Computer and Information Sciences College / Computer Science Department CS 207 D Computer Architecture The Computer Revolution Progress in computer technology Underpinned by Moore s Law Makes novel applications
More informationMeasuring Performance. Speed-up, Amdahl s Law, Gustafson s Law, efficiency, benchmarks
Measuring Performance Speed-up, Amdahl s Law, Gustafson s Law, efficiency, benchmarks Why Measure Performance? Performance tells you how you are doing and whether things can be improved appreciably When
More informationIssues in Parallel Processing. Lecture for CPSC 5155 Edward Bosworth, Ph.D. Computer Science Department Columbus State University
Issues in Parallel Processing Lecture for CPSC 5155 Edward Bosworth, Ph.D. Computer Science Department Columbus State University Introduction Goal: connecting multiple computers to get higher performance
More informationDesigning for Performance. Patrick Happ Raul Feitosa
Designing for Performance Patrick Happ Raul Feitosa Objective In this section we examine the most common approach to assessing processor and computer system performance W. Stallings Designing for Performance
More informationParallelization Principles. Sathish Vadhiyar
Parallelization Principles Sathish Vadhiyar Parallel Programming and Challenges Recall the advantages and motivation of parallelism But parallel programs incur overheads not seen in sequential programs
More informationSAS Meets Big Iron: High Performance Computing in SAS Analytic Procedures
SAS Meets Big Iron: High Performance Computing in SAS Analytic Procedures Robert A. Cohen SAS Institute Inc. Cary, North Carolina, USA Abstract Version 9targets the heavy-duty analytic procedures in SAS
More informationAnnouncements. Database Systems CSE 414. Why compute in parallel? Big Data 10/11/2017. Two Kinds of Parallel Data Processing
Announcements Database Systems CSE 414 HW4 is due tomorrow 11pm Lectures 18: Parallel Databases (Ch. 20.1) 1 2 Why compute in parallel? Multi-cores: Most processors have multiple cores This trend will
More informationPerformance. CS 3410 Computer System Organization & Programming. [K. Bala, A. Bracy, E. Sirer, and H. Weatherspoon]
Performance CS 3410 Computer System Organization & Programming [K. Bala, A. Bracy, E. Sirer, and H. Weatherspoon] Performance Complex question How fast is the processor? How fast your application runs?
More informationHYRISE In-Memory Storage Engine
HYRISE In-Memory Storage Engine Martin Grund 1, Jens Krueger 1, Philippe Cudre-Mauroux 3, Samuel Madden 2 Alexander Zeier 1, Hasso Plattner 1 1 Hasso-Plattner-Institute, Germany 2 MIT CSAIL, USA 3 University
More informationCO Computer Architecture and Programming Languages CAPL. Lecture 15
CO20-320241 Computer Architecture and Programming Languages CAPL Lecture 15 Dr. Kinga Lipskoch Fall 2017 How to Compute a Binary Float Decimal fraction: 8.703125 Integral part: 8 1000 Fraction part: 0.703125
More informationLecture 7: Parallel Processing
Lecture 7: Parallel Processing Introduction and motivation Architecture classification Performance evaluation Interconnection network Zebo Peng, IDA, LiTH 1 Performance Improvement Reduction of instruction
More informationLecture - 4. Measurement. Dr. Soner Onder CS 4431 Michigan Technological University 9/29/2009 1
Lecture - 4 Measurement Dr. Soner Onder CS 4431 Michigan Technological University 9/29/2009 1 Acknowledgements David Patterson Dr. Roger Kieckhafer 9/29/2009 2 Computer Architecture is Design and Analysis
More informationLecture 9: Workload-Driven Performance Evaluation. Parallel Computer Architecture and Programming CMU /15-618, Spring 2017
Lecture 9: Workload-Driven Performance Evaluation Parallel Computer Architecture and Programming CMU 15-418/15-618, Spring 2017 Tunes Nina Simone Feeling Good (I Put a Spell on You) 72/72 on Assignment
More informationProgramming as Successive Refinement. Partitioning for Performance
Programming as Successive Refinement Not all issues dealt with up front Partitioning often independent of architecture, and done first View machine as a collection of communicating processors balancing
More informationCS533 Modeling and Performance Evaluation of Network and Computer Systems
CS533 Modeling and Performance Evaluation of Network and Computer Systems Selection of Techniques and Metrics (Chapter 3) 1 Overview One or more systems, real or hypothetical You want to evaluate their
More informationScalability of Processing on GPUs
Scalability of Processing on GPUs Keith Kelley, CS6260 Final Project Report April 7, 2009 Research description: I wanted to figure out how useful General Purpose GPU computing (GPGPU) is for speeding up
More informationMean Value Analysis and Related Techniques
Mean Value Analysis and Related Techniques 34-1 Overview 1. Analysis of Open Queueing Networks 2. Mean-Value Analysis 3. Approximate MVA 4. Balanced Job Bounds 34-2 Analysis of Open Queueing Networks Used
More informationCS533 Modeling and Performance Evaluation of Network and Computer Systems
CS533 Modeling and Performance Evaluation of Network and Computer s Selection of Techniques and Metrics Overview One or more systems, real or hypothetical You want to evaluate their performance What technique
More informationIntroduction to Parallel and Distributed Computing. Linh B. Ngo CPSC 3620
Introduction to Parallel and Distributed Computing Linh B. Ngo CPSC 3620 Overview: What is Parallel Computing To be run using multiple processors A problem is broken into discrete parts that can be solved
More informationLecture 2: Parallel Programs. Topics: consistency, parallel applications, parallelization process
Lecture 2: Parallel Programs Topics: consistency, parallel applications, parallelization process 1 Sequential Consistency A multiprocessor is sequentially consistent if the result of the execution is achievable
More informationQuiz for Chapter 1 Computer Abstractions and Technology
Date: Not all questions are of equal difficulty. Please review the entire quiz first and then budget your time carefully. Name: Course: Solutions in Red 1. [15 points] Consider two different implementations,
More informationPerformance Evaluation for Parallel Systems: A Survey
Performance Evaluation for Parallel Systems: A Survey Lei Hu and Ian Gorton Department of Computer Systems School of Computer Science and Engineering University of NSW, Sydney 2052, Australia E-mail: {lei,
More informationBackground Heterogeneous Architectures Performance Modeling Single Core Performance Profiling Multicore Performance Estimation Test Cases Multicore
By Dan Stafford Background Heterogeneous Architectures Performance Modeling Single Core Performance Profiling Multicore Performance Estimation Test Cases Multicore Design Space Results & Observations General
More informationResponse Time and Throughput
Response Time and Throughput Response time How long it takes to do a task Throughput Total work done per unit time e.g., tasks/transactions/ per hour How are response time and throughput affected by Replacing
More informationPerformance Models for Evaluation and Automatic Tuning of Symmetric Sparse Matrix-Vector Multiply
Performance Models for Evaluation and Automatic Tuning of Symmetric Sparse Matrix-Vector Multiply University of California, Berkeley Berkeley Benchmarking and Optimization Group (BeBOP) http://bebop.cs.berkeley.edu
More informationInterconnect Technology and Computational Speed
Interconnect Technology and Computational Speed From Chapter 1 of B. Wilkinson et al., PARAL- LEL PROGRAMMING. Techniques and Applications Using Networked Workstations and Parallel Computers, augmented
More informationSome aspects of parallel program design. R. Bader (LRZ) G. Hager (RRZE)
Some aspects of parallel program design R. Bader (LRZ) G. Hager (RRZE) Finding exploitable concurrency Problem analysis 1. Decompose into subproblems perhaps even hierarchy of subproblems that can simultaneously
More informationCOMPUTER ORGANIZATION AND DESIGN The Hardware/Software Interface. 5 th. Edition. Chapter 1. Computer Abstractions and Technology
COMPUTER ORGANIZATION AND DESIGN The Hardware/Software Interface 5 th Edition Chapter 1 Computer Abstractions and Technology The Computer Revolution Progress in computer technology Underpinned by Moore
More informationPerformance, Power, Die Yield. CS301 Prof Szajda
Performance, Power, Die Yield CS301 Prof Szajda Administrative HW #1 assigned w Due Wednesday, 9/3 at 5:00 pm Performance Metrics (How do we compare two machines?) What to Measure? Which airplane has the
More informationDell EMC Ready Bundle for HPC Digital Manufacturing Dassault Systѐmes Simulia Abaqus Performance
Dell EMC Ready Bundle for HPC Digital Manufacturing Dassault Systѐmes Simulia Abaqus Performance This Dell EMC technical white paper discusses performance benchmarking results and analysis for Simulia
More informationCOSC 6374 Parallel Computation. Analytical Modeling of Parallel Programs (I) Edgar Gabriel Fall Execution Time
COSC 6374 Parallel Computation Analytical Modeling of Parallel Programs (I) Edgar Gabriel Fall 2015 Execution Time Serial runtime T s : time elapsed between beginning and the end of the execution of a
More informationHigh Performance Computing
The Need for Parallelism High Performance Computing David McCaughan, HPC Analyst SHARCNET, University of Guelph dbm@sharcnet.ca Scientific investigation traditionally takes two forms theoretical empirical
More informationThe Role of Performance
Orange Coast College Business Division Computer Science Department CS 116- Computer Architecture The Role of Performance What is performance? A set of metrics that allow us to compare two different hardware
More informationAn Effective Speedup Metric Considering I/O Constraint in Large-scale Parallel Computer Systems
ICACT Transactions on Advanced Communications Technology (TACT) Vol. 5, Issue 2, March 26 86 An Effective peedup Metric Considering I/O Constraint in Large-scale arallel Computer ystems Guilin Cai*, Wei
More informationPerformance COE 403. Computer Architecture Prof. Muhamed Mudawar. Computer Engineering Department King Fahd University of Petroleum and Minerals
Performance COE 403 Computer Architecture Prof. Muhamed Mudawar Computer Engineering Department King Fahd University of Petroleum and Minerals What is Performance? How do we measure the performance of
More informationAmdahl's Law in the Multicore Era
Amdahl's Law in the Multicore Era Explain intuitively why in the asymmetric model, the speedup actually decreases past a certain point of increasing r. The limiting factor of these improved equations and
More informationIntroduction to Data Management CSE 344
Introduction to Data Management CSE 344 Lecture 25: Parallel Databases CSE 344 - Winter 2013 1 Announcements Webquiz due tonight last WQ! J HW7 due on Wednesday HW8 will be posted soon Will take more hours
More informationParallel Programming Patterns. Overview and Concepts
Parallel Programming Patterns Overview and Concepts Outline Practical Why parallel programming? Decomposition Geometric decomposition Task farm Pipeline Loop parallelism Performance metrics and scaling
More informationPARALLEL AND DISTRIBUTED COMPUTING
PARALLEL AND DISTRIBUTED COMPUTING 2013/2014 1 st Semester 2 nd Exam January 29, 2014 Duration: 2h00 - No extra material allowed. This includes notes, scratch paper, calculator, etc. - Give your answers
More informationA Comparative Evaluation of Techniques for Studying Parallel System Performance
A Comparative Evaluation of Techniques for Studying Parallel System Performance Anand Sivasubramaniam Umakishore Ramachandran H. Venkateswaran Technical Report GIT-CC-94/38 September 1994 College of Computing
More informationExtracting Performance and Scalability Metrics from TCP. Baron Schwartz April 2012
Extracting Performance and Scalability Metrics from TCP Baron Schwartz April 2012 Agenda Fundamental Metrics of Performance Capturing TCP Data Part 1: Black-Box Performance Analysis Detecting Stalls and
More informationQuiz for Chapter 1 Computer Abstractions and Technology 3.10
Date: 3.10 Not all questions are of equal difficulty. Please review the entire quiz first and then budget your time carefully. Name: Course: 1. [15 points] Consider two different implementations, M1 and
More informationSelection of Techniques and Metrics
Selection of Techniques and Metrics Raj Jain Washington University in Saint Louis Saint Louis, MO 63130 Jain@cse.wustl.edu These slides are available on-line at: 3-1 Overview Criteria for Selecting an
More informationInstructor Information
CS 203A Advanced Computer Architecture Lecture 1 1 Instructor Information Rajiv Gupta Office: Engg.II Room 408 E-mail: gupta@cs.ucr.edu Tel: (951) 827-2558 Office Times: T, Th 1-2 pm 2 1 Course Syllabus
More informationWorkload-Driven Architectural Evaluation
Workload-Driven Architectural Evaluation Evaluation in Uniprocessors Decisions made only after quantitative evaluation For existing systems: comparison and procurement evaluation For future systems: careful
More informationCS370: System Architecture & Software [Fall 2014] Dept. Of Computer Science, Colorado State University
Frequently asked questions from the previous class survey CS 370: SYSTEM ARCHITECTURE & SOFTWARE [CPU SCHEDULING] Shrideep Pallickara Computer Science Colorado State University OpenMP compiler directives
More information6.2 DATA DISTRIBUTION AND EXPERIMENT DETAILS
Chapter 6 Indexing Results 6. INTRODUCTION The generation of inverted indexes for text databases is a computationally intensive process that requires the exclusive use of processing resources for long
More informationWhich is the best? Measuring & Improving Performance (if planes were computers...) An architecture example
1 Which is the best? 2 Lecture 05 Performance Metrics and Benchmarking 3 Measuring & Improving Performance (if planes were computers...) Plane People Range (miles) Speed (mph) Avg. Cost (millions) Passenger*Miles
More informationHOW TO WRITE PARALLEL PROGRAMS AND UTILIZE CLUSTERS EFFICIENTLY
HOW TO WRITE PARALLEL PROGRAMS AND UTILIZE CLUSTERS EFFICIENTLY Sarvani Chadalapaka HPC Administrator University of California Merced, Office of Information Technology schadalapaka@ucmerced.edu it.ucmerced.edu
More informationThis article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and
This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution
More informationComputer Architecture A Quantitative Approach, Fifth Edition. Chapter 2. Memory Hierarchy Design. Copyright 2012, Elsevier Inc. All rights reserved.
Computer Architecture A Quantitative Approach, Fifth Edition Chapter 2 Memory Hierarchy Design 1 Introduction Programmers want unlimited amounts of memory with low latency Fast memory technology is more
More informationCondusiv s V-locity Server Boosts Performance of SQL Server 2012 by 55%
openbench Labs Executive Briefing: May 20, 2013 Condusiv s V-locity Server Boosts Performance of SQL Server 2012 by 55% Optimizing I/O for Increased Throughput and Reduced Latency on Physical Servers 01
More informationCOMPUTER ORGANIZATION AND DESIGN. 5 th Edition. The Hardware/Software Interface. Chapter 1. Computer Abstractions and Technology
COMPUTER ORGANIZATION AND DESIGN The Hardware/Software Interface 5 th Edition Chapter 1 Computer Abstractions and Technology Classes of Computers Personal computers General purpose, variety of software
More informationCSC2/458 Parallel and Distributed Systems Machines and Models
CSC2/458 Parallel and Distributed Systems Machines and Models Sreepathi Pai January 23, 2018 URCS Outline Recap Scalability Taxonomy of Parallel Machines Performance Metrics Outline Recap Scalability Taxonomy
More information