Grid Computing M2 DL.
|
|
- Juliana Washington
- 6 years ago
- Views:
Transcription
1 Grid Computing M2 DL
2 A Brain is a Lot of Data! (Mark Ellisman, UCSD) And comparisons must be made among many We need to get to one micron to know location of every cell. We re just now starting to get to 10 microns 2
3 Outline A very short introduction to Grids A brief introduction to parallelism A not so short introduction to Grids Gridification of a sequential program Some grid middlewares and grid projects In-depth study of the Globus middleware 3
4 Grid concepts : an analogy Electric power distribution : the electric network and high voltage 4
5 Grid concepts Computer power distribution : Internet network and high performance (parallelism and distribution) 5
6 A Definition of Grid Computing Ian Foster: Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations. Current views: see for instance the survey from Heinz Stockinger in 2006 : rid-1.1.pdf Oldest: Corbato in 1965: See for 6 more or alternate definitions
7 Concrete examples of GridS Europe: EGEE institutions, 45 countries 72,000 CPU, 20 PB disk, 200,000 concurrent jobs High Energy Physics, BioInformatics, Astrophysics Several projects in US Labs Japan s Naregi, GridBus in Australia, France: 7
8 Virtual Organization: CERN s Large Hadron Collider 1800 Physicists, 150 Institutes, 32 Countries 8 30 PB of data yearly; 70,000 CPUs
9 A Market or a niche? Only a scientist thing? Market: 12 Billions $ by 2008 (IDC) Major vendors are on the hype: IBM, Sun, Intel, Microsoft, Oracle, HP, Hitachi, About 30 big companies involved in the Open Grid Forum: + myriad of smaller companies. + services/experts companies (CS-SI, IBM, Atos, Accenture, Cap Gemini, ) IN2P3, CEA, CNES, Metéo France, Airbus, IFP, Banques, Peugeot, FT, Simulations (Physics, Environment, Finance), Genetics, Chemistry, Games, Data centers, 9
10 Where does all this come from??? 10
11 Parallelism : an introduction Grids dates back only 1996 Parallelism is older! (first classification in 1972) Motivations : need more computing power (weather forecast, atomic simulation, genetics ) need more storage capacity (petabytes and more) in a word : improve performance! 3 ways... Work harder --> Use faster hardware Work smarter --> Optimize algorithms Get help --> Use more computers! 11
12 Parallelism : the old classification Flynn (1972) Parallel architectures classified by the number of instructions (single/multiple) performed and the number of data (single/multiple) treated at same time SISD : Single Instruction, Single Data SIMD : Single Instruction, Multiple Data MISD : Multiple Instructions, Single Data MIMD : Multiple Instructions, Multiple Data 12
13 SIMD architectures Vector machines in decline since 97 (disappeared from market place) concept : same instruction performed on several CPU (as much as 16384) on different data data are treated in parallel 13
14 MIMD architectures different instructions are performed in parallel on different data divide and conquer : many subtasks in parallel to shorten global execution time large heterogeneity of systems 14
15 Another taxonomy based on how memories and processors interconnect SMP : Symmetric Multiprocessors MPP : Massively Parallel Processors Constellations Clusters Distributed systems 15
16 Symmetric Multi-Processors (1/3) Small number of identical processors (2-64) Share-everything architecture single memory (shared memory architecture) single I/O single OS equal access to resources HD Memory network CPU CPU CPU 16
17 Symmetric Multi-Processors (2/3) Pro : Easy to program : only one address space to exchange data (but programmer must take care of synchronization in memory access : critical section) 17
18 Symmetric Multi-Processors (3/3) Cons : poor scalability : when the number of processors increases, the cost to transfer data becomes too high; more CPUs = more access memory by the network = more need in memory bandwidth! Direct transfer from proc. to proc. (->MPP) Different interconnection schema (full impossible!, growing in O(n2) when nb of procs increases by O(n)) : bus, crossbar, multistage crossbar,... 18
19 Massively Parallel Processors (1/2) Several hundred nodes with a high speed interconnection network/switch A share-nothing architecture each node owns a memory (distributed memory), one or more processors, each runs an OS copy CPU Memory network Memory CPU Memory CPU 19
20 Massively Parallel Processors (2/2) Pros : good scalability Cons : communication between nodes longer than in shared memory; improve interconnection schema : hypercube, (2D or 3D), torus, fat-tree, multistage crossbars harder to program : data and/or tasks have to be explicitly distributed to nodes remote procedure calls (RPC, JavaRMI) message passing between nodes (PVM, MPI), synchronous or asynchronous communications DSM : Distributed Shared Memory : a virtual memory upgrade : processors and/or communication? 20
21 Constellations a small number of processors (up to 16) clustered in SMP nodes (fast connection) SMPs are connected through a less costly network with poorer performance With DSM, memory may be addressed globally : each CPU has a global memory view, memory and cache coherence is guaranteed (ccnuma) Memory CPU network Memory CPU network Memory CPU network CPUCPU CPU CPUCPU CPU CPUCPU CPU Interconnection network 21 Periph.
22 Clusters a collection of workstations (PC for instance) interconnected through high speed network, acting as a MPP/DSM with network RAM and software RAID (redundant storage, // IO) clusters = specialized version of NOW : Network Of Workstation Pros : low cost standard components take advantage of unused computing power 22
23 Distributed systems interconnection of independent computers each node runs its own OS each node might be any of SMPs, MPPs, constellations, clusters, individual computer the heart of the Grid! «A distributed system is a collection of independent computers that appear to the users of the system as a single computer» Distributed Operating System. A. Tanenbaum, Prentice Hall,
24 Where are we today (2016)? a source for efficient and up-to-date information : the 500 best architectures! we are at 54 petaflops=54000 Teraflops 1 Flops = 1 floating point operation per second 1 TeraFlop = 1000 GigaFlops = MegaFlops = flops = one thousand billion operations per second 24
25 Today's bests comparison on a matrix maths test (Linpack) :Ax=b 25
26 NEC earth simulator Single stage crossbar : 2700 km of cables - a MIMD with Distributed Memory 700 TB disk space 1.6 PB mass storage area : 4 tennis court, 3 floors 26
27 How it grows? in 1993 (14 years ago!) n 1 : 59.7 GFlops n 500 : 0.4 Gflops Sum = 1.17 TFlops in nov 2012 (few days ago) n 1 : TFlops (x100) n 500 : 132 TFlops (x33) in 2007 (yesterday?) n 1 : 280 TFlops (x4666) n 500 : 4 TFlops (x10000) Sum = 4920 Tflops 27
28 MPP= Massive Parallel processing 28
29 29
30 30
31 31
32 Problems of the parallelism Two models of parallelism : driven by data flow : how to distribute data? driven by control flow : how to distribute tasks? Scheduling : which task to execute, on which data, when? how to insure highest compute time (overlap communication/computation?)? Communication using shared memory? using explicit node to node communication? what about the network? Concurrent access to memory (in shared memory systems) to input/output (parallel Input/Output) 32
33 The performance? Ideally grows linearly Speed-up : if TS is the best time to treat a problem in sequential, its time should be TP=TS/P with P processors! Speedup = TS/TP limited (Amdhal law): any program has a sequential and a parallel part : TS=F+T//, thus the speedup is limited : S = (F+T//)/(F+T///P)<1/F Scale-up : if TPS is the time to treat a problem of size S with P processors, then TPS should also be the time to treat a problem of size n*s with n*p processors 33
34 Network performance analysis scalability : can the network be extended? limited wire length, physical problems fault tolerance : if one node is down? for instance in an hypercube multiple access to media? inter-blocking? The metrics : latency : time to connect bandwidth : measured in MB/s 34
35 Tools/environment for parallelism (1/2) Communication between nodes : By global memory! (if possible, plain or virtual) Otherwise : low-level communication : sockets s = socket(af_inet, SOCK_STREAM, 0 ); mid-level communication library (PVM, MPI) info = pvm_initsend( PvmDataDefault ); info = pvm_pkint( array, 10, 1 ); info = pvm_send( tid, 3 ); remote service/object call (RPC, RMI, CORBA) service runs on distant node, only its name and parameters (in, out) have to be known 35
36 Tools/environment for parallelism (2/2) Programming tools threads : small processes data parallel language (for DM archi.): HPF (High Performance Fortran) say how data (arrays) are placed, the system will infer the best placement of computation (to minimize total computation time (e.g. further communications) task parallel language (for SM archi.): OpenMP : compiler directives and library routines; based on threads. The parallel program is close to sequential; it is a step by step transform Parallel loop directives (PARALLEL DO) Task parallel constructs (PARALLEL SECTIONS) PRIVATE and SHARED data declarations 36
37 Bibliography / Webography.C Fox, R.D William and P.C Messina, "Parallel Computing Works!" Morgan Kaufmann publisher, 1994, ISBN M. Cosnard and D Trystram, "Parallel Algorithms and Architectures" Thomson Learning publisher, 1994, ISBN M. Gengler, S. Ubéda and F. Desprez, "Initiation au parallélisme : concepts, architectures et algorithmes" Masson, 1995, ISBN Parallelism: Grids : TOP 500 : OpenMP : PVM: HPF : 37
38 Computational grid HW and SW infrastructure that provides dependable, consistent, pervasive and inexpensive access to high-end computational capabilities Performance criteria : security reliability computing power latency services throughput 38
39 Grid Definition Refined Use Open Protocols Is decentralized Deliver non-trivial QoS 39
40 Levels of cooperation End system (computer, disk, sensor ) Cluster (heterogeneous) synchronous communications, DSM, parallel I/O parallel processing Intranet multithreading, local I/O heterogeneity, distributed admin, distributed FS and databases low supervision, resource discovery high throughput Internet no control, collaborative systems, (international) WAN brokers, negotiation 40
41 Grid Characteristics Large Scale Heterogeneity Multiple Domains of Administration Autonomy but coordination Dynamicity Flexibility Extensibility Security 41
42 Basic services Authentication/Authorization/Traceability Activity control (monitoring) Resource information Resource brokering Scheduling Job submission, data access/migration and execution Accounting 42
43 Layered Grid Architecture (By Analogy to Internet Architecture) Coordinating multiple resources : ubiquitous infrastructure services, app-specific distributed services Sharing single resources : negotiating access, controlling use Talking to things : communication (Internet protocols) & security Controlling things locally : Access to, & control of, resources Collective Application Resource Connectivity Fabric Transport Internet Link From I. Foster 43 Internet Protocol Architecture Application
44 Elements of the Problem Resource sharing Coordinated problem solving Computers, storage, sensors, networks, Heterogeneity of device, mechanism, policy Sharing conditional: negotiation, payment, Integration of distributed resources Compound quality of service requirements Dynamic, multi-institutional virtual orgs Dynamic overlays on classic org structures Map to underlying control mechanisms 44 From I. Foster
45 Aspects of the Problem Need for interoperability when different groups want to share resources Diverse components, policies, mechanisms E.g., standard notions of identity, means of communication, resource descriptions Need for shared infrastructure services to avoid repeated development, installation E.g., one port/service/protocol for remote access to computing, not one per tool/application E.g., Certificate Authorities: expensive to run A common need for protocols & services 45 From I. Foster
46 Resources Description Advertising Cataloging Matching Claiming Reserving Checkpointing 46
47 Resource layers Application layer Application resource management layer resource matching, global brokering Owner layer intertask resource management, execution environment System layer tasks, resource requests owner policy : who may use what End-resource layer end-resource policy (e.g. O.S.) 47
48 Resource management (1) Services and protocols depend on the infrastructure Some parameters stability of the infrastructure (same set of resources or not) freshness of the resource availability information reservation facilities multiple resource or single resource brokering Example request : I need from 10 to 100 CE each with at least 128 MB RAM and a computing power of 50 Mips 48
49 Resource management and scheduling (1) Levels of scheduling Mapping/scheduling job scheduling (global level ; perf : throughput) resource scheduling (perf : fairness, utilization) application scheduling (perf : response time, speedup, produced data ) resource discovery and selection assignment of tasks to computing resources data distribution task scheduling on the computing resources (communication scheduling) Individual perfs are not necessarily consistent with the global (system) perf! 49
50 Resource management and scheduling (2) Grid problems predictions are not definitive : dynamicity! Heterogeneous platforms Checkpointing and migration 50
51 A Resource Management System example (Globus) RSL specialization Broker RSL Queries & Info Application Ground RSL Information Service Co-allocator Simple ground RSL Local resource managers GRAM GRAM GRAM LSF Condor NQE 51
52 Resource information (1) What is to be stored? Organization, people, computing resources, software packages, communication resources, event producers, devices what about data??? A key issue in such dynamics environments A first approach : (distributed) directory (LDAP) easy to use tree structure distribution static mostly read ; not efficient updating hierarchical poor procedural language 52
53 Resource information (2) But : dynamicity complex relationships frequent updates complex queries A second approach : (relational) database 53
54 Programming the grid: potential programming models Message passing (PVM, MPI) Distributed Shared Memory Data Parallelism (HPF, HPC++) Task Parallelism (Condor) Client/server - RPC Agents Integration system (Corba, DCOM, RMI) 54
55 Program execution : issues Parallelize the program with the right job structure, communication patterns/procedures, algorithms Discover the available resources Select the suitable resources Allocate or reserve these resources Migrate the data (or the code) Initiate computations Monitor the executions ; checkpoints? React to changes Collect results 55
56 Data management It was long forgotten!!! Though it is a key issue! Issues : indexing retrieval replication caching traceability (auditing) And security!!! 56
57 From computing grids to information grids 57
58 From computing grids to information grids (1) Grids for long was lacking most of the tools mandatory to share (index, search, access), analyze, secure, monitor semantic data (information) Several reasons : history money difficulty Why is it so difficult? Sensitivity but openness Multiple administrative domains, multiple actors, heterogeneousness but a single global architecture/view/system Dynamicity and unpredictability but robustness Wideness but high performance 58
59 From computing grids to information grids (2) ex : the Replica Management Problem Maintain a mapping between logical names for files and collections and one or more physical locations Decide where and when a piece of data must be replicated Important for many applications Example: CERN high-level trigger data Multiple petabytes of data per year Copy of everything at CERN (Tier 0) Subsets at national centers (Tier 1) Smaller regional centers (Tier 2) Individual researchers have copies of pieces of data Much more complex with sensitive and complex data like medical data!!! 59
60 From computing grids to information grids (3) some (still ) open issues Security, security, security (incl. privacy, monitoring, traceability )) at a semantic level Access protocols (incl. replication, caching, migration ) Indexing tools Brokering of data (incl. accounting) (Content-based) Query optimization and execution Mediation of data Data integration, data warehousing and analysis tools Knowledge discovery and data mining 60
61 Functional View of Grid Data Management Application Metadata Service Planner: Data location, Replica selection, Selection of compute and storage nodes Replica Location Service Information Services Location based on data attributes Location of one or more physical replicas State of grid resources, performance measurements and predictions Security and Policy Executor: Initiates data transfers and computations Data Movement Data Access Compute Resources Storage Resources 61
62 Security: Why Grid Security is Hard Resources being used may be extremely valuable & the problems being solved extremely sensitive Resources are often located in distinct administrative domains Users may be different The set of resources used by a single computation may be large, dynamic, and/or unpredictable Each resource may have own policies & procedures Not just client/server The security service must be broadly available & applicable Standard, well-tested, well-understood protocols Integration with wide variety of tools 62
63 Grid Security : various views User View Resource Owner View 1) Easy to use 1) Specify local access control 2) Single sign-on 2) Auditing, accounting, etc. 3) Run applications ftp,ssh,mpi,condor,web, 3) Integration w/ local system Kerberos, AFS, license mgr. 4) User based trust model 4) Protection from 5) Proxies, delegation compromised resources Developer View API/SDK with authentication, flexible message protection, flexible communication, delegation,... Direct calls to various security functions (e.g. GSS-API) Or security integrated into higher-level SDKs: E.g. GlobusIO, Condor 63
64 Grid security : requirements Authentication Authorization and delegation of authority Assurance Accounting Auditing and monitoring Traceability Integrity and confidentiality 64
65 Query optimization and execution Old wine in new bottles? Yes and no : it seems the problem has not changed but the operational context has so changed that classical heuristics and methods are not more pertinent Key issues : dynamicity, unpredictability, adaptability Very few works have specifically addressed this problem Use mobile agents? 65
66 Service Oriented Architecture Open Grid Service Architecture WSRF : Web Service Resource Framework Everything is resources, WS-Resources Access through services 66
67 Globus Tutorial 67
68 Bibliography The Grid 2: Blueprint for a New Computing Architecture. Ian Foster, Carl Kesselman Grid Computing: The Savvy Manager s Guide. Pawel Plaszczak, Richard Wellner Jr 68
Chapter 4:- Introduction to Grid and its Evolution. Prepared By:- NITIN PANDYA Assistant Professor SVBIT.
Chapter 4:- Introduction to Grid and its Evolution Prepared By:- Assistant Professor SVBIT. Overview Background: What is the Grid? Related technologies Grid applications Communities Grid Tools Case Studies
More informationLecture 9: MIMD Architectures
Lecture 9: MIMD Architectures Introduction and classification Symmetric multiprocessors NUMA architecture Clusters Zebo Peng, IDA, LiTH 1 Introduction MIMD: a set of general purpose processors is connected
More informationLecture 9: MIMD Architectures
Lecture 9: MIMD Architectures Introduction and classification Symmetric multiprocessors NUMA architecture Clusters Zebo Peng, IDA, LiTH 1 Introduction A set of general purpose processors is connected together.
More informationHigh Performance Computing Course Notes Grid Computing I
High Performance Computing Course Notes 2008-2009 2009 Grid Computing I Resource Demands Even as computer power, data storage, and communication continue to improve exponentially, resource capacities are
More informationComputing architectures Part 2 TMA4280 Introduction to Supercomputing
Computing architectures Part 2 TMA4280 Introduction to Supercomputing NTNU, IMF January 16. 2017 1 Supercomputing What is the motivation for Supercomputing? Solve complex problems fast and accurately:
More informationBİL 542 Parallel Computing
BİL 542 Parallel Computing 1 Chapter 1 Parallel Programming 2 Why Use Parallel Computing? Main Reasons: Save time and/or money: In theory, throwing more resources at a task will shorten its time to completion,
More informationLecture 7: Parallel Processing
Lecture 7: Parallel Processing Introduction and motivation Architecture classification Performance evaluation Interconnection network Zebo Peng, IDA, LiTH 1 Performance Improvement Reduction of instruction
More informationCOSC 6385 Computer Architecture - Multi Processor Systems
COSC 6385 Computer Architecture - Multi Processor Systems Fall 2006 Classification of Parallel Architectures Flynn s Taxonomy SISD: Single instruction single data Classical von Neumann architecture SIMD:
More informationWHY PARALLEL PROCESSING? (CE-401)
PARALLEL PROCESSING (CE-401) COURSE INFORMATION 2 + 1 credits (60 marks theory, 40 marks lab) Labs introduced for second time in PP history of SSUET Theory marks breakup: Midterm Exam: 15 marks Assignment:
More information30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy
Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Why the Grid? Science is becoming increasingly digital and needs to deal with increasing amounts of
More informationOutline. Definition of a Distributed System Goals of a Distributed System Types of Distributed Systems
Distributed Systems Outline Definition of a Distributed System Goals of a Distributed System Types of Distributed Systems What Is A Distributed System? A collection of independent computers that appears
More informationChap. 4 Multiprocessors and Thread-Level Parallelism
Chap. 4 Multiprocessors and Thread-Level Parallelism Uniprocessor performance Performance (vs. VAX-11/780) 10000 1000 100 10 From Hennessy and Patterson, Computer Architecture: A Quantitative Approach,
More informationDay 1 : August (Thursday) An overview of Globus Toolkit 2.4
An Overview of Grid Computing Workshop Day 1 : August 05 2004 (Thursday) An overview of Globus Toolkit 2.4 By CDAC Experts Contact :vcvrao@cdacindia.com; betatest@cdacindia.com URL : http://www.cs.umn.edu/~vcvrao
More informationComputer Architecture
Computer Architecture Chapter 7 Parallel Processing 1 Parallelism Instruction-level parallelism (Ch.6) pipeline superscalar latency issues hazards Processor-level parallelism (Ch.7) array/vector of processors
More informationComputer and Information Sciences College / Computer Science Department CS 207 D. Computer Architecture. Lecture 9: Multiprocessors
Computer and Information Sciences College / Computer Science Department CS 207 D Computer Architecture Lecture 9: Multiprocessors Challenges of Parallel Processing First challenge is % of program inherently
More informationMultiprocessors - Flynn s Taxonomy (1966)
Multiprocessors - Flynn s Taxonomy (1966) Single Instruction stream, Single Data stream (SISD) Conventional uniprocessor Although ILP is exploited Single Program Counter -> Single Instruction stream The
More informationLecture 9: MIMD Architecture
Lecture 9: MIMD Architecture Introduction and classification Symmetric multiprocessors NUMA architecture Cluster machines Zebo Peng, IDA, LiTH 1 Introduction MIMD: a set of general purpose processors is
More informationClient Server & Distributed System. A Basic Introduction
Client Server & Distributed System A Basic Introduction 1 Client Server Architecture A network architecture in which each computer or process on the network is either a client or a server. Source: http://webopedia.lycos.com
More informationAdvanced School in High Performance and GRID Computing November Introduction to Grid computing.
1967-14 Advanced School in High Performance and GRID Computing 3-14 November 2008 Introduction to Grid computing. TAFFONI Giuliano Osservatorio Astronomico di Trieste/INAF Via G.B. Tiepolo 11 34131 Trieste
More informationMultiprocessors & Thread Level Parallelism
Multiprocessors & Thread Level Parallelism COE 403 Computer Architecture Prof. Muhamed Mudawar Computer Engineering Department King Fahd University of Petroleum and Minerals Presentation Outline Introduction
More informationGrid Architectural Models
Grid Architectural Models Computational Grids - A computational Grid aggregates the processing power from a distributed collection of systems - This type of Grid is primarily composed of low powered computers
More informationComputer Systems Architecture
Computer Systems Architecture Lecture 24 Mahadevan Gomathisankaran April 29, 2010 04/29/2010 Lecture 24 CSCE 4610/5610 1 Reminder ABET Feedback: http://www.cse.unt.edu/exitsurvey.cgi?csce+4610+001 Student
More informationComputer Systems Architecture
Computer Systems Architecture Lecture 23 Mahadevan Gomathisankaran April 27, 2010 04/27/2010 Lecture 23 CSCE 4610/5610 1 Reminder ABET Feedback: http://www.cse.unt.edu/exitsurvey.cgi?csce+4610+001 Student
More informationIntroduction to Grid Computing
Milestone 2 Include the names of the papers You only have a page be selective about what you include Be specific; summarize the authors contributions, not just what the paper is about. You might be able
More informationUnit 9 : Fundamentals of Parallel Processing
Unit 9 : Fundamentals of Parallel Processing Lesson 1 : Types of Parallel Processing 1.1. Learning Objectives On completion of this lesson you will be able to : classify different types of parallel processing
More informationGrid Computing Fall 2005 Lecture 5: Grid Architecture and Globus. Gabrielle Allen
Grid Computing 7700 Fall 2005 Lecture 5: Grid Architecture and Globus Gabrielle Allen allen@bit.csc.lsu.edu http://www.cct.lsu.edu/~gallen Concrete Example I have a source file Main.F on machine A, an
More informationGrid Computing. MCSN - N. Tonellotto - Distributed Enabling Platforms
Grid Computing 1 Resource sharing Elements of Grid Computing - Computers, data, storage, sensors, networks, - Sharing always conditional: issues of trust, policy, negotiation, payment, Coordinated problem
More informationComputer parallelism Flynn s categories
04 Multi-processors 04.01-04.02 Taxonomy and communication Parallelism Taxonomy Communication alessandro bogliolo isti information science and technology institute 1/9 Computer parallelism Flynn s categories
More informationParallel Computers. c R. Leduc
Parallel Computers Material based on B. Wilkinson et al., PARALLEL PROGRAMMING. Techniques and Applications Using Networked Workstations and Parallel Computers c 2002-2004 R. Leduc Why Parallel Computing?
More informationLecture 7: Parallel Processing
Lecture 7: Parallel Processing Introduction and motivation Architecture classification Performance evaluation Interconnection network Zebo Peng, IDA, LiTH 1 Performance Improvement Reduction of instruction
More informationMultiprocessors and Thread Level Parallelism Chapter 4, Appendix H CS448. The Greed for Speed
Multiprocessors and Thread Level Parallelism Chapter 4, Appendix H CS448 1 The Greed for Speed Two general approaches to making computers faster Faster uniprocessor All the techniques we ve been looking
More informationIntroduction to Parallel Programming
Introduction to Parallel Programming January 14, 2015 www.cac.cornell.edu What is Parallel Programming? Theoretically a very simple concept Use more than one processor to complete a task Operationally
More informationDheeraj Bhardwaj May 12, 2003
HPC Systems and Models Dheeraj Bhardwaj Department of Computer Science & Engineering Indian Institute of Technology, Delhi 110 016 India http://www.cse.iitd.ac.in/~dheerajb 1 Sequential Computers Traditional
More informationChapter 18 Parallel Processing
Chapter 18 Parallel Processing Multiple Processor Organization Single instruction, single data stream - SISD Single instruction, multiple data stream - SIMD Multiple instruction, single data stream - MISD
More informationGrid Computing: Status and Perspectives. Alexander Reinefeld Florian Schintke. Outline MOTIVATION TWO TYPICAL APPLICATION DOMAINS
Grid Computing: Status and Perspectives Alexander Reinefeld Florian Schintke Schwerpunkte der Informatik" Ringvorlesung am 05.06.2003 1 Outline MOTIVATION o What s a Grid? Why using Grids? TWO TYPICAL
More informationParallel computer architecture classification
Parallel computer architecture classification Hardware Parallelism Computing: execute instructions that operate on data. Computer Instructions Data Flynn s taxonomy (Michael Flynn, 1967) classifies computer
More informationParallel & Cluster Computing. cs 6260 professor: elise de doncker by: lina hussein
Parallel & Cluster Computing cs 6260 professor: elise de doncker by: lina hussein 1 Topics Covered : Introduction What is cluster computing? Classification of Cluster Computing Technologies: Beowulf cluster
More informationIntroduction to Distributed Systems
Introduction to Distributed Systems Other matters: review of the Bakery Algorithm: why can t we simply keep track of the last ticket taken and the next ticvket to be called? Ref: [Coulouris&al Ch 1, 2]
More informationGrid Scheduling Architectures with Globus
Grid Scheduling Architectures with Workshop on Scheduling WS 07 Cetraro, Italy July 28, 2007 Ignacio Martin Llorente Distributed Systems Architecture Group Universidad Complutense de Madrid 1/38 Contents
More informationNon-Uniform Memory Access (NUMA) Architecture and Multicomputers
Non-Uniform Memory Access (NUMA) Architecture and Multicomputers Parallel and Distributed Computing Department of Computer Science and Engineering (DEI) Instituto Superior Técnico February 29, 2016 CPD
More informationNon-uniform memory access machine or (NUMA) is a system where the memory access time to any region of memory is not the same for all processors.
CS 320 Ch. 17 Parallel Processing Multiple Processor Organization The author makes the statement: "Processors execute programs by executing machine instructions in a sequence one at a time." He also says
More informationParallel Architectures
Parallel Architectures Part 1: The rise of parallel machines Intel Core i7 4 CPU cores 2 hardware thread per core (8 cores ) Lab Cluster Intel Xeon 4/10/16/18 CPU cores 2 hardware thread per core (8/20/32/36
More informationTop500 Supercomputer list
Top500 Supercomputer list Tends to represent parallel computers, so distributed systems such as SETI@Home are neglected. Does not consider storage or I/O issues Both custom designed machines and commodity
More informationGrid Computing Systems: A Survey and Taxonomy
Grid Computing Systems: A Survey and Taxonomy Material for this lecture from: A Survey and Taxonomy of Resource Management Systems for Grid Computing Systems, K. Krauter, R. Buyya, M. Maheswaran, CS Technical
More informationCOSC 6385 Computer Architecture - Thread Level Parallelism (I)
COSC 6385 Computer Architecture - Thread Level Parallelism (I) Edgar Gabriel Spring 2014 Long-term trend on the number of transistor per integrated circuit Number of transistors double every ~18 month
More informationCrossbar switch. Chapter 2: Concepts and Architectures. Traditional Computer Architecture. Computer System Architectures. Flynn Architectures (2)
Chapter 2: Concepts and Architectures Computer System Architectures Disk(s) CPU I/O Memory Traditional Computer Architecture Flynn, 1966+1972 classification of computer systems in terms of instruction
More informationNon-Uniform Memory Access (NUMA) Architecture and Multicomputers
Non-Uniform Memory Access (NUMA) Architecture and Multicomputers Parallel and Distributed Computing Department of Computer Science and Engineering (DEI) Instituto Superior Técnico September 26, 2011 CPD
More informationChapter 20: Database System Architectures
Chapter 20: Database System Architectures Chapter 20: Database System Architectures Centralized and Client-Server Systems Server System Architectures Parallel Systems Distributed Systems Network Types
More informationCloud Computing. Summary
Cloud Computing Lectures 2 and 3 Definition of Cloud Computing, Grid Architectures 2012-2013 Summary Definition of Cloud Computing (more complete). Grid Computing: Conceptual Architecture. Condor. 1 Cloud
More informationHigh Performance Computing
The Need for Parallelism High Performance Computing David McCaughan, HPC Analyst SHARCNET, University of Guelph dbm@sharcnet.ca Scientific investigation traditionally takes two forms theoretical empirical
More informationWhat are Clusters? Why Clusters? - a Short History
What are Clusters? Our definition : A parallel machine built of commodity components and running commodity software Cluster consists of nodes with one or more processors (CPUs), memory that is shared by
More informationGrid Computing. Lectured by: Dr. Pham Tran Vu Faculty of Computer and Engineering HCMC University of Technology
Grid Computing Lectured by: Dr. Pham Tran Vu Email: ptvu@cse.hcmut.edu.vn 1 Grid Architecture 2 Outline Layer Architecture Open Grid Service Architecture 3 Grid Characteristics Large-scale Need for dynamic
More informationIntroduction to Cluster Computing
Introduction to Cluster Computing Prabhaker Mateti Wright State University Dayton, Ohio, USA Overview High performance computing High throughput computing NOW, HPC, and HTC Parallel algorithms Software
More informationParallel Programming. Michael Gerndt Technische Universität München
Parallel Programming Michael Gerndt Technische Universität München gerndt@in.tum.de Contents 1. Introduction 2. Parallel architectures 3. Parallel applications 4. Parallelization approach 5. OpenMP 6.
More informationCS4230 Parallel Programming. Lecture 3: Introduction to Parallel Architectures 8/28/12. Homework 1: Parallel Programming Basics
CS4230 Parallel Programming Lecture 3: Introduction to Parallel Architectures Mary Hall August 28, 2012 Homework 1: Parallel Programming Basics Due before class, Thursday, August 30 Turn in electronically
More informationAn Introduction to the Grid
1 An Introduction to the Grid 1.1 INTRODUCTION The Grid concepts and technologies are all very new, first expressed by Foster and Kesselman in 1998 [1]. Before this, efforts to orchestrate wide-area distributed
More informationOverview. CS 472 Concurrent & Parallel Programming University of Evansville
Overview CS 472 Concurrent & Parallel Programming University of Evansville Selection of slides from CIS 410/510 Introduction to Parallel Computing Department of Computer and Information Science, University
More informationIntroduction. Distributed Systems IT332
Introduction Distributed Systems IT332 2 Outline Definition of A Distributed System Goals of Distributed Systems Types of Distributed Systems 3 Definition of A Distributed System A distributed systems
More informationIntroduction to Multiprocessors (Part I) Prof. Cristina Silvano Politecnico di Milano
Introduction to Multiprocessors (Part I) Prof. Cristina Silvano Politecnico di Milano Outline Key issues to design multiprocessors Interconnection network Centralized shared-memory architectures Distributed
More informationParallel Computing Platforms
Parallel Computing Platforms Jinkyu Jeong (jinkyu@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu SSE3054: Multicore Systems, Spring 2017, Jinkyu Jeong (jinkyu@skku.edu)
More informationChapter 9 Multiprocessors
ECE200 Computer Organization Chapter 9 Multiprocessors David H. lbonesi and the University of Rochester Henk Corporaal, TU Eindhoven, Netherlands Jari Nurmi, Tampere University of Technology, Finland University
More informationBlueGene/L (No. 4 in the Latest Top500 List)
BlueGene/L (No. 4 in the Latest Top500 List) first supercomputer in the Blue Gene project architecture. Individual PowerPC 440 processors at 700Mhz Two processors reside in a single chip. Two chips reside
More informationParallel Computer Architectures. Lectured by: Phạm Trần Vũ Prepared by: Thoại Nam
Parallel Computer Architectures Lectured by: Phạm Trần Vũ Prepared by: Thoại Nam Outline Flynn s Taxonomy Classification of Parallel Computers Based on Architectures Flynn s Taxonomy Based on notions of
More informationParallel Processors. The dream of computer architects since 1950s: replicate processors to add performance vs. design a faster processor
Multiprocessing Parallel Computers Definition: A parallel computer is a collection of processing elements that cooperate and communicate to solve large problems fast. Almasi and Gottlieb, Highly Parallel
More informationSystem types. Distributed systems
System types 1 Personal systems that are designed to run on a personal computer or workstation Distributed systems where the system software runs on a loosely integrated group of cooperating processors
More informationChapter 18: Database System Architectures.! Centralized Systems! Client--Server Systems! Parallel Systems! Distributed Systems!
Chapter 18: Database System Architectures! Centralized Systems! Client--Server Systems! Parallel Systems! Distributed Systems! Network Types 18.1 Centralized Systems! Run on a single computer system and
More informationParallel Computers. CPE 631 Session 20: Multiprocessors. Flynn s Tahonomy (1972) Why Multiprocessors?
Parallel Computers CPE 63 Session 20: Multiprocessors Department of Electrical and Computer Engineering University of Alabama in Huntsville Definition: A parallel computer is a collection of processing
More informationParallel Computing Platforms. Jinkyu Jeong Computer Systems Laboratory Sungkyunkwan University
Parallel Computing Platforms Jinkyu Jeong (jinkyu@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Elements of a Parallel Computer Hardware Multiple processors Multiple
More informationDistributed systems. Distributed Systems Architectures. System types. Objectives. Distributed system characteristics.
Distributed systems Distributed Systems Architectures Virtually all large computer-based systems are now distributed systems. Information processing is distributed over several computers rather than confined
More informationPractical Scientific Computing
Practical Scientific Computing Performance-optimised Programming Preliminary discussion, 17.7.2007 Dr. Ralf-Peter Mundani, mundani@tum.de Dipl.-Ing. Ioan Lucian Muntean, muntean@in.tum.de Dipl.-Geophys.
More information06-Dec-17. Credits:4. Notes by Pritee Parwekar,ANITS 06-Dec-17 1
Credits:4 1 Understand the Distributed Systems and the challenges involved in Design of the Distributed Systems. Understand how communication is created and synchronized in Distributed systems Design and
More informationDISTRIBUTED SYSTEMS. Second Edition. Andrew S. Tanenbaum Maarten Van Steen. Vrije Universiteit Amsterdam, 7'he Netherlands PEARSON.
DISTRIBUTED SYSTEMS 121r itac itple TAYAdiets Second Edition Andrew S. Tanenbaum Maarten Van Steen Vrije Universiteit Amsterdam, 7'he Netherlands PEARSON Prentice Hall Upper Saddle River, NJ 07458 CONTENTS
More information3/24/2014 BIT 325 PARALLEL PROCESSING ASSESSMENT. Lecture Notes:
BIT 325 PARALLEL PROCESSING ASSESSMENT CA 40% TESTS 30% PRESENTATIONS 10% EXAM 60% CLASS TIME TABLE SYLLUBUS & RECOMMENDED BOOKS Parallel processing Overview Clarification of parallel machines Some General
More informationDistributed Systems. Overview. Distributed Systems September A distributed system is a piece of software that ensures that:
Distributed Systems Overview Distributed Systems September 2002 1 Distributed System: Definition A distributed system is a piece of software that ensures that: A collection of independent computers that
More informationNon-Uniform Memory Access (NUMA) Architecture and Multicomputers
Non-Uniform Memory Access (NUMA) Architecture and Multicomputers Parallel and Distributed Computing MSc in Information Systems and Computer Engineering DEA in Computational Engineering Department of Computer
More informationIntroduction to Parallel Computing
Portland State University ECE 588/688 Introduction to Parallel Computing Reference: Lawrence Livermore National Lab Tutorial https://computing.llnl.gov/tutorials/parallel_comp/ Copyright by Alaa Alameldeen
More informationCOSC 6374 Parallel Computation. Parallel Computer Architectures
OS 6374 Parallel omputation Parallel omputer Architectures Some slides on network topologies based on a similar presentation by Michael Resch, University of Stuttgart Spring 2010 Flynn s Taxonomy SISD:
More informationIntroduction to Grid Technology
Introduction to Grid Technology B.Ramamurthy 1 Arthur C Clarke s Laws (two of many) Any sufficiently advanced technology is indistinguishable from magic." "The only way of discovering the limits of the
More informationGrid Challenges and Experience
Grid Challenges and Experience Heinz Stockinger Outreach & Education Manager EU DataGrid project CERN (European Organization for Nuclear Research) Grid Technology Workshop, Islamabad, Pakistan, 20 October
More informationCMSC 611: Advanced. Parallel Systems
CMSC 611: Advanced Computer Architecture Parallel Systems Parallel Computers Definition: A parallel computer is a collection of processing elements that cooperate and communicate to solve large problems
More informationPractical Scientific Computing
Practical Scientific Computing Performance-optimized Programming Preliminary discussion: July 11, 2008 Dr. Ralf-Peter Mundani, mundani@tum.de Dipl.-Ing. Ioan Lucian Muntean, muntean@in.tum.de MSc. Csaba
More informationConvergence of Parallel Architecture
Parallel Computing Convergence of Parallel Architecture Hwansoo Han History Parallel architectures tied closely to programming models Divergent architectures, with no predictable pattern of growth Uncertainty
More informationEITF20: Computer Architecture Part 5.2.1: IO and MultiProcessor
EITF20: Computer Architecture Part 5.2.1: IO and MultiProcessor Liang Liu liang.liu@eit.lth.se 1 Outline Reiteration I/O MultiProcessor Summary 2 Virtual memory benifits Using physical memory efficiently
More informationSerial. Parallel. CIT 668: System Architecture 2/14/2011. Topics. Serial and Parallel Computation. Parallel Computing
CIT 668: System Architecture Parallel Computing Topics 1. What is Parallel Computing? 2. Why use Parallel Computing? 3. Types of Parallelism 4. Amdahl s Law 5. Flynn s Taxonomy of Parallel Computers 6.
More informationMIMD Overview. Intel Paragon XP/S Overview. XP/S Usage. XP/S Nodes and Interconnection. ! Distributed-memory MIMD multicomputer
MIMD Overview Intel Paragon XP/S Overview! MIMDs in the 1980s and 1990s! Distributed-memory multicomputers! Intel Paragon XP/S! Thinking Machines CM-5! IBM SP2! Distributed-memory multicomputers with hardware
More informationDISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S. TANENBAUM MAARTEN VAN STEEN. Chapter 1. Introduction
DISTRIBUTED SYSTEMS Principles and Paradigms Second Edition ANDREW S. TANENBAUM MAARTEN VAN STEEN Chapter 1 Introduction Modified by: Dr. Ramzi Saifan Definition of a Distributed System (1) A distributed
More informationIntro to Multiprocessors
The Big Picture: Where are We Now? Intro to Multiprocessors Output Output Datapath Input Input Datapath [dapted from Computer Organization and Design, Patterson & Hennessy, 2005] Multiprocessor multiple
More informationProcessor Architecture and Interconnect
Processor Architecture and Interconnect What is Parallelism? Parallel processing is a term used to denote simultaneous computation in CPU for the purpose of measuring its computation speeds. Parallel Processing
More informationLect. 2: Types of Parallelism
Lect. 2: Types of Parallelism Parallelism in Hardware (Uniprocessor) Parallelism in a Uniprocessor Pipelining Superscalar, VLIW etc. SIMD instructions, Vector processors, GPUs Multiprocessor Symmetric
More informationINSTITUTO SUPERIOR TÉCNICO. Architectures for Embedded Computing
UNIVERSIDADE TÉCNICA DE LISBOA INSTITUTO SUPERIOR TÉCNICO Departamento de Engenharia Informática Architectures for Embedded Computing MEIC-A, MEIC-T, MERC Lecture Slides Version 3.0 - English Lecture 11
More informationCommunication has significant impact on application performance. Interconnection networks therefore have a vital role in cluster systems.
Cluster Networks Introduction Communication has significant impact on application performance. Interconnection networks therefore have a vital role in cluster systems. As usual, the driver is performance
More informationCS 590: High Performance Computing. Parallel Computer Architectures. Lab 1 Starts Today. Already posted on Canvas (under Assignment) Let s look at it
Lab 1 Starts Today Already posted on Canvas (under Assignment) Let s look at it CS 590: High Performance Computing Parallel Computer Architectures Fengguang Song Department of Computer Science IUPUI 1
More informationCOSC 6374 Parallel Computation. Parallel Computer Architectures
OS 6374 Parallel omputation Parallel omputer Architectures Some slides on network topologies based on a similar presentation by Michael Resch, University of Stuttgart Edgar Gabriel Fall 2015 Flynn s Taxonomy
More informationCA464 Distributed Programming
1 / 25 CA464 Distributed Programming Lecturer: Martin Crane Office: L2.51 Phone: 8974 Email: martin.crane@computing.dcu.ie WWW: http://www.computing.dcu.ie/ mcrane Course Page: "/CA464NewUpdate Textbook
More informationHandout 3 Multiprocessor and thread level parallelism
Handout 3 Multiprocessor and thread level parallelism Outline Review MP Motivation SISD v SIMD (SIMT) v MIMD Centralized vs Distributed Memory MESI and Directory Cache Coherency Synchronization and Relaxed
More informationLecture 23 Database System Architectures
CMSC 461, Database Management Systems Spring 2018 Lecture 23 Database System Architectures These slides are based on Database System Concepts 6 th edition book (whereas some quotes and figures are used
More informationIntroduction to GT3. Introduction to GT3. What is a Grid? A Story of Evolution. The Globus Project
Introduction to GT3 The Globus Project Argonne National Laboratory USC Information Sciences Institute Copyright (C) 2003 University of Chicago and The University of Southern California. All Rights Reserved.
More informationDistributed Systems LEEC (2006/07 2º Sem.)
Distributed Systems LEEC (2006/07 2º Sem.) Introduction João Paulo Carvalho Universidade Técnica de Lisboa / Instituto Superior Técnico Outline Definition of a Distributed System Goals Connecting Users
More informationDistributed Systems Architectures. Ian Sommerville 2006 Software Engineering, 8th edition. Chapter 12 Slide 1
Distributed Systems Architectures Ian Sommerville 2006 Software Engineering, 8th edition. Chapter 12 Slide 1 Objectives To explain the advantages and disadvantages of different distributed systems architectures
More informationIssues in Multiprocessors
Issues in Multiprocessors Which programming model for interprocessor communication shared memory regular loads & stores SPARCCenter, SGI Challenge, Cray T3D, Convex Exemplar, KSR-1&2, today s CMPs message
More information