Parallel Computing and Grids

Transcription

1 Parallel Computing and Grids Implications of Advances in Computing Power and the Internet Revolution Ken Kennedy Rice University

2 Outline Computing Faster and faster? Why? Parallel Computing More, rather than faster, processors Communications National networking National LambdaRail Grids National distributed computing

3 The Computer Processor L1 Cache On Chip L2 Cache System Bus Storage (RAM) IO Interface Network Interface

4 Why We Need Powerful Computers To solve bigger problems faster Weather prediction Aircraft design Environmental cleanup Airline crew scheduling Responsive transaction processing instantaneous response to 500 million users To run more complex software Image processing (Adobe Photoshop) Video editing and delivery (Final Cut Pro, QuickTime TV) Animation and graphics (Pixar and LucasFilms) computer games, flight training systems

5 Computer Performance Today Personal Computer 3.2 gigahertz single-processor P4 Workstations 4-16 processors (51.2 gigaops) Servers: 128 processor Unix or NT System (408 gigaops) Supercomputers 2048 processor supercluster from Cray, IBM, HP, SGI, (6.5 gigaops/gigaflops) One-of-a-kind systems 8192 processor systems for Government (26 teraops/teraflops)

6 Moore s Law Pentium II Pentium Transistors on Chip (thousands)

7 Trouble with Moore s Law Transistor count does not necessarily translate to speed Architectural tricks are needed Parallel instruction issue Actual application speed doubles more slowly Losses due to difficulty of exploiting architecture Physical limits will eventually be reached Speed of light Chip fabrication facilities becoming too expensive Recently: $2 billion Soon: $10 billion We can t wait for the processing power to arrive Supercomputer as time machine

8 Moore s Law and Architecture

9 Going Beyond Moore s Law Solution to Limitations: Architectural Change Parallel Computing Use of more than one processor for the same application Cost: programming complexity Application decomposition Synchronization and communication Distributed Computing Use of more than one computer for each application Assigning different tasks to specialized facilities database access, transaction processing Cost: more programming complexity All complexity of parallelism plus Security, reliability, performance guarantees, network traffic

10 Symmetric Multiprocessor (SMP) Processor 1 Processor 2 Processor 3 Processor 4 Cache Cache Cache Cache System Bus Storage (RAM) Network Interface Key Problem: Parallel programming

11 Example: Parallel Maximum Maximum of a set of numbers on each computer

12 Example: Parallel Maximum

13 Example: Parallel Maximum

14 Example: Parallel Maximum

15 Example: Parallel Maximum What if the first four processors are twice as fast?

16

17 Distributed Computing Collection of Computers On a network Is it a Parallel Computer? Answer: maybe If we have: System software Tools to build apps Data format translation Existence Proofs SETI@Home PCs download a part of the data processing problem Client-server systems Database server connected to transaction management system

18 Grid Application: Are We Alone? Search for Extra-Terrestrial Intelligence (SETI) on the Grid How it works Download data from Arecibo radio telescope to Berkeley Split it into 107-second work units (on Berkeley server) Client program (your screensaver) Asks for & receives a work unit via internet Analyzes it Sends back candidate ET signals to server Server sorts through the results (verifies correctness, etc.) Why it works 299,473 computers (representing 132,181 users) running client can do a lot of work! For comparison, the most powerful single computers have about 5000 to processors each

19 - Message from ET

20 - Message from ET

21 Example: A Reservation System Reservations Flights and Seats Data Base Engine 256-Processor Distributed-Memory System

22 Example: A Reservation System Reservations Call-Handling Computer System Flights and Seats Data Base Engine 16-processor SMP 256-Processor Distributed-Memory System

23 Example: A Reservation System Reservations Travel Agent Worstation PC Call-Handling Computer System Flights and Seats Data Base Engine 16-processor SMP 256-Processor Distributed-Memory System

24 National Distributed Problem Solving Database Supercomputer Database Supercomputer

25 Applications that Work on the Grid Parameter sweep applications Image processing Workflow applications Many different applications that pass data using files Staged calculations Grid services applications Functions invoked on the Grid via client-server mechanism Program with mixture of computation and fixed resources Computation with database access Example: sequencing matching (FASTA and Smith-Waterman)

26 CS Challenges How can we make it easy to write and run Grid applications? Scientists should focus science Write Grid applications in high-level scripting languages How can we make new algorithms for the Grid that achieve effective utilization? Minimize time to solution, maximize throughput How can we make it safe to run Grid applications? Avoiding fraud and misuse of resources How can we locate resources and schedule computations? Who pays? How can we develop economic models to make Grids work?