TDP3471 Distributed and Parallel Computing

Transcription

1 TDP3471 Distributed and Parallel Computing Lecture 1

2 Dr. Ian Chai FIT Building: Room BR1024 Office : Schedule for Dr. Ian (including consultation hours) available at Since MMLS isn t ready, lecture notes are at

3 Text Book and References Text Book Barry Wilkinson & Michael Allen, "Parallel Programming", Prentice Hall, These lecture notes have been adapted from the original lecture notes by Dr. K.H.Poo

4 Course Assessment 40% Final Examination 60% Assignments 2 Assignments PLAGIARISM will not be tolerated and will be handled with seriously in this course

5 Course Objectives The objective of the subject is to provide an integrated view of the various aspects of the distributed and parallel computing, which includes security considerations. The hardware architectures, programming languages, models and software development tools. Identify some solving techniques in parallel systems. Able to develop a parallel applications using MPI libraries, Threads and OpenMP.

6 Agenda Introduction Distributed Systems Goals Hardware Software

7 Definition of a Distributed System A distributed system is: A collection of independent computers that appears to its users as a single coherent system. It is where all the components located at networked computers communicate and coordinate their actions only by passing messages.

8 Definition of a Distributed System 1.1 A distributed system organized as middleware. Note that the middleware layer extends over multiple machines.

9 Figure 1.1 A typical portion of the Internet intranet % ISP % % % backbone satellite link desktop computer: server: network link:

10 Figure 1.2 A typical intranet print and other servers server Desktop computers Web server Local area network server File server print other servers the rest of the Internet router/firewall

11 Figure 1.3 Portable and handheld devices in a distributed system Internet Host intranet Wireless LAN WAP Cellular gateway network Home intranet Printer Camera Mobile phone Laptop Host site

12 Figure 1.4 Web servers and web browsers Web servers Browsers Internet File system of Protocols Activity.html

13 Characteristics of DS A distributed system has the following significant characteristics: Concurrency : A DS system able the capacity to handle and coordinate shared concurrent resources No global clock : There is no single notion of the correct time Independent failure : Normally in a DS, faults in one system is isolated from others

14 Transparency Transparency Access Location Migration Relocation Replication Concurrency Failure Persistence Description Hide differences in data representation and how a resource is accessed Hide where a resource is located Hide that a resource may move to another location Hide that a resource may be moved to another location while in use Hide that a resource may be shared by several competitive users Hide that a resource may be shared by several competitive users Hide the failure and recovery of a resource Hide whether a (software) resource is in memory or on disk

15 Challenges of a DS: Challenges Heterogeneity Heterogeneity in networks, computer hardware, OS, programming languages and implementation Openness whether the system can be extended and re-implemented in various ways Security Scalability - a system is described as scalable if it will remain effective where there is a significant increase in number of resources and users

16 3 dimensions Scalability Problems Scalable with respect to size Geographically scalable Administratively scalable Concept Example Centralized services A single server for all users Centralized data A single on-line telephone book Centralized algorithms Doing routing based on complete information

17 Scaling Techniques 3 techniques for scaling Hiding communication latencies Avoid waiting for responses to remote service requests as much as possible Uses asynchronous communication But not all application can effectively use asynchronous communication

18 Scaling Techniques 1.4 The difference between letting: a) a server or b) a client check forms as they are being filled

19 3 techniques for scaling Hiding communication latencies Avoid waiting for responses to remote service requests as much as possible Uses asynchronous communication Distribution Scaling Techniques Taking component, splitting it to smaller parts and spreading it across system eg. DNS

20 Scaling Techniques 1.5 An example of dividing the DNS name space into zones.

21 3 techniques for scaling Hiding communication latencies Avoid waiting for responses to remote service requests as much as possible Uses asynchronous communication Distribution Scaling Techniques Taking component, splitting it to smaller parts and spreading it across system eg. DNS Replicate increase availability and load balancing

22 Pitfalls of a DS Common assumptions of a DS The network is reliable The network is secure The network is homogeneous The topology does not change Latency is zero Bandwidth is infinite Transport cost is zero There is one administrator

23 Types of DS There are various types of DS Cluster computing collection of similar workstations or PCs which are closely connected by high speed LAN and each node runs on the same OS Grid Computing subgroup consists of DS that are often constructed as a federation of computer systems where each system may have different administrative domains and may be different in hardware, software and deployed network technology

24 Cluster Computing Systems Figure 1-6. An example of a cluster computing system.

25 Grid Computing Systems Figure 1-7. A layered architecture for grid computing systems.

26 Architectures

27 Introduction Distributed systems are often complex pieces of software of which components are by definition dispersed across multiple machines It is crucial that these systems are properly organized An architectural model of a DS is concerned with the placement of its parts and the relationships between them. Eg. Client-server and the peer-topeer (P2P) model. There is no global time in a DS, thus clocks on different computers don t necessary give the same time as one another.

28 Introduction All communication between processes is achieved by means of messages. The organization of DS is mostly about the software components that constitute the system software architectures System architectures final instantiation of a software architecture

29 Figure 2.1 Software and hardware service layers in distributed systems Applications, services Middleware Operating system Computer and network hardware Platform

30 Architectural Styles Logical organization of a DS by the division of software components or software architecture A component is a modular unit with well-defined required and provided interfaces that is replaceable within its environment There are 4 styles Layered architectures Object-based architectures Data-centered architectures Event-based architectures

31 Architectural Styles Figure 2-1. The (a) layered architectural style

32 Architectural Styles Figure 2-1. (b) The object-based architectural style.

33 Architectural Styles Figure 2-2. (a) The event-based architectural style Figure 2-2. (b) The shared data-space architectural style.

34 Hardware Concept The architecture of a system is its structure in terms of separately specified components. The overall goal is to ensure that the structure will meet present and likely future demands on it. Major concern reliability, manageability, adaptability and cost effective.

35 Hardware Concepts 1.6 Different basic organizations and memories in distributed computer systems

36 Multiprocessors A bus-based multiprocessor. 1.7

37 Multiprocessors a) A crossbar switch b) An omega switching network 1.8

38 Homogeneous Multicomputer Systems a) Grid b) Hypercube 1-9

39 Software Concepts System Description Main Goal DOS NOS Middleware Tightly-coupled operating system for multiprocessors and homogeneous multicomputers Loosely-coupled operating system for heterogeneous multicomputers (LAN and WAN) Additional layer atop of NOS implementing general-purpose services Hide and manage hardware resources Offer local services to remote clients Provide distribution transparency An overview of DOS (Distributed Operating Systems) NOS (Network Operating Systems) Middleware

40 Uniprocessor OS Separating applications from operating system code through a microkernel.

41 Multiprocessor OS A monitor to protect an integer against concurrent access. monitor Counter { private: int count = 0; public: } int value() { return count;} void incr () { count = count + 1;} void decr() { count = count 1;}

42 Multiprocessor OS A monitor to protect an integer against concurrent access, but blocking a process. monitor Counter { private: int count = 0; int blocked_procs = 0; condition unblocked; public: int value () { return count;} void incr () { if (blocked_procs == 0) count = count + 1; else signal (unblocked); } void decr() { if (count ==0) { blocked_procs = blocked_procs + 1; wait (unblocked); blocked_procs = blocked_procs 1; } else count = count 1; } }

43 Multicomputer OS General structure of a multicomputer operating system

44 Distributed Shared Memory Systems a) Pages of address space distributed among four machines b) Situation after CPU 1 references page 10 c) Situation if page 10 is read only and replication is used

45 Distributed Shared Memory Systems False sharing of a page between two independent processes.

46 Network Operating System General structure of a network operating system.

47 Network Operating System Two clients and a server in a network operating system.

48 Network Operating System Different clients may mount the servers in different places. 1.21

49 Positioning Middleware General structure of a distributed system as middleware. 1-22

50 Middleware and Openness 1.23 In an open middleware-based distributed system, the protocols used by each middleware layer should be the same, as well as the interfaces they offer to applications.

51 Comparison between Systems Item Distributed OS Multiproc. Multicomp. Network OS Middlewarebased OS Degree of transparency Very High High Low High Same OS on all nodes Yes Yes No No Number of copies of OS 1 N N N Basis for communication Resource management Shared memory Global, central Messages Files Model specific Global, distributed Per node Per node Scalability No Moderately Yes Varies Openness Closed Closed Open Open

52 Clients and Servers General interaction between a client and a server.

53 Figure 2.10 Processes and channels process p process q send m receive Outgoing message buffer Communication channel Incoming message buffer

54 Processing Level The general organization of an Internet search engine into three different layers

55 Multitiered Architectures Alternative client-server organizations (a) (e).

56 Multitiered Architectures An example of a server acting as a client. 1-30

57 Vertical Distribution Dividing applications into a user-interface, processing components and a data level Different tiers correspond directly with a logical organization of applications Functions can be logically and physically split across multiple machines, where each machines is tailored to a specific group of functions

58 Horizontal Distribution Client or server may be physically split up into logically equivalent parts Each part is operating on its own share of the complete data set, thus balancing the load Eg. Peer-to-peer systems Each process will act as a client and a server at the same time

59 Modern Architectures An example of horizontal distribution of a Web service.

60 Structured Peer-to-Peer Architectures Overlay network is constructed using a deterministic procedure i.e distributed hash table (DHT) Data items are assigned a random key from large identifier space such as 128-bit or 160-bit identifier Nodes in the system are also assigned a random number from the same identifier space

61 Structured Peer-to-Peer Architectures Figure 2-7. The mapping of data items onto nodes in Chord.

62 Unstructured Peer-to-Peer Architectures Rely on randomized algorithms for constructing an overlay network The main idea is that each node maintains a list of neighbors where the list is constructed in a more or less random way Data items are assumed to be randomly placed on nodes Resembles a random graph

63 Figure 2.5 A distributed application based on peer processes Application Coordination code Application Coordination code Application Coordination code

64 Summary Goals of DS Users and Resources Transparency Openness Scalability Hardware and Software Concepts

65 Next week There will be a quiz based on this week s lecture during the lecture on 27 th October, so don t be late. Lab on 28 th October, 11am, in AR3007.