Grid Computing: SGE,Jxta and Jini

Transcription

1 Grid Computing: SGE,Jxta and Jini Dr Simon See Director High Performance Computing Technology Adjunct Associate Professor, Nanyang Technological University 1

2 2

3 3

4 4 Department Grid Campus Grid Research Grid Enterprise Grid Access Grid Public Grid Broker Grid Clusterload balancing Heterogeneity, HPC, priorities,... Internet, multi-tier,collaboration,... Securitiy,policies, RAS,... Imersive collaboration, training,... Resource discovery,reservation,... Level of service,economy, accounting Other Grids: Application /Storage /Network Service Providers, Peer-to- Peer, Collaboration Grid, Data Grid,HealthGrid, BioGrid, Entertainment Grid, Information Grid,...

5 5 Extends clustercomputing toward LANs and WANs for paralleland distributed computing. Problems to be solved: bandwidth and latency reliability,availability,faulttolerance wide area parallelprocessing interoperability heterogeneity asingle global name space protection,security,authentication efficient scheduling and comprehensiveresources management

6 6

7 7 Load balancing maximizes resource utilization Transparentjob submission & machineselection Monitoring and accounting ==> SGE Sun Grid Engine,free,open source Guaranteeing requiredresources Fullcontrol overresourceutilization Fairand share based resource usage Implementation of management policies ==> SGE/GRD GlobalResource Director

8 8

9 9

10 ! "! #" 10

11 11 Sun Microsystems Swiss Centerfor Scientific Computing (CSCS) Research! Not a current or future Sun product Technology Exploration: Java TM in HPC Domain:Distributed Resource Management Technologies employed Java TM,Jini TM, JavaSpaces TM Sun TM Grid Engine

12 12 Resources Applications Existing, un modified HPC codes Computecycles Localresource manager queues Geographically Distributed Intranet domain Integrated Application selection & execution framework

13 13

14 The Expanded Web Challenges Current AccessModels 14

15 15!" Search the entire Web --and allitsconnected devices --forinformation orresources (deeper Web) Share computeservices,files, music,etc.with anyone,regardless wherethey relocated or what type of device they re using (pervasive networking) Storefilesand data anywhere on the network, notjusttolocal harddrives (distributed networking)

16 16 # Jun 1999: BillJoy s Mantra Pervasive,Simple & Secure Jan 2000: First Design (Protocols, XML, Messaging,Space) Jun 2000: First Demo Prototype (Palm) Nov 2001: New Design (Protocols, X M L, Peers,Pipes,PeerGroups) Com mitters (5 Sun) Apr 2001:J2SE Reference Implementation on Jun 2001: 50,000 Downloads! StartPlatform Community Projects Com mitters (14 Sun, 8 Non-Sun) New V1.0 Protocols Spec (Revision 1.1)

17 17 $! % Java-specific A networking API A new way to browse the web Anotherfile-sharing or chattool Jinirepackaged A Sun product

18 18 $! Peer-to-peer platform Platform, language,and transportagnostic A way forpeersto find each otherand share information A set ofprotocols thatenable P2P applications and services A spec and a couple of reference implementations

19 19 & 1. It s Allabout Protocols (platform & transport & Serviceagnostics) 2. Pipes (XM L Messaging -Structured Data - Pipeline) 3. Core Principles 1. Provide Essential Building Blocks (Pipes, PeerGroups, Monitoring, Security) % PureP2P (No centralized services DNS) 3. Information flows towardsthe edges 4. Performance improve assystem ages

20 ' ( () Peer Groups Managesidentity of peers and peer groups Join, Leave, Create Peer Pipes Send any content(code, data,etc.) to peers Messagesin XM L format Secure pipessupported Peer Monitoring Check peers statusremotely,controlremote peer behavior Meter peers usage,setusage or bandwidthlimits Security Pluggable architecturesupportsa variety ofstandard security solutions Control group membership Encrypt datasent over pipes 20

21 21 *

22 22 $ + Define adomain oftrust Define asetofcom mon services (protocols) Define a scope ofpeerinteraction Define an administration and monitoring domain

23 23 Virtual Connection Channelbetween Peers Dynamic Pipe Endpoint Binding Structured Data Messages Pipes: point-to-point Propagate (1-to-Many)

24 24 ' Me mbership Peer Membership Protocol Discovery Peer Discovery Protocol Peer Resolver Protocol Pipe Pipe Binding Protocol Peer Endpoint Protocol Monitoring PeerInformation Protocol

25 25

26 26 &, Leverage surrounding peers,and groups Meetings pointsforfaraway peers,and groups No centralized naming Asynchronous protocol Pre-defined meeting points (bootstrapping) ReverseDiscovery Learn about otherpeer'sabilities

27 27 *! - Route messages between multi-hops peers Every peer can be arouter HTTP Routing Server Traverse Firewalls N AT Support Message Gate ways (TTL)

28 28

29 29! Consumerfile sharing (like Gnutella) Distributed processing (like Distributed datastorage Com munications (likeim)

30 30 Both: Are distributed P2P technologies Are platform-independent Allow the creation of spontaneous,self-healing networks Define how to find other peers,but not how to com municate with them

31 31 Project JXTA is: Protocol spec Language-independent Inherently secure Uses published advertisements Concerned with natural grouping ofpeerservices Provides a way for peers to monitor each other Jiniis: API Java only Security now being added Uses serviceregistries Concerned with dynamic service delivery Requires no prior knowledge ofservices

32 32 $!!!"! $! ' ProjectJXTA s Technology open sourceavailable for download now (based on the Apache open source model) Developer tools,specifications, and documentation available at Applications using ProjectJXTA stechnologies in development Multipleimplementationsin development (Java, C,etc.)

33 33,! jxta.org Allsource, projects,docs, examples Apache-style software license No barriers to getting started Meritocracy The moreyou ve done,the more you areallowed to do

34 34 ' JX ME -JXTA binding forj2me JXTA- W ire - Many-to-many pipes ipeers -AI forjxta P2P - Decentralized Instant P2P -JXTA instant messager CMS -Contentmanagement system JXTASpaces -JavaSpaces forjxta

35 35 ' *!) Base on pricing mechanism Decentralized Markets Market Resource Agents(Providers) Act on behave of providersex.end-users,labs Market Resource Brokers(Consumers) Act on behave ofconsumers ex.scientists, enterprises.

36 36

37 37

38 38 './ // Simulation Space Client-Simulation Ad ministration HPC Space Complex Computation CADaemon Broker Both spaces require a point of reference which other entities within the network can relatesto.

39 39,!,! Component1 Component2 Component3 Component4 Library Server A Component6 Component5 Library Server B Component7 CADaemon Component8

40 40 0) w.onjava.com/pub/a/onjava/2001/04/25/jxta.html w.clip2.com/jxta_chat.html w w.jxta.org w w.jxta.org/project/w w w/getstart.html A

41 Simon See 41