Informatization and Grid Technology

Transcription

1 Informatization and Grid Technology Qian Depei Xian Jiaotong University May 21, 2004

2 Opportunity and challenge China is experiencing a rapid development in informatization Huge investment on information infrastructure Networks High performance computers Servers Software Great effort in developing applications Operations are being transferred to network environment People more and more rely on network and computers

3 Problems we are facing Waste of resources Huge investment every year to buy new computers and software while at the same time many machines are idle or under-utilized Economically undeveloped areas lack of computing resources

4 Problems we are facing Information exchange and sharing are obstructed by information island Applications are isolated and unable to talk to each other Precious data are owned by organizations and difficult to share Applications requiring comprehensive data are hard to develop

5 Problems we are facing High complexity of development restricts the applications Distributive, heterogeneous, versatile, and dynamically changing network environment makes the development of applications difficult Lack of talent people for developing applications

6 Problems we are facing High operation cost is intolerable Frequent upgrading of hardware and software is a heavy burden of enterprises and organizations Dedicated operation team has to be maintained by each enterprise or organization

7 Informatization is expensive! Initial cost Computers Networks Software Computer rooms Operation cost Staff Electricity maintenance Short life cycle of equipment

8 Role of grid technology in informatization Support informatization by enabling Resource sharing Cooperative work Service-based application development

9 How to judge the success of grid Lower the technical threshold for people Lower the complexity in development Lower the cost of operation

10 What is Grid? Inherently support to resource sharing and cooperative work Characterized by the service nature Support to virtual organizations

11 Is the concept new? The goals the grid pursue are what people have been working on in the past decades Grid is in growing stage

12 Can grid be the infrastructure? Parameters connect ability Op. & manage ment security availabi lity demand devices Power grids vol., frequency, phase Same vol, frequency, synchronization in phase Scheduling, billing Protection, stability control ubiquitous indispensable countless Computing grids Processor, OS, DB, language, Format, protocols,languages, runtime environment, Resource, user, job management, accounting Protection of data & privacy, against attacks & virus 7x24,pervasive, ubiquitous? What are killer applications? Servers, desktops, H/PC, PDA, IAs,

13 Grid has a long way to go to become the infrastructure Pervasiveness Support to applications Ease of use Efficiency of management Robustness of the system

14 Issues to address: Computing model Will the computing model developed for the current computer architecture be suitable for the grid system? How to map the problems into a dynamically growing grid environment? What will be the appropriate programming languages to convert the programming model into executable programs?

15 Issues to address: Storage model How to distribute data in the grid environment? How to organize the data storage to enable convenient data access and sharing? How to efficiently locate and access desired data?

16 Issues to address: Information model What is the appropriate scheme to represent the information in a highly heterogeneous environment? How to efficiently exchange and accurately acquire the information?

17 Issues to address: Resource management How to manage the ever-changing resources? How to efficiently schedule the resources scattered in different geographical locations? How to guarantee the QoS for different applications?

18 Issues to address: Service mechanism How to represent the semantics of the services? How could the application developers understand the semantic meaning of the services and match them with their application? How to manage the growing set of grid services. What will be the effective mechanism for service deployment, locating, and activation?

19 Issues to address: Security How to establish the trust relationship between the resource providers and the resource consumers who are totally irrelevant in the virtual resource space? How to ensure the safety of different entities in the grid system? How to protect privacy while enhancing sharing?

20 Issues to address: Pervasive computing In a grid-enabled pervasive computing environment consisting of billions of stable, mobile, general purpose, or embedded computing devices, how to confine the actions of different entities while providing the convenience in interaction? How to enhance the functionality and performance by coordination among entities?

21 Issues to address: Standards Establishing standards has critical importance to the development of the grid technology and its application.

22 Issues to address: Business model Will it be feasible to centrally manage and run the grid system as the current telecommunication systems does? What will be the appropriate mechanism for running the grid business? What will the key features to attract venture capitals?

23 Overview of China National Grid A key project supported by the National High-Tech R&D Program (the 863 program), continuing effort to the HPCE project 4-year effort 100 million fund from the Chinese government (MOST) Two to three times more associated funds from local governments, application organizations, and industry

24 Overview of China National Grid Enabling resource sharing and cooperative works under the Internet environment Providing the most powerful computing capability to research Exploring new models and patterns of applications and services in the Internet Promote China s industry in high performance computers and grids Training of talent people in the field of the grid

25 Major tasks of the project Develop a test-bed for the new information infrastructure China National Grid (CNGrid) About 8 grid nodes across the country Interconnected by available networks Develop a 4TFlops high performance computer to equip the grid nodes Develop a grid software to support grid applications and grid operation and management Develop several application grids Productive systems Demonstrate the future grid applications

26 Guidelines of the project Application driven Technological leaping Multilateral cooperation Focus on the grid

27 Three levels of activities Appl. Grids Research Environment Manufacturing Service Grid Software Development Use Environment Environment Grid System Software HPC Database Data Apps Grid Resources Internet

28 The R&D supported An environment Test-bed for Grid technology--china National Grid Two HPCs One set of software Grid software to support operation of the test-bed and development of applications Application grids Applications in selected areas

29 Grid test-bed development Build China National Grid (CDGrid) Eight nodes across the country CAS Network Centers (Beijing, major node) Shanghai Supercomputer Center (Shanghai, major node Tsinghua University (Beijing) Applied Physics Institute (Beijing) University of Science and Technology of China (Hefei, Anhui) Xi an Jiaotong University (Xi an, Shaanxi) UNDT (Changsha) Hong Kong University (Hongkong)

30 Requirement to grid nodes Major nodes >1 Tflops Equipped with the machine developed by the project Ordinary nodes >100GFlops Provided by the organization supporting the node

31 grid nodes across China Beijing Xi an Changsha Hefei Shanghai Hong Kong

32 HPC development Two >4TFlops machines are being developed Dawning computer AMD 64-bit Opteron processor 11 Tflops peak performance Lenovo computer Intel Itanium 2 processor Tflops peak performance

33 Lenovo 6800 Completed in Nov TB momory 81TB disks Linpack performance (78.5% efficiency) Ranked at 14 in Top500 ranking Installed at CAS Network Center in the coming December

34 Lenovo 6800

35 Dawning 4000A Completed in Apr TFlops Linpack performance (67.8% efficiency) To be installed at the Shanghai Supercomputer Center

36 Dawning 4000A configuration 640 nodes, GHz AMD Opteron CPU 5TB memory 22.5TB local disks Three sets of networks Myrinet Gigabit Ethernet Fast Ethernet

37 Features of Dawning 4000A Clustering file system Job/resource management System management Grid router

38 Grid software development Grid system software Interfacing to heterogeneous systems Basic resource management & scheduling Communication Grid application development environment Remote debugging Performance tuning Grid user environment Grid browser GSML

39 Grid software position in OGSA Domainspecific profiles OGSA Platform Environmentspecific profiles More specialized & domain-specific services OGSA Platform services: registry, authorization, Grid software monitoring, data access, etc., etc. OGSI Host. Env. & Protocol Bindings Hosting Environment Hosting Environment Transport Protocol Other models Models for resources & other entities

40 Grid software layer Comp Data Transact Other Application System Security service Job service User manage Data service Grid manage Grid monitor User Envir. Grid S/W Information service Resource Grid resources (host, programs, instruments, file systems, database) wrapped as WS or GS

41 Deployment of grid software 1.0 Web Portal RA2 Web 浏览器 基于网格软件网格管理工具的应用网格操作环境 GS GT3 注册 Internet CNGrid GOS APIs CNGrid 信息管理和资源管理 GT3 core 网格服务器 GS or wrappers CNGrid 信息管理和资源管理 注册 GT3 core 网格服务器 RA3 本地资源驱动 GS or wrappers 资源外包 RA1 网格结点 A 本地资源驱动 RB2 网格结点 B RB3 RB1 WS WebSphere,.Net GS OGSA/OGSI 图 7 网格软件部署结构 网格应用 CNGrid GOS APIs 其它 APIs 网格应用服务器

42 Research on grid software technology Grid software architecture Device layer, bus layer, operation environment Grid address space model Physical address, virtual address, effective address Mapping between different addresses Grid job management Virtual address based service invoke Job schedule, load balance, fault-tolerance Grid information management Distributed grid information management Grid resource management Dynamic resource deployment

43 Grid software deployment Version 1.0 in Feb Deployed in March 2004 on the test-bed

44 Application grids development Selected areas Resource and Environment Research Services Manufacturing Must be real applications

45 Geological Survey Grid Undertaken by China National Geologic Survey Bureau Nation-wide productive grid Data exchange and sharing Resource evaluation services Shorten the evaluation time from 3 years to 3 months Delivering achievements in North China ground water resources evaluation Mineral resources evaluation

46 Digital Forest Grid Undertaken by the Institute of Forest Resources Information Monitoring the status of forests in China Integration of remote sensing data, GIS information, and management information Evaluation of the process of turning the farmland back to forests Reflect true condition Guarantee the quality

47 Meteorological Service Grid Undertaken by China National Meteorological Center Three sites in Beijing, Guangdong, and Hunan, forming a grid Sharing of computing resources Platform for digital weather forecasting pattern research Provide new weather forecast services (time and location-specific)

48 Scientific Data Grid Undertaken by the Network & Information Center of CAS Integration of more than 40 scientific databases across the country Virtual astronomical observatory

49 New drug discovery grid Undertaken by Shanghai Institute of Materia Medica CAS Compound screening for new drug discovery Anti-SARS drugs discovery New drug for diabetes Using HPC in P2P mode

50 Bio-informatics Grid Undertaken by Genomics & Bioinformatics Institute, CAS Computing grid, data grid, and information grid for bioinformation research in the country

51 Manufacturing Grid Undertaken by China Aviation industry Corporation II Support distributed design, simulation, and manufacturing CAD software sharing Data exchange and sharing Computing resource sharing

52 China Grid Forum Promote research and development Exchange experiences Propose standards Set up a bridge between academic community and industry

53 China Grid Forum Established on Oct. 24, 2002 The first CGF conference on Sept. 12, 2003 Organizational Structure Advisory Committee Executive Committee Working Groups Grid architecture Data grids Information Grids Grid Applications

54 International cooperation IBM and Platform have participated in the work of the project Grid software Application grids IBM signed MOU with the expert group of the project Joint research is encouraged China international cooperation projects UK e-science

55 Expecting cooperation Interconnection with other grids Sharing of resources Computing power Application software Software tools Cooperation in proposing standards Personnel exchange Workshops

56 The role of government in promoting new technology Resource integration Set up collaboration mechanisms Regulation by policies Training

57 A good example The Shanghai Supercomputer Center is jointly supported by National high-tech R&D program Shanghai city government China Academy of Sciences

58 Our opportunities China has less legacy systems It is possible to adopt the newest technology to speed up informatization process with lower investment

59 Thank you!