Presentation Title. Grid Computing Project Officer / Research Assistance. InfoComm Development Center (idec) & Department of Communication

Size: px

Start display at page:

Download "Presentation Title. Grid Computing Project Officer / Research Assistance. InfoComm Development Center (idec) & Department of Communication"

Jonah Richard
5 years ago
Views:

Rafizan Ramly and Suhaimi Napis Grid Computing Project Officer / Research Assistance Institution Technology Event and

1 BIRUNI Grid glite Middleware Deployment : From Zero to Hero, towards a certified EGEE site Presentation Title M. Farhan Sjaugi,, Mohamed Othman, Mohd. Zul Yusoff, Speaker Mohd. Rafizan Ramly and Suhaimi Napis Grid Computing Project Officer / Research Assistance Institution Technology Event and Network, Name Faculty of Comp. Science and Information InfoComm Development Center (idec) & Department of Communication Tech. Universiti Putra Malaysia farhansj@biruni.upm.my 1

2 Outline Introduction BIRUNI GRID Infrastructure Line-Up Network Support Framework glite Middleware Deployment Planning Deployment Issues and Challenges Future Plans Concluding Remarks References 2

3 Introduction Motivation Academic staffs, research officers/assistants, and post- graduate students made up of over 80% of the entire research community in Malaysiai! (Not ncluding undergraduates) Working on Grid computing and its application in various disciplines Requires minimal facilities in their labs of respective universities iti to carry out their R&D Need something that they can play around and should anything happens to the facilities (eg. Nodes crashing, hanging, etc) they can simply poweroff and restart! 3

4 Introduction Basically two types of users: Researchers/students doing research on certain aspects of GRID Researchers/students using GRID for computation; (Comp. Science, Bioinformatics, Cheminformatics, Medical Informatics) Need basic Grid facilities and have total control of the entire Grid Ecosystem (all aspects of Grid) After acquiring certain level of expertise or if they want more compute power, then they can tap on other production facilities Hence the need for a learning and discovery Grid 4

5 BIRUNI Grid UPM has just commisioned BIRUNI GRID with the hope of making it the Nucleus HPC Clusters for A- Grid Hardware Racking and Power up were done by supppliers while the rest of the configuration and deployment was done by UPM Grid Team; Excellent learning opportunities. 5

6 Infrastructure Line-Up BIRUNI GRID has: 50 IBM Blade HS21 Servers (2x Intel Xeon Quad Core 2 Ghz, with8gbrams) rams). 3 IBM x3650 servers, one as Head Node and two as Storage Nodes. 2 IBM DS3000 series SAN with 24 Terabytes of storage capacity. BIRUNI GRID consists of three clusters: Khaldun Sandbox Cluster (7 worker nodes) Razi Cluster (28 worker nodes) Haitham Cluster (10 worker nodes) 6

7 Infrastructure Line-Up Khaldun Serve as a platform for research, application development and grid tutorial. The environmental provided almost identical with the production grid. The inter-node interconnection is done through gigabit ethernet technology. Razi Serve as the main platform for production grid. More intended for compute-intensive application. The inter-node interconnection is done through gigabit ethernet technology. 7

8 Infrastructure Line-Up Khaitam Serve as the main platform for production grid. More intended for data-intensive application. The inter-node interconnection is done through highspeed infiniband technology. 8

9 Network Support 9

10 Network Support 10

11 Development Framework GRID APPLICATION (Parallel Programming Libraries, Java-RMI RMI, RPC, etc..) MIDDLEWARE (Distributed Algorithm, Grid Services, etc..) NETWORK SUPPORT (Reliable High-Speed Data Transfer, etc..) 11

12 glite Middleware Deployment The glite distribution is an integrated set of components designed to enable resource sharing. In other words, this is middleware for building a grid. The glite middleware is produced by the EGEE project. The distribution model is to construct different services ('node-types') from these components and then ensure easy installation and configuration on the chosen platforms (currently Scientific Linux versions 3 and 4). glite middleware is currently deployed on hundreds of sites as part of the EGEE project and enables global science in a number of disciplines, notably serving the LCG project. 12

13 glite Middleware Framework 13

14 Planning Biruni Grid team members start with minimum experience in Grid Middleware deployment (i.e. glite Middleware), though most of the team members are familiar with HPC and system administration. Trial-and-Error methodology to familiarize with the middleware. Virtualization was chosen as the test-bed. Work side-by-side with Academia Sinica Grid Computing (ASGC), Taiwan to accelerate the adoption of the middleware as well as building expertise. 14

15 Deployment With guidance from ASGC, BIRUNI Grid team start deploying glite Middleware. Start with Computing Element and 14 worker nodes, followed by Storage Element with 1 Tera-Byte storage capacity. It takes about two months with intensive communication with ASGC for BIRUNI Grid to deploy and test. Finally BIRUNI Grid to got EGEE-site certified status. Incrementally other glite middleware service and policy are added (i.e. WMS, UI, Grid Portal). 15

16 Issues and Challenges Still find difficulties to maintain the grid site from error, but.. BIRUNI Grid team learn a lot on how to troubleshoot the problems. Now Razi receives in average 74 jobs per-hour (through EUASIA VO) with its 80 jobs per-hour of current maximum job capacity. The capacity of production grid will be increased from time-to-time. Though grid utilization is high, most of the submitted job is come from partners. Still very low utilization from internal organization. 16

17 Issues and Challenges 17

18 Future Plans Providing backup grid service for EUASIA grid VO (WMS, LFC). Grid Application deployment for Bioinformatics/Cheminformatics and Medical Informatics and Graphic/Image Rendering Promote greater awareness on Grid Computing and shared resources among researchers and stakeholders of Institutions of Higher Learning in Malaysia through the Ministry of Higher Education which will soon be the custodian of MYREN Phase 2. Becoming Certification Authority to serve regional users. 18

19 Concluding Remarks Universities not only should have enough resources to build massive compute facilities but also should have access to sandbox, preproduction and also production Grid A small but modest compute facilities where students have the freedom to try out new things without fear of wrecking a big Grid system should be available BIRUNI Grid provide a platform and compute facilities where students and academics learn about Grid technology and its applications, 19

20 Concluding Remarks Happy to have been able to build BIRUNI GRID from scratch (nearly ); very satisfying and at the same time acquire the skill to deploy HPC Clusters UPM wishes to acknowledge Academia Sinica for their undying support right from the planning of what to purchase right up to certification All these has been made possible through EUAsiaGrid Project; bridging not just between EU and Asia but also equally important among Asian countries 20

21 References BIRUNI Grid Center my BIRUNI Grid Statistic - tw/gstat/my-upm-biruni- 01/ glite Middleware - ch/ EUAsiaGrid 21

Journey Towards Science DMZ. Suhaimi Napis Technical Advisory Committee (Research Computing) MYREN-X Universiti Putra Malaysia

Journey Towards Science DMZ. Suhaimi Napis Technical Advisory Committee (Research Computing) MYREN-X Universiti Putra Malaysia Malaysia's Computational Journey Towards Science DMZ Suhaimi Napis Technical Advisory Committee (Research Computing) MYREN-X Universiti Putra Malaysia suhaimi@upm.my In the Beginning... Research on parallel/distributed