IEEE WETICE Reference Model. Software. Wenke Ji /Jason Ma

Transcription

1 IEEE WETICE 2009 A Cloud Operating Reference Model and dopen Source Software Wenke Ji /Jason Ma

2 Agenda Authors Brief A Reference Model of Cloud Operating The Reference Model in Detail Sample Application with the Reference Model Lesson Learned and Our Next Step Cloud Heads-up in China 2

3 Authors Brief We Are From China Wenke Ji: Independent Consultant, 11 years industry experience from Huawei, Motorola etc. Graduated from Nanjing University with honor, BSEE,1998 Contact : jiwenke@gmail.com Jason Ma : Technical Manger in Alcatel-Lucent, 8 years industry experience in International Telecom Company Graduated d from Nanjing University,MSEE,2001 it Contact : majb822@gmail.com Project is Supported by Network Information Center, Nanjing University, P.R.C 3

4 The Source Of the Model Internet Cloud Cloud Non - Internet et OS Based Unix for Server Windows for PC RTOS for embed Network Based Browser Mobile Device Open Source Software Question Is the simplicity of Unix still work? How open source software play in Cloud? Now 4

5 The Reference Model Cloud Users XaaS Stack Application & Cloud Application Developers & Simplicity of UNIX Platform Infrastructure Perspective Analogy 3 Layer Hierarchy 5

6 The Reference Model Cont. Infrastructure layer for the Resource CPU Memory Bandwidth Storage Resource virtualization Platform layer for the service integration Kernel Cloud IO Computing driver and engine Management and UI interface Application layer hosts domain specific service It depends (Search, SNS...etc.) 6

7 The Reference Model in Detail Cloud computing infrastructure Mainframe, cluster of commodity PC and storage High speed network connection Resource virtualization OS for individual node Open source software can play in: Linux for individual computation node XEN for CPU virtualization Amazon EC2/S3 as public service on demand Question : How to virtualize various resources (CPU/Storage/Network) in a unified way to reduce the infrastructure operation cost? Can resource be dynamically provisioned and load balance on demand? 7

8 The Reference Model in Detail Distributed File System Provide high data availability Consistent view of data to client and transactional capability Scale-up ability Effective secure manner Open Source Software Implementation Apache Hadoop DFS (HDFS) Others Java based Standalone file system as dependency Google File System(GFS) Red Hat Global File System(RGFS) Amazon S3 as commercial service 8

9 The Reference Model in Detail Kernel Component Distribute task management Task creation, assignment, schedule, exception handling Map-Reduce model All-in Paris model Distribute memory management Serve large amount memory for consistent cache System status monitor CPU, Memory, Bandwidth and Storage Communication utility location independent data exchange Service for task and memory management 9

10 The Reference Model in Detail Open Source Software Implementation Apache Hadoop: MapReduce as distributed task management model Erlang Question: Is there any QoS(Quality of Service) assurance for the task? 10

11 Cloud IO The Reference Model in Detail A library to deliver data exchange service for the distributed tasks Support different data exchange protocol Open Source Software Implementation: Apache Hadoop IO package Web service such like Burlap/Hessian Question: How to support data exchange between different clouds? Is there any standard protocol needed for the data exchange? 11

12 The Reference Model in Detail Computing Engine and Driver Domain specific computing utility service to the application Third party software integration Open Source Software Implementation GNU Octave Apache Lucene 12

13 The Reference Model in Detail Management and User Interface Management console for the system admin and user as interface to the cloud Support monitor and manage cloud platform and applications FCAPS model implementation Application life cycle management Open Source Implementation Web2.0 technology and Open Source J2EE stack Apache Wicket / Flex /GWT Spring Framework Hibernate/Apache ibatis MySQL as management DB 13

14 The Reference Model in Detail System and User Application System application serve other application User application serve end-user Domain specific service Question: How to manage the application's life cycle? Can application "write once and run anywhere(on any cloud)"? What is the security model for the cloud application? 14

15 Sample ONE with the Model Parallel HDL(Hardware Description Language) Simulation System o HDL simulation is computation intensive especially for regression verification o Setup up parallel simulation environment based on open source software o Integrate the simulation engine into cloud platform Implementation Architecture o Kernel Layer : Hadoop on local Linux cluster and Amazon EC2 for computation scale o Driver Layer: o Verilog PLI C library development o Verilog Icarus o Open Source Verilog Simulator o Integrated as driver into the application o Application Layer: o Verilog testbench o JUnit extension application 15

16 Sample TWO with the model Face Search Engine on Cloud Computing Functionality: Find 5 best matched face through a large collections of face database, including faces; Implementation Architecture MapReduce as Kernel Hadoop DFS Hadoop IO Face Recognition Engine Cloud Computing Infrastructure 16

17 Sample TWO with the Model Face Search Engine on Cloud Computing Cont. MapReduce as Kernel: receive a search request, break down the request to work items, distribute work items to one node and combine the result; Hadoop DFS: store a large collections of face database among nodes. Hadoop IO: As a pre-defined IO, MapReduce use it for face database indexing and break down the search request; Face Recognition Engine: a SDK toolbox based on GNU Octave including: face detect algorithm, face feature extraction algorithm face recognition algorithm. 17

18 Lesson Learned from the Sample Application Lesson Learned Using open source software, don t hesitate to build, hack the source code and extend it when it needs Use the Java for the application environment, because it has wealthy open source software support and good portability of the code Use local Linux cluster for application prototype and use Amazon AWS for the service scalability Reuse the existed open source component, focus on the value added part of the application and the integration with Cloud platform Considering integrate the driver/engine as inter-process computation, it will improve the application performance 18

19 Our Best Guess for the Question Question in P.7 for the infrastructure layer: o Virtualization technology is the key of cloud o Various resource has its own virtualization implementation(cpu/storage/bandwidth) o Cloud, to be serve user as an utility, various virtualized resource needed to be managed in a unified way, it can reduce the cost and complexity of the management o If the management is across clouds, the standard for the virtualized resource management is needed o Unified way virtualization resource management provide capability for the resource provisioned dynamically 19

20 Our Best Guess for the Question Question in P.10 for the kernel component: o QoS of the task o In Cloud, the service task share the resource( CPU/Bandwidth/Storage) which means there are task race existed o For task management, the information is needed about the QoS management o If Cloud can provide various task model to the application, more kinds of service can be hosted 20

21 Our Best Guess for the Question Question in P.11 for the Cloud IO component: Data exchange protocol is needed for the interoperability between the different clouds Move computation instead of data to reduce the data transfer load Web service is a good way for data exchange, but if the API can be standardized, it will in great help Amazon AWS is a good example for its web service API 21

22 Our Best Guess for the Question Question in P.14 for the Cloud application component: For application life cycle management, maybe experience in other domain can be referred such like OSGi in Java application. With the life cycle management support and a set of standard cloud resource usage API, the application can be portable in different physical cloud. Cloud resource are shared by different applications, an security model is needed for the individual application. 22

23 Our Next Step in Plan Open Source Software Cloud Integration o Cloud management layer with web2.0 technology Apache Wicket/Flex/GWT as UI SpringFrameworks Hibernate/Apache ibatis MySQL o Apache Hadoop o Erlang Cloud Security Research SaaS S application as cloud application 23

24 Cloud Heads-up in China Jumping Start and in Great Demand o Service Provider by Internet - Baidu(Search Engine), alibaba(b2b), taobao(c2c), Tencent(IM), 51.com(SNS), SNDA(Online Game) o XaaS Service Provider SaaS superstar like Saleforces in China :? Alisoft : PaaS to independent SaaS provider IaaS service: IDC(Internet Data Center) transition to Cloud based 24

25 Cloud Heads-up in China (Cont.) Jumping Start and in Great Demand o Voice from telecommunication industry: "The cloud technology and the mobile computing are the two trends of the industry" - Wang JianZou, CEO of China Mobile at Davos Forum 2009 "It is an emerging trillion US$ biz in contend and application service by cloud technology" - Huawei Tech. 10 vision for telecommunication technology o Public cloud service Wuxi science park with IBM blue cloud operation, Wuxi, JiangSu province Alisoft Operating e-biz cloud in Nanjing 25

26 Thank you! Here we are: Nanjing famous for science/tech in China 26