GEOSS Clearinghouse onto Amazon EC2/Azure

Transcription

1 GEOSS Clearinghouse onto Amazon EC2/Azure Qunying Huang1, Chaowei Yang1, Doug Nebert 2 Kai Liu1, Zhipeng Gui1, Yan Xu3 1Joint Center of Intelligent Computing George Mason University 2Federal Geographic Data Committee (FGDC) AAG 2011 April 13th, Microsoft External Research, Microsoft

2 Outline Introduction Related Work Deploying GEOSS Clearinghouse onto Amazon EC2/Microsoft Azure Conclusion

3 Introduction Data, computational, spatialtemporal, user intensive (Yang et al., 2010) Scientific problems Spatial analysis /algorithm /applications Computing paradigm Cluster computing Grid computing Cloud Computing

4 Project Objectives GEOSS Clearinghouse Metadata catalogues search facility for the Intergovernmental Group on Earth Observation (GEO). GeoCloud Sandbox Common operating system and software suites Deployment and management strategies Usage and costing of Cloud services Security Amazon EC2 Cloud Enable GEOSS Clearinghouse Microsoft Azure

5 Cloud Computing Services Software as a Service (SaaS) Almost any IT services Users: End-user Platform as a Service (PaaS) Platform for developing and delivering applications, abstracted from infrastructures Users: Developer Infrastructure as a Service (IaaS) Access computing resources over internet Users: System Administrator ᄎ

6 Cloud Computing Platform Implementation VM VM VM Virtual Machine App App App OS OS OS VIM (OpenNebula, Eucalyptus Nimbus, Hadoop) Hypervisor Hypervisorr Hypervisorr Hypervisorr Hypervisorr Hardware Hypervisor Implement the capability of virtualization Xen VMWare KVM(Kernel-based Virtual Machine) Physical Infrastructure Virtual Infrastructure Middleware (VIM) Scheduling & monitoring Networking Management

7 Amazon EC2/Azure Implementation Amazon EC2 Virtualization Azure Middleware

8 Deployment of GEOSS Clearinghouse on Amazon/Azure Amazon EC2 Azure

9 Deployment of GEOSS Clearinghouse on Amazon EC2/Azure -cont Scalability Amazon: CloudWatch, Load balancer, Auto Scalar, CloudFormation Azure: API, Content delivery network (CDN) Reliability Network: SLA(Service Level Agreements) >99.95% Others: >99.99%

10 Performance Test GetCapabilies request from different number of concurrent requests Concurrent Request Numbers Amazon Failure Time Azure

11 Conclusion Cloud computing We are at a prescient time Technologies and open sources Cloud Architecture Platform independent languages Open data standards Spatial Cloud Computing Geospatial Middleware

12 Acknowledgements Sponsored by FGDC through GeoCloudSanbox Initiative project, and Microsoft through Cloud Enable GEOSS Clearinghouse Project Team: Chaowei Yang, Kai Liu, Jing Li, Min Sun, Chen Xu, Jizhe Xia

13 Reference Huang Q., Yang C., NebertD., Liu K., Wu H., Cloud computing for geosciences: deployment of GEOSS clearinghouse on Amazon's EC2, Proceedings of ACM GIS HPDGIS workshop, Armbrust, M., Fox, A. and Griffith, R., et al Above the Clouds: A Berkeley View of Cloud Computing. Tech. Rep. UCB/EECS , EECS Department, University of California, Berkeley, CA, (Accessed March, 2010). NASA Cloud Computing Platform. (Accessed March, 2010). Huang Q. and Yang C Optimizing Grid Computing Configuration and Scheduling for Geospatial Analysis-- An Example with Interpolating DEM. Computers & Geosciences, doi: /j.cageo Sun White Paper, Introduction to Cloud Computing Architecture, 1 edition, June, Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., and Warfield, A Xen and the art of virtualization. In Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles (Bolton Landing, NY, USA, October 19-22, 2003). SOSP '03. ACM, New York, NY, DOI= Amazon Web Service. Available at: (Access June, 2010). Lucene. Available at: ( (Access Sep, 2010). Pavlo, A., Paulson, E., Rasin, A., Abadi, D. J., DeWitt, D. J., Madden, S., and Stonebraker, M A comparison of approaches to large-scale data analysis. In Proceedings of the 35th SIGMOD international Conference on Management of Data (Providence, Rhode Island, USA, June 29 - July 02, 2009). C. Binnig and B. Dageville, Eds. SIGMOD '09. ACM, New York, NY, DOI= Yang C., Raskin R., Goodchild M.F., GaheganM., 2010a, Geospatial Cyberinfrastructure: Past, Present and Future, Computers, Environment, and Urban Systems, 34(4): Yang, C., Wu. H., Huang, Q., Li, Z. and Li, J., 2010b. Spatial Computing for Supporting Physical Sciences. PNAS. (in press). Armbrust, M. et al A view of cloud computing. Communications of the ACM, 53(4):

14 Thank You! Qunying Huang Chaowei Yang Doug Nebert CISC GEOSS Hosted in CISC Data Center Azure GEOSS Instance Amazon EC2 Instance 1.amazonaws.com/geonetwork/