Comparative evaluation of software tools accessing relational databases from a (real) grid environments
|
|
- Sharlene Carpenter
- 5 years ago
- Views:
Transcription
1 Comparative evaluation of software tools accessing relational databases from a (real) grid environments Giacinto Donvito, Guido Cuscela, Massimiliano Missiato, Vicenzo Spinoso, Giorgio Maggi INFN-Bari 18
2 Outline The tools under evaluation The test plan Results of simple query test Grid test: description Grid test: results Grid test: Considerations About standards 19
3 The tools under evaluation G-DSE (INAF + INFN) Developed by INAF and INFN Edgardo Ambrosi (amborsi@cnaf.infn.it) Giuliano Taffoni (taffoni@oats.inaf.it) Andrea Barisani (lcars@infis.units.it) Project site: OGSA-DAI Developed as part of the Open Middleware Infrastructure Institute UK (OMII-UK) project. Project site: AMGA Developed as part of glite by: Birger Koblitz: Initial design, project responsible. Tony Calanducci: RPM building and testing. User support. Salvatore Scifo: Java API maintainer Claudio Cherubino: PHP client Project site: 4 2
4 Enabling Grids for E-sciencE Metadata Catalog of the glite Middleware (EGEE) Main usages: File metadata service, providing means of describing and discovering data files required by users and their jobs; Grid Enabled Database, for applications requiring to structure their data, providing database-like service supporting Grid Security Access existing RDBMS from a grid environment Several groups of users among the EGEE community: High Energy Physics Biomed The AMGA Metadata Catalog Main features: Dynamic schemas Hierarchical organization Security: Authentication: user/pass, X509 Certs and proxy, GSI Authorization: Unix-like, VOMS Role/Groups, ACLs per collections or per entry SQL-like query language: selectattr /DLAudio:FILE /DLAudio:Author /DLAudio:Album like(/dlaudio:file, %.mp3") Authentication: user/pass, Different join types (inner, outer, left, right) Support for views, indexes, constraints INFSO-RI
5 Enabling Grids for E-sciencE C++ multiprocess server Backends Oracle, MySQL 4/5, PostgreSQL, SQLite Front Ends TCP text streaming High performance Client API for C++, Java, Python, Perl, PHP SOAP (web services) Interoperability Scalability WS-DAIR interface - NEW - contribid=46&sessionid=28&confid=22351 Standalone Python Library implementation Data stored on file system AMGA Implementation AMGA server runs on SLC3/4 (32 or 64bit), Fedora Core, Gentoo, Debian AMGA provides a replication/federation mechanisms Motivation Scalability Support hundreds/thousands of concurrent users Geographical distribution Hide network latency Reliability No single point of failure DB Independent replication Heterogeneous DB systems Disconnected computing Off-line access (laptops) Architecture Asynchronous replication Master-slave writes only allowed on the master Application level replication Replicate Metadata commands Partial replication supports replication of only sub-trees of the metadata hierarchy EGEE-II INFSO-RI
6 The new G-DSE An updated version of the G-DSE has been issued in December 2007 which is now installed at INAF-Trieste and at INFN-Bari and under evaluation in the framework of the Grid & DB test campaign The new G-DSE includes a number of new features: Fully integrated with LFC (Logical File Catalogue) Support for Encryption
7 The new G-DSE/Features and improvement Support for GSI and SSL security Client/server structure A new server is forked by the listener when a new client asks for the service The server exits when the client disconnects Interactive It is possible to creates an interactive session that can be used for several queries The client server architecture overcomes the overhead problem Previous version: each job simply submits a single query it is prone to the Grid overhead Current version: the job initiates an interactive session that allows the user to submit multiple queries Backward compatibility: the user can still work in batch mode or single query per job
8 OGSA-DAI An extensible framework accessed via web services that executes data-centric workflows involving heterogeneous data resources for the purposes of data access, integration, transformation and delivery within a Grid and is intended as a toolkit for building higher-level application-specific data services Grid is about sharing resources OGSA-DAI is about sharing structured data resources 8
9 OGSA-DAI generic web services Manipulate data using OGSA-DAI s generic web services Clients sees the data in its raw format, e.g. Tables, columns, rows for relational data Collections, elements etc. for XML data Clients can obtain the schema of the data Clients send queries in appropriate query language, e.g. SQL, XPath 9
10 The test plan Sequential tests extraction of 10, 100, 1000, 10000, simple tuples extraction of result sets of increasing dimensions: O(kb), O(MB), O(100MB) Submission of complex queries (join, multiple queries, etc) Submission of INSERT, UPDATE, and DELETE queries Performance evaluation for a single action Evaluate the differences between LAN and WAN queries Concurrent tests Whit O(10) concurrent clients extract Zero, 10, 100, 1000 tuples Repeat the extractions with O(100) concurrent clients Use a common working environment for the three tools
11 Security Tools GSI VOMS Authentication X OGSA-DAI Yes No Yes Transport Layer Security Yes Data Encryption Yes G-DSE Yes Yes Yes No X Yes AMGA Yes Yes Yes Yes
12 Simple query: test results (2) Number of Tuples OGSA-DAI CSV CLI (s) OGSA-DAI CSV API (s) OGSA-DAI CLI (s) GDSE (s) AMGA (s) 1 5,07 0,23 4,7 0,180 0, ,13 0,26 4,77 0,198 0, ,31 0,27 4,65 0,215 0, ,32 0,304 5,25 0,196 0, ,43 0,324 6,15 0,198 0, ,61 0,45 6,37 0,283 0, ,63 0,65 7 0,343 0, ,74 2,41 14,8 0,853 2, ,96 4,61 24,46 1,485 4, ,31 15,21 95,63 6,521 21, ,42 34,21 188,86 12,639 41,336 The new release of GDSE reduces, not only the overhead for smallest query, but also the time spent retrieving the largest ones. The previous table is available at:
13 Simple query: test results (2) 2,500 1,938 1,375 0,813 0,250-0,313-0,875-1,438-2,000 1Tuple 5 Tuples 10 Tuples 50 Tuples 100 Tuples 500Tuples 1000 Tuples 5000 Tuples Tuples Tuples Tuples OGSA-DAI CSV CLI (s) OGSA-DAI CSV API (s) OGSA-DAI CLI (s) GDSE (s) AMGA (s)
14 Concurrent test from Grid: description In order to test the tools in an environment as close as possible to a real case queries were issued from WNs over the grid The number of jobs/clients running at a given time was measured Also all the successful queries executed against each server were accounted. The full EGEE infrastructure was used to execute the test Using always the same proxy certificate for all the jobs. The test looks for the number of served queries, the stability and reliability The test allows also to weight to the ability of each specific tool to overcome the network latency while delivering the output
15 Concurrent test from Grid: description A slightly modified version of JST (Job Submission Tool: a tool developed inside LIBI project) was used to monitor, in real time, the status of each job over the grid JST, in fact, allows the monitoring of any information coming out from a job (for example: Successful completion of a query) Two different types of query were used: select * from molecule where id<2 to select only a single tuple (< 1Kbyte) select * from molecule where id<1001 to select 1000 tuples (~500Kbyte) The plots present the number of successful queries in a 5 minute interval at server side (averaged over several runs)
16 Concurrent test from Grid: how we run it Job Submission Monitoring informations Query to DB Access Middleware WN WN WNWNWN WN WNWNWNWN WMS UI Monitoring DB Software under test Back-end DB
17 Concurrent test from Grid: how it works When the job lends on a WN, it: asks (to the central JST DB) for the tool (AMGA, G-DSE, OGSA-DAI) to test prepares the environment to run the right client makes the query: for each tool, it uses a CLI with one single query checks the output of the query if the query was correctly executed it updates the the central JST DB and... repeats all operations for 5000 more time
18 Concurrent test from Grid: which client we uses AMGA: a loop of: GDSE:./mdcli "selectattr /molecule:id /molecule:id_name / molecule:id_molecule_type /molecule:id_data_class / molecule:id_sequence_type /molecule:seq /molecule:seq_check / molecule:mw /molecule:length1 /molecule:createddate / molecule:lastupdateddate /molecule:lastannoteddate / molecule:descr /molecule:id_database_division /molecule:id_note / molecule:id_db '/molecule:id<1000 and /molecule:id>0'" > out_test (with a given configuration file) a loop of the new client: OGSA-DAI: gdse-connect -h server.name -p ${PORT} "select * from molecule where id<2;" we developed a ad-hoc client using: SQLQueryCSVAggregate class, with a single query built-in.
19 Concurrent test from Grid: possible issues Is the monitoring DB fast enough? we tested, successfully, the monitoring DB up to ~550 inserts per second and 200 concurrent clients This is far from the highest limit observed with db access tool under test Are there any network latency (or other kind of bias) on the client side? The jobs are distributed with FazzyRank enabled then several different farms are used at the same time: This should avoid any systematic differences between the tools do to client s network (or others client side problems ) Is the bandwidth on the server a bottleneck? We monitored the server constantly and this never occurred The result set is quite small: up to 500 Kbytes Is the test result depended from the geographical job distribution? We can change in real time the tool under test in order to be sure that no bias was introduced by a particular job distribution
20 Concurrent test from Grid: possible issues Is the back-end DB a bottleneck? We designed the test in order to do a lot of small select query: This would benefit from large memory buffers on the DB side Anyway this happened only in one case: and we easily spotted-out mostly due to AMGA usage of the DB and we fixed the problem There are some other activity on the server side? We worked in an absolute controlled environment Each server is used only for this test and no other service is running on the same machine We monitored the server using top and GANGLIA for all the time of each test How accurate is the measure of number of the successful queries : We compared the monitoring procedure against AMGA logs: we found -> 0.021% of difference between the number of queries logged into monitoring DB and the one logged into AMGA logs
21 Concurrent test from Grid: results # of queries every 5 minutes 77787, , , ,75 1 Tuple It seems that AMGA can still scale increasing the number of concurrent clients with the given hardware in this scenario Good reliability and stability of the numbers achieved which extends also beyond 150 client shown here # of client AMGA OGSA-DAI
22 Concurrent test from Grid: results Tuple Good reliability and stability of the numbers achieved which extends also beyond 150 client shown here AMGA OGSA-DAI # of clients
23 Concurrent test from Grid: comments AMGA show a very high number of queries served per time unit: it puts much more load on the back-end DB SQL support is increasing, but it is still not SQL compliant (maybe in the future: AMGA 2.0?) In this test we used: the TCP (not the fully Web Service interface) connection to the server SSL session recycling on the server side Very interesting functionality of replication and federation An AMGA Web Interface is available to browse/edit/create entries, make queries, define schemas, define permissions GDSE was not used in this test: Still in beta-release : the developers need some time to work in order to gain in stability, but: really fast for largest queries good interoperability with glite infrastructure and services Client/server architecture
24 Concurrent test from Grid: comments OGSA-DAI shows a good number of queries served per time unit: The result was retrieved using CSV format some stability issues where encountered with more than 80 concurrent clients are running against the same server, but... The developers have investigated the problem and give us suggestion on how to tune the software to achieve better performances Now good stability also with high load (proved up to 190 concurrent clients) Full Web-services solution for the query submission Advanced workflow features provided
25 Toward a standard and common interface? As it is already for flat-file access (many communities are using successfully SRM), there is the need for a standard interface for querying databases. There is already standards agreed within the OGF WS-DAI: a collection of generic data interfaces developed by the Database Access and Integration Services (DAIS) Working Group. WS-DAIR: a specification for a collection of data access interfaces for relational data resources, which extends interfaces defined in the Web Services Data Access and Integration document [WS-DAI]. WS-DAIX: a specification for a collection of data access interfaces for XML data resources, which extends interfaces defined in the Web Services Data Access and Integration document [WS-DAI].
26 Status of the WS-DAI interfaces We are starting a comparison with the available WS-DAI interfaces What is the status? OGSA-DAI: WS-DAIX already released... we are starting our tests WS-DAIR will be released in 3 month from now AMGA: see in this workshop -> A WS-DAIR interface for the AMGA metadata catalogue We asked to test this interface and we will do the tests as soon as developers give us No news from others tools
27 Acknowledgments TAFFONI, Giuliano (INAF) VUERLI, Claudio (INAF) BARISANI, andrea (INAF) PASIAN, Fabio (INAF) MANNA, Valeria (INAF) GISEL, Andreas (CNR-ITB) CALANDUCCI, Antonio (INFN) AIFTIMIEI, Cristina (INFN) ATUL, Jain (INFN+Politecnico Bari) PIERRO, Antonio (INFN) Work supported in part by BioinfoGRID and LIBI projects 30 17
AMGA metadata catalogue system
AMGA metadata catalogue system Hurng-Chun Lee ACGrid School, Hanoi, Vietnam www.eu-egee.org EGEE and glite are registered trademarks Outline AMGA overview AMGA Background and Motivation for AMGA Interface,
More informationDr. Giuliano Taffoni INAF - OATS
Query Element Demo The Grid Query Element for glite Dr. Giuliano Taffoni INAF - OATS Overview What is a G-DSE? Use and Admin a DB: the Query Element; Upcoming Features; Working on QE People: Edgardo Ambrosi
More informationThe AMGA Metadata Service
The AMGA Metadata Service Antonio Calanducci National Institute of Nuclear Physics (INFN) - Catania EGEE Grid tutorial for Users and Sysadmin Barcelona, 14th-18th April 2008 www.eu-egee.org EGEE and glite
More informationglite Grid Services Overview
The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) glite Grid Services Overview Antonio Calanducci INFN Catania Joint GISELA/EPIKH School for Grid Site Administrators Valparaiso,
More informationAstrophysics and the Grid: Experience with EGEE
Astrophysics and the Grid: Experience with EGEE Fabio Pasian INAF & VObs.it IVOA 2007 Interoperability Meeting Astro-RG session INAF experience with the grid (from the IVOA 2006 Interop): In INAF there
More informationUnderstanding StoRM: from introduction to internals
Understanding StoRM: from introduction to internals 13 November 2007 Outline Storage Resource Manager The StoRM service StoRM components and internals Deployment configuration Authorization and ACLs Conclusions.
More informationGrid services. Enabling Grids for E-sciencE. Dusan Vudragovic Scientific Computing Laboratory Institute of Physics Belgrade, Serbia
Grid services Dusan Vudragovic dusan@phy.bg.ac.yu Scientific Computing Laboratory Institute of Physics Belgrade, Serbia Sep. 19, 2008 www.eu-egee.org Set of basic Grid services Job submission/management
More informationEUROPEAN MIDDLEWARE INITIATIVE
EUROPEAN MIDDLEWARE INITIATIVE VOMS CORE AND WMS SECURITY ASSESSMENT EMI DOCUMENT Document identifier: EMI-DOC-SA2- VOMS_WMS_Security_Assessment_v1.0.doc Activity: Lead Partner: Document status: Document
More informationThe glite middleware. Ariel Garcia KIT
The glite middleware Ariel Garcia KIT Overview Background The glite subsystems overview Security Information system Job management Data management Some (my) answers to your questions and random rumblings
More informationAMGA Metadata catalogue service for managing mass data of high-energy physics
AMGA Metadata catalogue service for managing mass Geunchul Park, Taesang Huh, SoonWook Hwang, Jaehyuck Kwak KISTI 2012. 02. 20 EMI is partially funded by the European Commission under Grant Agreement RI-261611
More informationWHEN the Large Hadron Collider (LHC) begins operation
2228 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 53, NO. 4, AUGUST 2006 Measurement of the LCG2 and Glite File Catalogue s Performance Craig Munro, Birger Koblitz, Nuno Santos, and Akram Khan Abstract When
More informationSPINOSO Vincenzo. Optimization of the job submission and data access in a LHC Tier2
EGI User Forum Vilnius, 11-14 April 2011 SPINOSO Vincenzo Optimization of the job submission and data access in a LHC Tier2 Overview User needs Administration issues INFN Bari farm design and deployment
More informationMetadaten Workshop 26./27. März 2007 Göttingen. Chimera. a new grid enabled name-space service. Martin Radicke. Tigran Mkrtchyan
Metadaten Workshop 26./27. März Chimera a new grid enabled name-space service What is Chimera? a new namespace provider provides a simulated filesystem with additional metadata fast, scalable and based
More informationI Tier-3 di CMS-Italia: stato e prospettive. Hassen Riahi Claudio Grandi Workshop CCR GRID 2011
I Tier-3 di CMS-Italia: stato e prospettive Claudio Grandi Workshop CCR GRID 2011 Outline INFN Perugia Tier-3 R&D Computing centre: activities, storage and batch system CMS services: bottlenecks and workarounds
More information30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy
Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Why the Grid? Science is becoming increasingly digital and needs to deal with increasing amounts of
More informationBookkeeping and submission tools prototype. L. Tomassetti on behalf of distributed computing group
Bookkeeping and submission tools prototype L. Tomassetti on behalf of distributed computing group Outline General Overview Bookkeeping database Submission tools (for simulation productions) Framework Design
More informationLCG-2 and glite Architecture and components
LCG-2 and glite Architecture and components Author E.Slabospitskaya www.eu-egee.org Outline Enabling Grids for E-sciencE What are LCG-2 and glite? glite Architecture Release 1.0 review What is glite?.
More informationDATABASE SYSTEMS. Introduction to MySQL. Database System Course, 2016
DATABASE SYSTEMS Introduction to MySQL Database System Course, 2016 AGENDA FOR TODAY Administration Database Architecture on the web Database history in a brief Databases today MySQL What is it How to
More informationg-eclipse A Framework for Accessing Grid Infrastructures Nicholas Loulloudes Trainer, University of Cyprus (loulloudes.n_at_cs.ucy.ac.
g-eclipse A Framework for Accessing Grid Infrastructures Trainer, University of Cyprus (loulloudes.n_at_cs.ucy.ac.cy) EGEE Training the Trainers May 6 th, 2009 Outline Grid Reality The Problem g-eclipse
More informationThe CORAL Project. Dirk Düllmann for the CORAL team Open Grid Forum, Database Workshop Barcelona, 4 June 2008
The CORAL Project Dirk Düllmann for the CORAL team Open Grid Forum, Database Workshop Barcelona, 4 June 2008 Outline CORAL - a foundation for Physics Database Applications in the LHC Computing Grid (LCG)
More informationAnalisi Tier2 e Tier3 Esperienze ai Tier-2 Giacinto Donvito INFN-BARI
Analisi Tier2 e Tier3 Esperienze ai Tier-2 Giacinto Donvito INFN-BARI outlook Alice Examples Atlas Examples CMS Examples Alice Examples ALICE Tier-2s at the moment do not support interactive analysis not
More informationDistributed production managers meeting. Armando Fella on behalf of Italian distributed computing group
Distributed production managers meeting Armando Fella on behalf of Italian distributed computing group Distributed Computing human network CNAF Caltech SLAC McGill Queen Mary RAL LAL and Lyon Bari Legnaro
More informationDSpace Fedora. Eprints Greenstone. Handle System
Enabling Inter-repository repository Access Management between irods and Fedora Bing Zhu, Uni. of California: San Diego Richard Marciano Reagan Moore University of North Carolina at Chapel Hill May 18,
More informationDATABASE SYSTEMS. Introduction to MySQL. Database System Course, 2016
DATABASE SYSTEMS Introduction to MySQL Database System Course, 2016 AGENDA FOR TODAY Administration Database Architecture on the web Database history in a brief Databases today MySQL What is it How to
More informationData Management 1. Grid data management. Different sources of data. Sensors Analytic equipment Measurement tools and devices
Data Management 1 Grid data management Different sources of data Sensors Analytic equipment Measurement tools and devices Need to discover patterns in data to create information Need mechanisms to deal
More informationEdinburgh Research Explorer
Edinburgh Research Explorer Profiling OGSA-DAI Performance for Common Use Patterns Citation for published version: Dobrzelecki, B, Antonioletti, M, Schopf, JM, Hume, AC, Atkinson, M, Hong, NPC, Jackson,
More informationCluster Setup and Distributed File System
Cluster Setup and Distributed File System R&D Storage for the R&D Storage Group People Involved Gaetano Capasso - INFN-Naples Domenico Del Prete INFN-Naples Diacono Domenico INFN-Bari Donvito Giacinto
More informationR-GMA (Relational Grid Monitoring Architecture) for monitoring applications
R-GMA (Relational Grid Monitoring Architecture) for monitoring applications www.eu-egee.org egee EGEE-II INFSO-RI-031688 Acknowledgements Slides are taken/derived from the GILDA team Steve Fisher (RAL,
More informationUtilizing Databases in Grid Engine 6.0
Utilizing Databases in Grid Engine 6.0 Joachim Gabler Software Engineer Sun Microsystems http://sun.com/grid Current status flat file spooling binary format for jobs ASCII format for other objects accounting
More informationProvide Real-Time Data To Financial Applications
Provide Real-Time Data To Financial Applications DATA SHEET Introduction Companies typically build numerous internal applications and complex APIs for enterprise data access. These APIs are often engineered
More informationAssignment 5. Georgia Koloniari
Assignment 5 Georgia Koloniari 2. "Peer-to-Peer Computing" 1. What is the definition of a p2p system given by the authors in sec 1? Compare it with at least one of the definitions surveyed in the last
More informationBeob Kyun KIM, Christophe BONNAUD {kyun, NSDC / KISTI
2010. 6. 17 Beob Kyun KIM, Christophe BONNAUD {kyun, cbonnaud}@kisti.re.kr NSDC / KISTI 1 1 2 Belle Data Transfer Metadata Extraction Scalability Test Metadata Replication Grid-awaring of Belle data Test
More informationA Simple Mass Storage System for the SRB Data Grid
A Simple Mass Storage System for the SRB Data Grid Michael Wan, Arcot Rajasekar, Reagan Moore, Phil Andrews San Diego Supercomputer Center SDSC/UCSD/NPACI Outline Motivations for implementing a Mass Storage
More informationAdvanced School in High Performance and GRID Computing November Introduction to Grid computing.
1967-14 Advanced School in High Performance and GRID Computing 3-14 November 2008 Introduction to Grid computing. TAFFONI Giuliano Osservatorio Astronomico di Trieste/INAF Via G.B. Tiepolo 11 34131 Trieste
More informationPrototypes of a Computational Grid for the Planck Satellite
ASTRONOMICAL DATA ANALYSIS SOFTWARE AND SYSTEMS XIV ASP Conference Series, Vol. 347, 2005 P. L. Shopbell, M. C. Britton, and R. Ebert, eds. Prototypes of a Computational Grid for the Planck Satellite Giuliano
More informationRecent Evolutions of GridICE: a Monitoring Tool for Grid Systems
Recent Evolutions of GridICE: a Monitoring Tool for Grid Systems Cristina Aiftimiei INFN-Padova Padova, Italy cristina.aiftimiei@pd.infn.it Vihang Dudhalkar INFN-Bari and Dipartimento di Fisica - Politecnico
More informationGrid Programming: Concepts and Challenges. Michael Rokitka CSE510B 10/2007
Grid Programming: Concepts and Challenges Michael Rokitka SUNY@Buffalo CSE510B 10/2007 Issues Due to Heterogeneous Hardware level Environment Different architectures, chipsets, execution speeds Software
More informationOracle Database Heterogeneous Connectivity User Guide
Oracle Database Heterogeneous Connectivity User Guide Database Heterogeneous Connectivity User's Guide. Contents This chapter describes the major features provided by Oracle Database gateways. Integrate
More informationThe glite middleware. Presented by John White EGEE-II JRA1 Dep. Manager On behalf of JRA1 Enabling Grids for E-sciencE
The glite middleware Presented by John White EGEE-II JRA1 Dep. Manager On behalf of JRA1 John.White@cern.ch www.eu-egee.org EGEE and glite are registered trademarks Outline glite distributions Software
More informationELFms industrialisation plans
ELFms industrialisation plans CERN openlab workshop 13 June 2005 German Cancio CERN IT/FIO http://cern.ch/elfms ELFms industrialisation plans, 13/6/05 Outline Background What is ELFms Collaboration with
More informationExtending the SDSS Batch Query System to the National Virtual Observatory Grid
Extending the SDSS Batch Query System to the National Virtual Observatory Grid María A. Nieto-Santisteban, William O'Mullane Nolan Li Tamás Budavári Alexander S. Szalay Aniruddha R. Thakar Johns Hopkins
More informationChapter 17 Web Services Additional Topics
Prof. Dr.-Ing. Stefan Deßloch AG Heterogene Informationssysteme Geb. 36, Raum 329 Tel. 0631/205 3275 dessloch@informatik.uni-kl.de Chapter 17 Web Services Additional Topics Prof. Dr.-Ing. Stefan Deßloch
More informationCOMMUNICATION PROTOCOLS
COMMUNICATION PROTOCOLS Index Chapter 1. Introduction Chapter 2. Software components message exchange JMS and Tibco Rendezvous Chapter 3. Communication over the Internet Simple Object Access Protocol (SOAP)
More informationFederated Authentication with Web Services Clients
Federated Authentication with Web Services Clients in the context of SAML based AAI federations Thomas Lenggenhager thomas.lenggenhager@switch.ch Mannheim, 8. March 2011 Overview SAML n-tier Delegation
More informationSecurity in distributed metadata catalogues
Security in distributed metadata catalogues Nuno Santos 1, and Birger Koblitz 2 1 Distributed Systems Laboratory, Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland 2 European Organization
More informationData Storage Infrastructure at Facebook
Data Storage Infrastructure at Facebook Spring 2018 Cleveland State University CIS 601 Presentation Yi Dong Instructor: Dr. Chung Outline Strategy of data storage, processing, and log collection Data flow
More informationHow to build Scientific Gateways with Vine Toolkit and Liferay/GridSphere framework
How to build Scientific Gateways with Vine Toolkit and Liferay/GridSphere framework Piotr Dziubecki, Piotr Grabowski, Michał Krysiński, Tomasz Kuczyński, Dawid Szejnfeld, Dominik Tarnawczyk, Gosia Wolniewicz
More informationForeScout Open Integration Module: Data Exchange Plugin
ForeScout Open Integration Module: Data Exchange Plugin Version 3.2.0 Table of Contents About the Data Exchange Plugin... 4 Requirements... 4 CounterACT Software Requirements... 4 Connectivity Requirements...
More informationAccessibility Features in the SAS Intelligence Platform Products
1 CHAPTER 1 Overview of Common Data Sources Overview 1 Accessibility Features in the SAS Intelligence Platform Products 1 SAS Data Sets 1 Shared Access to SAS Data Sets 2 External Files 3 XML Data 4 Relational
More informationGrid and Cloud Activities in KISTI
Grid and Cloud Activities in KISTI March 23, 2011 Soonwook Hwang KISTI, KOREA 1 Outline Grid Operation and Infrastructure KISTI ALICE Tier2 Center FKPPL VO: Production Grid Infrastructure Global Science
More informationLHCb Distributed Conditions Database
LHCb Distributed Conditions Database Marco Clemencic E-mail: marco.clemencic@cern.ch Abstract. The LHCb Conditions Database project provides the necessary tools to handle non-event time-varying data. The
More informationDatabase Assessment for PDMS
Database Assessment for PDMS Abhishek Gaurav, Nayden Markatchev, Philip Rizk and Rob Simmonds Grid Research Centre, University of Calgary. http://grid.ucalgary.ca 1 Introduction This document describes
More informationSZDG, ecom4com technology, EDGeS-EDGI in large P. Kacsuk MTA SZTAKI
SZDG, ecom4com technology, EDGeS-EDGI in large P. Kacsuk MTA SZTAKI The EDGI/EDGeS projects receive(d) Community research funding 1 Outline of the talk SZTAKI Desktop Grid (SZDG) SZDG technology: ecom4com
More informationAMGA tutorial. Enabling Grids for E-sciencE
AMGA tutorial Vladimir Slavnić slavnic@phy.bg.ac.yu Scientific Computing Laboratory Institute of Physics Belgrade, Serbia Sep. 19, 2008 www.eu-egee.org Arda Metadata Catalogue Project Data about data Simple
More informationDATABASE SYSTEMS. Introduction to MySQL. Database System Course, 2018
DATABASE SYSTEMS Introduction to MySQL Database System Course, 2018 CAUTION! *This class is NOT a recitation* We will NOT discuss the course material relevant to the exam and homework assignment We have
More informationAdvanced Database Systems
Lecture II Storage Layer Kyumars Sheykh Esmaili Course s Syllabus Core Topics Storage Layer Query Processing and Optimization Transaction Management and Recovery Advanced Topics Cloud Computing and Web
More informationSybase Adaptive Server Enterprise on Linux
Sybase Adaptive Server Enterprise on Linux A Technical White Paper May 2003 Information Anywhere EXECUTIVE OVERVIEW ARCHITECTURE OF ASE Dynamic Performance Security Mission-Critical Computing Advanced
More informationGrid Infrastructure For Collaborative High Performance Scientific Computing
Computing For Nation Development, February 08 09, 2008 Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi Grid Infrastructure For Collaborative High Performance Scientific
More informationEasy Access to Grid Infrastructures
Easy Access to Grid Infrastructures Dr. Harald Kornmayer (NEC Laboratories Europe) On behalf of the g-eclipse consortium WP11 Grid Workshop Grenoble, France 09 th of December 2008 Background in astro particle
More informationArgus Vulnerability Assessment *1
Argus Vulnerability Assessment *1 Manuel Brugnoli and Elisa Heymann Universitat Autònoma de Barcelona June, 2011 Introduction Argus is the glite Authorization Service. It is intended to provide consistent
More informationDatabase Server. 2. Allow client request to the database server (using SQL requests) over the network.
Database Server Introduction: Client/Server Systems is networked computing model Processes distributed between clients and servers. Client Workstation (usually a PC) that requests and uses a service Server
More informationBob Jones. EGEE and glite are registered trademarks. egee EGEE-III INFSO-RI
Bob Jones EGEE project director www.eu-egee.org egee EGEE-III INFSO-RI-222667 EGEE and glite are registered trademarks Quality: Enabling Grids for E-sciencE Monitoring via Nagios - distributed via official
More informationReal World Web Scalability. Ask Bjørn Hansen Develooper LLC
Real World Web Scalability Ask Bjørn Hansen Develooper LLC Hello. 28 brilliant methods to make your website keep working past $goal requests/transactions/sales per second/hour/day Requiring minimal extra
More informationITP 140 Mobile Technologies. Databases Client/Server
ITP 140 Mobile Technologies Databases Client/Server Databases Data: recorded facts and figures Information: knowledge derived from data Databases record data, but they do so in such a way that we can produce
More informationWeb Services in Cincom VisualWorks. WHITE PAPER Cincom In-depth Analysis and Review
Web Services in Cincom VisualWorks WHITE PAPER Cincom In-depth Analysis and Review Web Services in Cincom VisualWorks Table of Contents Web Services in VisualWorks....................... 1 Web Services
More informationWeb Applications. Software Engineering 2017 Alessio Gambi - Saarland University
Web Applications Software Engineering 2017 Alessio Gambi - Saarland University Based on the work of Cesare Pautasso, Christoph Dorn, Andrea Arcuri, and others ReCap Software Architecture A software system
More informationDatabase System Concepts and Architecture
CHAPTER 2 Database System Concepts and Architecture Copyright 2017 Ramez Elmasri and Shamkant B. Navathe Slide 2-2 Outline Data Models and Their Categories History of Data Models Schemas, Instances, and
More informationOracle 10g and IPv6 IPv6 Summit 11 December 2003
Oracle 10g and IPv6 IPv6 Summit 11 December 2003 Marshal Presser Principal Enterprise Architect Oracle Corporation Agenda Oracle Distributed Computing Role of Networking IPv6 Support Plans Early IPv6 Implementations
More informationMonitoring the ALICE Grid with MonALISA
Monitoring the ALICE Grid with MonALISA 2008-08-20 Costin Grigoras ALICE Workshop @ Sibiu Monitoring the ALICE Grid with MonALISA MonALISA Framework library Data collection and storage in ALICE Visualization
More informationPromoting Open Standards for Digital Repository. case study examples and challenges
Promoting Open Standards for Digital Repository Infrastructures: case study examples and challenges Flavia Donno CERN P. Fuhrmann, DESY, E. Ronchieri, INFN-CNAF OGF-Europe Community Outreach Seminar Digital
More informationDistributed Data Management with Storage Resource Broker in the UK
Distributed Data Management with Storage Resource Broker in the UK Michael Doherty, Lisa Blanshard, Ananta Manandhar, Rik Tyer, Kerstin Kleese @ CCLRC, UK Abstract The Storage Resource Broker (SRB) is
More informationCIB Session 12th NoSQL Databases Structures
CIB Session 12th NoSQL Databases Structures By: Shahab Safaee & Morteza Zahedi Software Engineering PhD Email: safaee.shx@gmail.com, morteza.zahedi.a@gmail.com cibtrc.ir cibtrc cibtrc 2 Agenda What is
More informationNewSQL Databases. The reference Big Data stack
Università degli Studi di Roma Tor Vergata Dipartimento di Ingegneria Civile e Ingegneria Informatica NewSQL Databases Corso di Sistemi e Architetture per Big Data A.A. 2017/18 Valeria Cardellini The reference
More informationMySQL Database Administrator Training NIIT, Gurgaon India 31 August-10 September 2015
MySQL Database Administrator Training Day 1: AGENDA Introduction to MySQL MySQL Overview MySQL Database Server Editions MySQL Products MySQL Services and Support MySQL Resources Example Databases MySQL
More informationMonitoring System for the GRID Monte Carlo Mass Production in the H1 Experiment at DESY
Journal of Physics: Conference Series OPEN ACCESS Monitoring System for the GRID Monte Carlo Mass Production in the H1 Experiment at DESY To cite this article: Elena Bystritskaya et al 2014 J. Phys.: Conf.
More informationExperiences Running OGSA-DQP Queries Against a Heterogeneous Distributed Scientific Database
2009 15th International Conference on Parallel and Distributed Systems Experiences Running OGSA-DQP Queries Against a Heterogeneous Distributed Scientific Database Helen X Xiang Computer Science, University
More informationSTATUS UPDATE ON THE INTEGRATION OF SEE-GRID INTO G- SDAM AND FURTHER IMPLEMENTATION SPECIFIC TOPICS
AUSTRIAN GRID STATUS UPDATE ON THE INTEGRATION OF SEE-GRID INTO G- SDAM AND FURTHER IMPLEMENTATION SPECIFIC TOPICS Document Identifier: Status: Workpackage: Partner(s): Lead Partner: WP Leaders: AG-DM-4aA-1c-1-2006_v1.doc
More informationRequirements for data catalogues within facilities
Requirements for data catalogues within facilities Milan Prica 1, George Kourousias 1, Alistair Mills 2, Brian Matthews 2 1 Sincrotrone Trieste S.C.p.A, Trieste, Italy 2 Scientific Computing Department,
More informationForeScout CounterACT. Configuration Guide. Version 3.4
ForeScout CounterACT Open Integration Module: Data Exchange Version 3.4 Table of Contents About the Data Exchange Module... 4 About Support for Dual Stack Environments... 4 Requirements... 4 CounterACT
More informationEGEE and Interoperation
EGEE and Interoperation Laurence Field CERN-IT-GD ISGC 2008 www.eu-egee.org EGEE and glite are registered trademarks Overview The grid problem definition GLite and EGEE The interoperability problem The
More informationConfiguring the Oracle Network Environment. Copyright 2009, Oracle. All rights reserved.
Configuring the Oracle Network Environment Objectives After completing this lesson, you should be able to: Use Enterprise Manager to: Create additional listeners Create Oracle Net Service aliases Configure
More informationInterconnect EGEE and CNGRID e-infrastructures
Interconnect EGEE and CNGRID e-infrastructures Giuseppe Andronico Interoperability and Interoperation between Europe, India and Asia Workshop Barcelona - Spain, June 2 2007 FP6 2004 Infrastructures 6-SSA-026634
More informationGrid monitoring with MonAlisa
Grid monitoring with MonAlisa Iosif Legrand (Caltech) Iosif.Legrand@cern.ch Laura Duta (UPB) laura.duta@cs.pub.ro Alexandru Herisanu (UPB) aherisanu@cs.pub.ro Agenda What is MonAlisa? Who uses it? A little
More informationWorkload Management. Stefano Lacaprara. CMS Physics Week, FNAL, 12/16 April Department of Physics INFN and University of Padova
Workload Management Stefano Lacaprara Department of Physics INFN and University of Padova CMS Physics Week, FNAL, 12/16 April 2005 Outline 1 Workload Management: the CMS way General Architecture Present
More informationHigh Throughput WAN Data Transfer with Hadoop-based Storage
High Throughput WAN Data Transfer with Hadoop-based Storage A Amin 2, B Bockelman 4, J Letts 1, T Levshina 3, T Martin 1, H Pi 1, I Sfiligoi 1, M Thomas 2, F Wuerthwein 1 1 University of California, San
More informationDataSunrise Database Security Suite Release Notes
www.datasunrise.com DataSunrise Database Security Suite 4.0.4 Release Notes Contents DataSunrise Database Security Suite 4.0.4... 3 New features...3 Known limitations... 3 Version history... 5 DataSunrise
More informationOverview of WMS/LB API
Overview of WMS/LB API Emidio Giorgio www.eu-egee.org Outline API vs. CLI Overview of WMProxy API Overview of Logging and Bookkeeping API Usage examples API vs. CLI Users applications can be integrated
More informationInternet2 Meeting September 2005
End User Agents: extending the "intelligence" to the edge in Distributed Systems Internet2 Meeting California Institute of Technology 1 OUTLINE (Monitoring Agents using a Large, Integrated s Architecture)
More informationGrid Computing Fall 2005 Lecture 5: Grid Architecture and Globus. Gabrielle Allen
Grid Computing 7700 Fall 2005 Lecture 5: Grid Architecture and Globus Gabrielle Allen allen@bit.csc.lsu.edu http://www.cct.lsu.edu/~gallen Concrete Example I have a source file Main.F on machine A, an
More informationJob Reference Guide. SLAMD Distributed Load Generation Engine. Version 1.8.1
Job Reference Guide SLAMD Distributed Load Generation Engine Version 1.8.1 December 2004 Contents 1. Introduction...3 2. The Utility Jobs...4 3. The LDAP Search Jobs...11 4. The LDAP Authentication Jobs...22
More informationBy Ian Foster. Zhifeng Yun
By Ian Foster Zhifeng Yun Outline Introduction Globus Architecture Globus Software Details Dev.Globus Community Summary Future Readings Introduction Globus Toolkit v4 is the work of many Globus Alliance
More informationGROWL Scripts and Web Services
GROWL Scripts and Web Services Grid Technology Group E-Science Centre r.j.allan@dl.ac.uk GROWL Collaborative project (JISC VRE I programme) between CCLRC Daresbury Laboratory and the Universities of Cambridge
More informationRegular Forum of Lreis. Speechmaker: Gao Ang
Regular Forum of Lreis Speechmaker: Gao Ang Content: A. Overview of Eclipse Project B. Rich Client Platform C. The progress of ustudio Project D. The development of Grid technology and Grid GIS E. Future
More informationTHEBES: THE GRID MIDDLEWARE PROJECT Project Overview, Status Report and Roadmap
THEBES: THE GRID MIDDLEWARE PROJECT Project Overview, Status Report and Roadmap Arnie Miles Georgetown University adm35@georgetown.edu http://thebes.arc.georgetown.edu The Thebes middleware project was
More informationGlobus GTK and Grid Services
Globus GTK and Grid Services Michael Rokitka SUNY@Buffalo CSE510B 9/2007 OGSA The Open Grid Services Architecture What are some key requirements of Grid computing? Interoperability: Critical due to nature
More informationGRID Stream Database Managent for Scientific Applications
GRID Stream Database Managent for Scientific Applications Milena Ivanova (Koparanova) and Tore Risch IT Department, Uppsala University, Sweden Outline Motivation Stream Data Management Computational GRIDs
More informationDistributed Systems. Overview. Distributed Systems September A distributed system is a piece of software that ensures that:
Distributed Systems Overview Distributed Systems September 2002 1 Distributed System: Definition A distributed system is a piece of software that ensures that: A collection of independent computers that
More informationA User-level Secure Grid File System
A User-level Secure Grid File System Ming Zhao, Renato J. Figueiredo Advanced Computing and Information Systems (ACIS) Electrical and Computer Engineering University of Florida {ming, renato}@acis.ufl.edu
More informationWP3 Final Activity Report
WP3 Final Activity Report Nicholas Loulloudes WP3 Representative On behalf of the g-eclipse Consortium Outline Work Package 3 Final Status Achievements Work Package 3 Goals and Benefits WP3.1 Grid Infrastructure
More informationPoint-in-Polygon linking with OGSA-DAI
Point-in-Polygon linking with OGSA-DAI Xiaochi Ma MSc in High Performance Computing The University of Edinburgh Year of Presentation: 2006 1 Point-in-Polygon linking With OGSA-DAI Xiaochi Ma 2 Abstract
More information