The CORAL Project. Dirk Düllmann for the CORAL team Open Grid Forum, Database Workshop Barcelona, 4 June 2008

Size: px
Start display at page:

Download "The CORAL Project. Dirk Düllmann for the CORAL team Open Grid Forum, Database Workshop Barcelona, 4 June 2008"

Transcription

1 The CORAL Project Dirk Düllmann for the CORAL team Open Grid Forum, Database Workshop Barcelona, 4 June 2008

2 Outline CORAL - a foundation for Physics Database Applications in the LHC Computing Grid (LCG) Project architecture and evolution Review of remaining deployment issues Secure database access Efficient use of server resources for many concurrent clients Software distribution CORAL server design study Integration with existing code base Server connection and threading model Network protocol structure Results from early prototype studies 2

3 Disclaimer This is a development proposal and does therefore contains in several areas: vapour ware, hand waving implementation choices may still change Still needs discussion beyond pure development deployment model/resources at T0 and T1 Still, the aim of this presentation is to convince you that the development is necessary to insure scalability and high availability of DB services design aspects are reasonably well understood could be introduced on a time scale of end of this year without major disruption to existing CORAL apps/services can be done with rather limited resources in the Persistency Framework X

4 CORAL DB Foundation Physics Applications common or experiment specific COOL Validity Intervals, Conditions Data POOL Object Storage, Meta Data and Collections, File Catalogs CORAL RDBMS Access, Indirection, Authentication/Authorisation Computing Services Files, Databases, Grid Services 3

5 SQL-free API Example 1: Table creation coral::ischema& schema = session.nominalschema(); coral::tabledescription tabledescription; tabledescription.setname( T_t ); tabledescription.insertcolumn( I, long long ); tabledescription.insertcolumn( X, double ); schema.createtable( tabledescription); Oracle MySQL CREATE TABLE T_t ( I NUMBER(20), X BINARY_DOUBLE) CREATE TABLE T_t ( I BIGINT, X DOUBLE PRECISION) X

6 Vendor independent SQL-free API Example 2: Issuing a query coral::itable& table = schema.tablehandle( T_t ); coral::iquery* query = table.newquery(); query->addtooutputlist( X ); query->addtoorderlist( I ); query->limitreturnedrows( 5 ); coral::icursor& cursor = query->execute(); Oracle MySQL SELECT * FROM ( SELECT X FROM T_t ORDER BY I ) WHERE ROWNUM < 6 SELECT X FROM T_t ORDER BY I LIMIT 5 4

7 CORAL - Software Components Monitoring Service implementation Client Software CORAL Interfaces (C++ abstract classes user-level API) CORAL C++ types (Row buffers, BLOBs, Date, TimeStamp,...) RDBMS Implementation RDBMS Implementation (mysql) (frontier) RDBMS Implementation RDBMS Implementation (sqlite) (oracle) Authentication Service (environment) Authentication Service (xml) Authentication Service (lfc) Lookup Service (xml) Connection Service Pools, Retry, Failover Lookup Service (lfc) Common Implementation developer-level interfaces Relational Service implementation Plug-in libraries, loaded at run-time, interacting only through the interfaces 5

8 Supported Database Back-ends Oracle - 10g R2 based on C level interface (OCI) performance and flexibility in compiler choice all API concepts / optimisations supported natively bind variables, bulk operations, session pooling MySQL - V5 (and 4) small to medium sized services standalone development set-ups SQLight - V3.4 server-less, file based access popular transport/replication medium for read-only data FroNTier/SQUID - caching layer read-only access to Oracle via http multi-level caching between server and client to avoid network and server latencies for repeated queries 6

9 Applications using CORAL LCG Persistency Framework components POOL and COOL fully replaced direct database dependencies with CORAL POOL File catalogs POOL Event collections COOL conditions database CORAL is also used directly ATLAS: detector geometry several online CMS: conditions data, POPCON framework LHCb: online applications Astrophysics: LSST project is evaluating CORAL for storage of large volume image data 7

10 Building reliable Applications and Services with CORAL The reliability of a composite service depends crucially on core services (eg DB server) but can easily exceed those if a valid retry and fail-over mechanisms are used Unfortunately this often happens only late in the application development or is only incomplete Many different retry and fail-over strategies complicate deployment of db apps CORAL implements a consistent retry and fail-over strategy which can be parametrised according to the application needs parameters are kept in one place (eg service look-up file or DB catalog) This does not completely eliminate the need for specialised handling of error conditions during data manipulation but covers the most frequent problems to obtain a database connection (eg on a overloaded or temporarily unavailable DB server) without loosing the execution state of an application/service X

11 Database Authentication Different requirements in different areas of physics computing Online databases - controlled environment, few applications,users and roles User/password pairs still manageable Integration between online and offline accounts is an issue Offline data production - reconstruction/simulation More users (and roles), apps environment shared with grid DB passwords still widely used, but with security issues Grid jobs - reprocessing, end user analysis Large user community, spreading many organisation boundaries User/password pairs can not be shipped with job to a worker node be maintained for a large virtual organisation Certificates (X.509 proxy cert s) are used by scientific grids support from commercial vendors still missing CORAL support the above via a set of plug-ins coupling to grid services as required X

12 Lookup & Connection Service Problem: DB clients need to obtain physical DB connection parameters (aka connect string ) avoids hard-code credentials, handles DB replica selection, connection retry and fail-over in case of problems Several CORAL plug-ins can be selected at runtime to handle the above in different computing environments via shell environment (assumes controlled machine access) via XML files (assumes controlled config file access) via LFC catalog (certificates protected DB access credentials), VOMS integration for authorisation CORAL maintains a client side pool of connections network connection is shared by several logical DB sessions authentication overhead is minimised 8

13 Remaining Issues - Access Security Oracle server does not support grid authentication (X.509 proxy certs) and grid authorisation (eg VOMS groups) CORAL implemented a secure grid enabled store for database credentials (db user/pw pairs) based on LFC as a workaround being picked up slowly additional dependency on LFC service Many physics application still exposed to possible security related outages, which severely impact on production disclosed user/password pairs vulnerabilities/exploits of Oracle servers exposed to the internet Need progressively to move to grid certificates for external connections apart from applications for which the password distribution (and change - after eg an password exposure) is well controlled minimise external exposure of Oracle server ports eg by moving CORAL based applications to a secure protocol 9

14 Secure External Connections Server protocol is split into control path: connection requests, coral client requests data path: server response eg query result sets For external connections use GSI or SSL for control path client certificate required to connect from outside the site network data path can stay open (at least for HEP) credentials will be evaluated once per session VOMS groups from certificate define database user and privileges Real database user and password used only by CORAL server Online environments may continue with open control path coral://some-online-database (LAN connection) plus existing coral authentication file corals://some-offline-database (WAN connection) no further information apart from certificate required 10

15 Cacheable Network Protocol Several experiments have introduced caching layers to increase scalability by exploiting many identical queries (r/o) CMS: FroNTier plug-in to CORAL ATLAS online: caching MySQL proxy Propose to move caching layer in front of the CORAL server expose caching relevant data in network protocol Goal: allow to create a generic CORAL proxy cache (main user initially ATLAS online) allow to multiplex connections in a hierarchical set of cache servers CoralProxyServer under development by ATLAS contributors 11

16 Efficient Database Server Use Physics apps tend to use databases with thousands of concurrent connections/sessions rather limited data traffic on each connection per time unit Result Oracle clusters manages very many idle processes waste of server memory and CPU cycles, which could be used for caching database blocks and executing queries Need to bundle connections in front of the database server the existing Connection Service component does that, but is limited to pool only connection in the hosting process need to move this component closer to the database server 12

17 Remaining Issues - Client s/w Distribution CORAL clients link (at runtime) against the specific database access libraries eg Oracle instant client, MySQL client Even though an agreement exists with Oracle to distribute the client software within LCG configuration management issues because of compiler dependencies (eg for OCCI) risk of failing to properly implement export constraints requested by the Oracle license Both could be avoided/reduced by making the CORAL client fully independent of any vendor provided client s/w X

18 Stateless vs Stateful Database connections are intrinsically stateful Transaction context spans many operations Bulk queries/inserts are spread over several client server communications Authentication and authorisation state is expensive to obtain and dominates CPU use of many existing services Stateless connection model would lead to many repeated operations to recover existing state problems with lifetime management of transaction/query/ security related data in the server if client programs fail CORAL server does not keep state beyond transaction boundaries several servers can run in parallel (eg DNS load balanced) servers failure looses only the state of uncommitted transactions 13

19 Reuse in the CORAL Server User Code CORAL API Connection Pool User Code CORAL API Connection Pool new component Oracle Plug-in Oracle Client Network Proxy Plug-in CORAL Protocol Oracle Protocol Oracle Protocol DB Server 14 Coral Server Coral Server Coral Server Connection Pool Connection Pool Connection Pool Oracle Plug-in Oracle Plug-in Oracle Plug-in Oracle Client Oracle Client Oracle Client

20 Reuse in the CORAL Server User Code CORAL API Connection Pool User Code CORAL API Connection Pool new component Oracle Plug-in Oracle Client Network Proxy Plug-in CORAL Protocol ProxyCache ProxyCache ProxyCache Oracle Protocol Oracle Protocol DB Server 14 Coral Server Coral Server Coral Server Connection Pool Connection Pool Connection Pool Oracle Plug-in Oracle Plug-in Oracle Plug-in Oracle Client Oracle Client Oracle Client

21 Summary CORAL is widely used as foundation for physics online and offline database applications By introducing a CORAL server in front of the Oracle/MySQL databases we could remove several of the remaining deployment issues CORAL wire protocol is open and designed to facilitate external caching proxies The development profits from significant reuse of existing components and which are integrated adiabatically with minimal impact on existing code. Production deployment planned for end of this year Read-only, caching prototype earlier under test with ATLAS online now 15

22 Other CORAL Features Python access to CORAL user level API Same semantics as C++ components, but maintaining Python look-and-feel Generic database copy tool copy full DBs or slices between CORAL supported back-ends Several developments in cooperation with RRCAT, Indore (India) X

LCG Conditions Database Project

LCG Conditions Database Project Computing in High Energy and Nuclear Physics (CHEP 2006) TIFR, Mumbai, 13 Feb 2006 LCG Conditions Database Project COOL Development and Deployment: Status and Plans On behalf of the COOL team (A.V., D.Front,

More information

The LCG 3D Project. Maria Girone, CERN. The 23rd Open Grid Forum - OGF23 4th June 2008, Barcelona. CERN IT Department CH-1211 Genève 23 Switzerland

The LCG 3D Project. Maria Girone, CERN. The 23rd Open Grid Forum - OGF23 4th June 2008, Barcelona. CERN IT Department CH-1211 Genève 23 Switzerland The LCG 3D Project Maria Girone, CERN The rd Open Grid Forum - OGF 4th June 2008, Barcelona Outline Introduction The Distributed Database (3D) Project Streams Replication Technology and Performance Availability

More information

Access to ATLAS Geometry and Conditions Databases

Access to ATLAS Geometry and Conditions Databases Access to ATLAS Geometry and Conditions Databases Vakho Tsulaia University of Pittsburgh ATLAS South Caucasus Software/Computing Workshop & Tutorial Tbilisi, 2010 10 26 Outline Introduction to ATLAS Geometry

More information

Database monitoring and service validation. Dirk Duellmann CERN IT/PSS and 3D

Database monitoring and service validation. Dirk Duellmann CERN IT/PSS and 3D Database monitoring and service validation Dirk Duellmann CERN IT/PSS and 3D http://lcg3d.cern.ch LCG Database Deployment Plan After October 05 workshop a database deployment plan has been presented to

More information

Remote power and console management in large datacenters

Remote power and console management in large datacenters Remote power and console management in large datacenters A Horváth IT department, CERN, CH-1211 Genève 23, Switzerland E-mail: Andras.Horvath@cern.ch Abstract. Today s datacenters are often built of a

More information

Building a Scalable Architecture for Web Apps - Part I (Lessons Directi)

Building a Scalable Architecture for Web Apps - Part I (Lessons Directi) Intelligent People. Uncommon Ideas. Building a Scalable Architecture for Web Apps - Part I (Lessons Learned @ Directi) By Bhavin Turakhia CEO, Directi (http://www.directi.com http://wiki.directi.com http://careers.directi.com)

More information

ELFms industrialisation plans

ELFms industrialisation plans ELFms industrialisation plans CERN openlab workshop 13 June 2005 German Cancio CERN IT/FIO http://cern.ch/elfms ELFms industrialisation plans, 13/6/05 Outline Background What is ELFms Collaboration with

More information

AMGA metadata catalogue system

AMGA metadata catalogue system AMGA metadata catalogue system Hurng-Chun Lee ACGrid School, Hanoi, Vietnam www.eu-egee.org EGEE and glite are registered trademarks Outline AMGA overview AMGA Background and Motivation for AMGA Interface,

More information

Active Server Pages Architecture

Active Server Pages Architecture Active Server Pages Architecture Li Yi South Bank University Contents 1. Introduction... 2 1.1 Host-based databases... 2 1.2 Client/server databases... 2 1.3 Web databases... 3 2. Active Server Pages...

More information

Technical Brief. A Checklist for Every API Call. Managing the Complete API Lifecycle

Technical Brief. A Checklist for Every API Call. Managing the Complete API Lifecycle Technical Brief A Checklist for Table of Contents Introduction: The API Lifecycle 2 3 Security professionals API developers Operations engineers API product or business owners Apigee Edge 7 A Checklist

More information

The ATLAS Production System

The ATLAS Production System The ATLAS MC and Data Rodney Walker Ludwig Maximilians Universität Munich 2nd Feb, 2009 / DESY Computing Seminar Outline 1 Monte Carlo Production Data 2 3 MC Production Data MC Production Data Group and

More information

Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science

Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science Evolution of the ATLAS PanDA Workload Management System for Exascale Computational Science T. Maeno, K. De, A. Klimentov, P. Nilsson, D. Oleynik, S. Panitkin, A. Petrosyan, J. Schovancova, A. Vaniachine,

More information

<Insert Picture Here> MySQL Cluster What are we working on

<Insert Picture Here> MySQL Cluster What are we working on MySQL Cluster What are we working on Mario Beck Principal Consultant The following is intended to outline our general product direction. It is intended for information purposes only,

More information

LHCb Distributed Conditions Database

LHCb Distributed Conditions Database LHCb Distributed Conditions Database Marco Clemencic E-mail: marco.clemencic@cern.ch Abstract. The LHCb Conditions Database project provides the necessary tools to handle non-event time-varying data. The

More information

Configuring the Oracle Network Environment. Copyright 2009, Oracle. All rights reserved.

Configuring the Oracle Network Environment. Copyright 2009, Oracle. All rights reserved. Configuring the Oracle Network Environment Objectives After completing this lesson, you should be able to: Use Enterprise Manager to: Create additional listeners Create Oracle Net Service aliases Configure

More information

Datacenter replication solution with quasardb

Datacenter replication solution with quasardb Datacenter replication solution with quasardb Technical positioning paper April 2017 Release v1.3 www.quasardb.net Contact: sales@quasardb.net Quasardb A datacenter survival guide quasardb INTRODUCTION

More information

Efficient HTTP based I/O on very large datasets for high performance computing with the Libdavix library

Efficient HTTP based I/O on very large datasets for high performance computing with the Libdavix library Efficient HTTP based I/O on very large datasets for high performance computing with the Libdavix library Authors Devresse Adrien (CERN) Fabrizio Furano (CERN) Typical HPC architecture Computing Cluster

More information

TOPLink for WebLogic. Whitepaper. The Challenge: The Solution:

TOPLink for WebLogic. Whitepaper. The Challenge: The Solution: Whitepaper The Challenge: Enterprise JavaBeans (EJB) represents a new standard in enterprise computing: a component-based architecture for developing and deploying distributed object-oriented applications

More information

THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES

THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES 1 THE ATLAS DISTRIBUTED DATA MANAGEMENT SYSTEM & DATABASES Vincent Garonne, Mario Lassnig, Martin Barisits, Thomas Beermann, Ralph Vigne, Cedric Serfon Vincent.Garonne@cern.ch ph-adp-ddm-lab@cern.ch XLDB

More information

CouchDB-based system for data management in a Grid environment Implementation and Experience

CouchDB-based system for data management in a Grid environment Implementation and Experience CouchDB-based system for data management in a Grid environment Implementation and Experience Hassen Riahi IT/SDC, CERN Outline Context Problematic and strategy System architecture Integration and deployment

More information

ZENworks Mobile Workspace High Availability Environments. September 2017

ZENworks Mobile Workspace High Availability Environments. September 2017 ZENworks Mobile Workspace High Availability Environments September 2017 Legal Notice For information about legal notices, trademarks, disclaimers, warranties, export and other use restrictions, U.S. Government

More information

Using the vrealize Orchestrator Operations Client. vrealize Orchestrator 7.5

Using the vrealize Orchestrator Operations Client. vrealize Orchestrator 7.5 Using the vrealize Orchestrator Operations Client vrealize Orchestrator 7.5 You can find the most up-to-date technical documentation on the VMware website at: https://docs.vmware.com/ If you have comments

More information

BlackBerry Enterprise Server for IBM Lotus Domino Version: 5.0. Administration Guide

BlackBerry Enterprise Server for IBM Lotus Domino Version: 5.0. Administration Guide BlackBerry Enterprise Server for IBM Lotus Domino Version: 5.0 Administration Guide SWDT487521-636611-0528041049-001 Contents 1 Overview: BlackBerry Enterprise Server... 21 Getting started in your BlackBerry

More information

ForeScout Open Integration Module: Data Exchange Plugin

ForeScout Open Integration Module: Data Exchange Plugin ForeScout Open Integration Module: Data Exchange Plugin Version 3.2.0 Table of Contents About the Data Exchange Plugin... 4 Requirements... 4 CounterACT Software Requirements... 4 Connectivity Requirements...

More information

Heckaton. SQL Server's Memory Optimized OLTP Engine

Heckaton. SQL Server's Memory Optimized OLTP Engine Heckaton SQL Server's Memory Optimized OLTP Engine Agenda Introduction to Hekaton Design Consideration High Level Architecture Storage and Indexing Query Processing Transaction Management Transaction Durability

More information

Database Assessment for PDMS

Database Assessment for PDMS Database Assessment for PDMS Abhishek Gaurav, Nayden Markatchev, Philip Rizk and Rob Simmonds Grid Research Centre, University of Calgary. http://grid.ucalgary.ca 1 Introduction This document describes

More information

Service Mesh and Microservices Networking

Service Mesh and Microservices Networking Service Mesh and Microservices Networking WHITEPAPER Service mesh and microservice networking As organizations adopt cloud infrastructure, there is a concurrent change in application architectures towards

More information

Dispatcher. Phoenix. Dispatcher Phoenix Enterprise White Paper Version 0.2

Dispatcher. Phoenix. Dispatcher Phoenix Enterprise White Paper Version 0.2 Dispatcher Phoenix Dispatcher Phoenix Enterprise CONTENTS Introduction... 3 Terminology... 4 Planning & Considerations... 5 Security Features... 9 Enterprise Features... 10 Cluster Overview... 11 Deployment

More information

WebSphere Application Server, Version 5. What s New?

WebSphere Application Server, Version 5. What s New? WebSphere Application Server, Version 5 What s New? 1 WebSphere Application Server, V5 represents a continuation of the evolution to a single, integrated, cost effective, Web services-enabled, J2EE server

More information

Using the Cisco ACE Application Control Engine Application Switches with the Cisco ACE XML Gateway

Using the Cisco ACE Application Control Engine Application Switches with the Cisco ACE XML Gateway Using the Cisco ACE Application Control Engine Application Switches with the Cisco ACE XML Gateway Applying Application Delivery Technology to Web Services Overview The Cisco ACE XML Gateway is the newest

More information

ebay Marketplace Architecture

ebay Marketplace Architecture ebay Marketplace Architecture Architectural Strategies, Patterns, and Forces Randy Shoup, ebay Distinguished Architect QCon SF 2007 November 9, 2007 What we re up against ebay manages Over 248,000,000

More information

Internet2 Meeting September 2005

Internet2 Meeting September 2005 End User Agents: extending the "intelligence" to the edge in Distributed Systems Internet2 Meeting California Institute of Technology 1 OUTLINE (Monitoring Agents using a Large, Integrated s Architecture)

More information

Document Sub Title. Yotpo. Technical Overview 07/18/ Yotpo

Document Sub Title. Yotpo. Technical Overview 07/18/ Yotpo Document Sub Title Yotpo Technical Overview 07/18/2016 2015 Yotpo Contents Introduction... 3 Yotpo Architecture... 4 Yotpo Back Office (or B2B)... 4 Yotpo On-Site Presence... 4 Technologies... 5 Real-Time

More information

Scaling DreamFactory

Scaling DreamFactory Scaling DreamFactory This white paper is designed to provide information to enterprise customers about how to scale a DreamFactory Instance. The sections below talk about horizontal, vertical, and cloud

More information

Contents Overview of the Compression Server White Paper... 5 Business Problem... 7

Contents Overview of the Compression Server White Paper... 5 Business Problem... 7 P6 Professional Compression Server White Paper for On-Premises Version 17 July 2017 Contents Overview of the Compression Server White Paper... 5 Business Problem... 7 P6 Compression Server vs. Citrix...

More information

IBM Spectrum Protect Version Introduction to Data Protection Solutions IBM

IBM Spectrum Protect Version Introduction to Data Protection Solutions IBM IBM Spectrum Protect Version 8.1.2 Introduction to Data Protection Solutions IBM IBM Spectrum Protect Version 8.1.2 Introduction to Data Protection Solutions IBM Note: Before you use this information

More information

EUROPEAN MIDDLEWARE INITIATIVE

EUROPEAN MIDDLEWARE INITIATIVE EUROPEAN MIDDLEWARE INITIATIVE VOMS CORE AND WMS SECURITY ASSESSMENT EMI DOCUMENT Document identifier: EMI-DOC-SA2- VOMS_WMS_Security_Assessment_v1.0.doc Activity: Lead Partner: Document status: Document

More information

TANDBERG Management Suite - Redundancy Configuration and Overview

TANDBERG Management Suite - Redundancy Configuration and Overview Management Suite - Redundancy Configuration and Overview TMS Software version 11.7 TANDBERG D50396 Rev 2.1.1 This document is not to be reproduced in whole or in part without the permission in writing

More information

Evolution of Database Replication Technologies for WLCG

Evolution of Database Replication Technologies for WLCG Evolution of Database Replication Technologies for WLCG Zbigniew Baranowski, Lorena Lobato Pardavila, Marcin Blaszczyk, Gancho Dimitrov, Luca Canali European Organisation for Nuclear Research (CERN), CH-1211

More information

Deploy. A step-by-step guide to successfully deploying your new app with the FileMaker Platform

Deploy. A step-by-step guide to successfully deploying your new app with the FileMaker Platform Deploy A step-by-step guide to successfully deploying your new app with the FileMaker Platform Share your custom app with your team! Now that you ve used the Plan Guide to define your custom app requirements,

More information

IBM Tivoli Storage Manager Version Introduction to Data Protection Solutions IBM

IBM Tivoli Storage Manager Version Introduction to Data Protection Solutions IBM IBM Tivoli Storage Manager Version 7.1.6 Introduction to Data Protection Solutions IBM IBM Tivoli Storage Manager Version 7.1.6 Introduction to Data Protection Solutions IBM Note: Before you use this

More information

MySQL Cluster Student Guide

MySQL Cluster Student Guide MySQL Cluster Student Guide D62018GC11 Edition 1.1 November 2012 D79677 Technical Contributor and Reviewer Mat Keep Editors Aju Kumar Daniel Milne Graphic Designer Seema Bopaiah Publishers Sujatha Nagendra

More information

70-487: Developing Windows Azure and Web Services

70-487: Developing Windows Azure and Web Services 70-487: Developing Windows Azure and Web Services Candidates for this certification are professional developers that use Visual Studio 2015112017 11 and the Microsoft.NET Core Framework 4.5 to design and

More information

WHEN the Large Hadron Collider (LHC) begins operation

WHEN the Large Hadron Collider (LHC) begins operation 2228 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 53, NO. 4, AUGUST 2006 Measurement of the LCG2 and Glite File Catalogue s Performance Craig Munro, Birger Koblitz, Nuno Santos, and Akram Khan Abstract When

More information

PROOF-Condor integration for ATLAS

PROOF-Condor integration for ATLAS PROOF-Condor integration for ATLAS G. Ganis,, J. Iwaszkiewicz, F. Rademakers CERN / PH-SFT M. Livny, B. Mellado, Neng Xu,, Sau Lan Wu University Of Wisconsin Condor Week, Madison, 29 Apr 2 May 2008 Outline

More information

Setting up Microsoft Exchange Server 2016 with Avi

Setting up Microsoft Exchange Server 2016 with Avi Page 1 of 14 Setting up Microsoft Exchange Server 2016 with Avi Networks view online What is Exchange Server 2016 Microsoft Exchange Server 2016 is an e-mail server solution, with calendar and contact

More information

Example Azure Implementation for Government Agencies. Indirect tax-filing system. By Alok Jain Azure Customer Advisory Team (AzureCAT)

Example Azure Implementation for Government Agencies. Indirect tax-filing system. By Alok Jain Azure Customer Advisory Team (AzureCAT) Example Azure Implementation for Government Agencies Indirect tax-filing system By Alok Jain Azure Customer Advisory Team (AzureCAT) June 2018 Example Azure Implementation for Government Agencies Contents

More information

DEPLOYMENT GUIDE DEPLOYING F5 WITH ORACLE ACCESS MANAGER

DEPLOYMENT GUIDE DEPLOYING F5 WITH ORACLE ACCESS MANAGER DEPLOYMENT GUIDE DEPLOYING F5 WITH ORACLE ACCESS MANAGER Table of Contents Table of Contents Introducing the F5 and Oracle Access Manager configuration Prerequisites and configuration notes... 1 Configuration

More information

Architecting C++ apps

Architecting C++ apps Architecting C++ apps with a multi-device application platform John JT Thomas Director of Product Management jt@embarcadero.com @FireMonkeyPM blogs.embarcadero.com/jtembarcadero/ What is a multi-device

More information

Kubernetes Integration with Virtuozzo Storage

Kubernetes Integration with Virtuozzo Storage Kubernetes Integration with Virtuozzo Storage A Technical OCTOBER, 2017 2017 Virtuozzo. All rights reserved. 1 Application Container Storage Application containers appear to be the perfect tool for supporting

More information

Metadaten Workshop 26./27. März 2007 Göttingen. Chimera. a new grid enabled name-space service. Martin Radicke. Tigran Mkrtchyan

Metadaten Workshop 26./27. März 2007 Göttingen. Chimera. a new grid enabled name-space service. Martin Radicke. Tigran Mkrtchyan Metadaten Workshop 26./27. März Chimera a new grid enabled name-space service What is Chimera? a new namespace provider provides a simulated filesystem with additional metadata fast, scalable and based

More information

1 Modular architecture

1 Modular architecture 1 Modular architecture UI customization IIS ID assignment Authorizer selection HTML/CSS/JS HTML/CSS/JS skin skin API User module Admin module Attribute validation Resource assignment Escalation / delegation

More information

Comprehensive Guide to Evaluating Event Stream Processing Engines

Comprehensive Guide to Evaluating Event Stream Processing Engines Comprehensive Guide to Evaluating Event Stream Processing Engines i Copyright 2006 Coral8, Inc. All rights reserved worldwide. Worldwide Headquarters: Coral8, Inc. 82 Pioneer Way, Suite 106 Mountain View,

More information

Bookkeeping and submission tools prototype. L. Tomassetti on behalf of distributed computing group

Bookkeeping and submission tools prototype. L. Tomassetti on behalf of distributed computing group Bookkeeping and submission tools prototype L. Tomassetti on behalf of distributed computing group Outline General Overview Bookkeeping database Submission tools (for simulation productions) Framework Design

More information

CIT 668: System Architecture. Amazon Web Services

CIT 668: System Architecture. Amazon Web Services CIT 668: System Architecture Amazon Web Services Topics 1. AWS Global Infrastructure 2. Foundation Services 1. Compute 2. Storage 3. Database 4. Network 3. AWS Economics Amazon Services Architecture Regions

More information

AD FS v3. Deployment Guide

AD FS v3. Deployment Guide Deployment Guide UPDATED: 15 November 2017 Copyright Notices Copyright 2002-2017 KEMP Technologies, Inc. All rights reserved. KEMP Technologies and the KEMP Technologies logo are registered trademarks

More information

Vlad Vinogradsky

Vlad Vinogradsky Vlad Vinogradsky vladvino@microsoft.com http://twitter.com/vladvino Commercially available cloud platform offering Billing starts on 02/01/2010 A set of cloud computing services Services can be used together

More information

DATABASE SYSTEMS. Introduction to MySQL. Database System Course, 2018

DATABASE SYSTEMS. Introduction to MySQL. Database System Course, 2018 DATABASE SYSTEMS Introduction to MySQL Database System Course, 2018 CAUTION! *This class is NOT a recitation* We will NOT discuss the course material relevant to the exam and homework assignment We have

More information

A RESTful Java Framework for Asynchronous High-Speed Ingest

A RESTful Java Framework for Asynchronous High-Speed Ingest A RESTful Java Framework for Asynchronous High-Speed Ingest Pablo Silberkasten Jean De Lavarene Kuassi Mensah JDBC Product Development October 5, 2017 3 Safe Harbor Statement The following is intended

More information

Event cataloguing and other database applications in ATLAS

Event cataloguing and other database applications in ATLAS Event cataloguing and other database applications in ATLAS Dario Barberis Genoa University/INFN 1 Topics Event cataloguing: the new EventIndex DB Database usage by ATLAS in LHC Run2 PCD WS Ideas for a

More information

DATABASE SYSTEMS. Introduction to MySQL. Database System Course, 2016

DATABASE SYSTEMS. Introduction to MySQL. Database System Course, 2016 DATABASE SYSTEMS Introduction to MySQL Database System Course, 2016 AGENDA FOR TODAY Administration Database Architecture on the web Database history in a brief Databases today MySQL What is it How to

More information

Benchmarking the ATLAS software through the Kit Validation engine

Benchmarking the ATLAS software through the Kit Validation engine Benchmarking the ATLAS software through the Kit Validation engine Alessandro De Salvo (1), Franco Brasolin (2) (1) Istituto Nazionale di Fisica Nucleare, Sezione di Roma, (2) Istituto Nazionale di Fisica

More information

Security in the CernVM File System and the Frontier Distributed Database Caching System

Security in the CernVM File System and the Frontier Distributed Database Caching System Security in the CernVM File System and the Frontier Distributed Database Caching System D Dykstra 1 and J Blomer 2 1 Scientific Computing Division, Fermilab, Batavia, IL 60510, USA 2 PH-SFT Department,

More information

Storage Resource Sharing with CASTOR.

Storage Resource Sharing with CASTOR. Storage Resource Sharing with CASTOR Olof Barring, Benjamin Couturier, Jean-Damien Durand, Emil Knezo, Sebastien Ponce (CERN) Vitali Motyakov (IHEP) ben.couturier@cern.ch 16/4/2004 Storage Resource Sharing

More information

Distributed File Systems II

Distributed File Systems II Distributed File Systems II To do q Very-large scale: Google FS, Hadoop FS, BigTable q Next time: Naming things GFS A radically new environment NFS, etc. Independence Small Scale Variety of workloads Cooperation

More information

ebay s Architectural Principles

ebay s Architectural Principles ebay s Architectural Principles Architectural Strategies, Patterns, and Forces for Scaling a Large ecommerce Site Randy Shoup ebay Distinguished Architect QCon London 2008 March 14, 2008 What we re up

More information

Atlas Technology White Paper

Atlas Technology White Paper Atlas Technology White Paper 2017 Bomgar Corporation. All rights reserved worldwide. BOMGAR and the BOMGAR logo are trademarks of Bomgar Corporation; other trademarks shown are the property of their respective

More information

White Paper. Major Performance Tuning Considerations for Weblogic Server

White Paper. Major Performance Tuning Considerations for Weblogic Server White Paper Major Performance Tuning Considerations for Weblogic Server Table of Contents Introduction and Background Information... 2 Understanding the Performance Objectives... 3 Measuring your Performance

More information

High Noon at AWS. ~ Amazon MySQL RDS versus Tungsten Clustering running MySQL on AWS EC2

High Noon at AWS. ~ Amazon MySQL RDS versus Tungsten Clustering running MySQL on AWS EC2 High Noon at AWS ~ Amazon MySQL RDS versus Tungsten Clustering running MySQL on AWS EC2 Introduction Amazon Web Services (AWS) are gaining popularity, and for good reasons. The Amazon Relational Database

More information

What s New in MySQL 5.7 Geir Høydalsvik, Sr. Director, MySQL Engineering. Copyright 2015, Oracle and/or its affiliates. All rights reserved.

What s New in MySQL 5.7 Geir Høydalsvik, Sr. Director, MySQL Engineering. Copyright 2015, Oracle and/or its affiliates. All rights reserved. What s New in MySQL 5.7 Geir Høydalsvik, Sr. Director, MySQL Engineering Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes

More information

BEAAquaLogic. Service Bus. Interoperability With EJB Transport

BEAAquaLogic. Service Bus. Interoperability With EJB Transport BEAAquaLogic Service Bus Interoperability With EJB Transport Version 3.0 Revised: February 2008 Contents EJB Transport Introduction...........................................................1-1 Invoking

More information

Boundary control : Access Controls: An access control mechanism processes users request for resources in three steps: Identification:

Boundary control : Access Controls: An access control mechanism processes users request for resources in three steps: Identification: Application control : Boundary control : Access Controls: These controls restrict use of computer system resources to authorized users, limit the actions authorized users can taker with these resources,

More information

Dynamic Server Allocation in a Real-Life Deployable Communications Architecture for

Dynamic Server Allocation in a Real-Life Deployable Communications Architecture for Dynamic Server Allocation in a Real-Life Deployable Communications Architecture for Networked Games Peter Quax Bart Cornelissen Jeroen Dierckx Gert Vansichem Wim Lamotte Hasselt University & Androme NV

More information

ProxySQL Tutorial. With a GPL license! High Performance & High Availability Proxy for MySQL. Santa Clara, California April 23th 25th, 2018

ProxySQL Tutorial. With a GPL license! High Performance & High Availability Proxy for MySQL. Santa Clara, California April 23th 25th, 2018 ProxySQL Tutorial High Performance & High Availability Proxy for MySQL With a GPL license! Santa Clara, California April 23th 25th, 2018 Who we are René Cannaò ProxySQL Founder Derek Downey Director of

More information

Implementing the Twelve-Factor App Methodology for Developing Cloud- Native Applications

Implementing the Twelve-Factor App Methodology for Developing Cloud- Native Applications Implementing the Twelve-Factor App Methodology for Developing Cloud- Native Applications By, Janakiram MSV Executive Summary Application development has gone through a fundamental shift in the recent past.

More information

Jyotheswar Kuricheti

Jyotheswar Kuricheti Jyotheswar Kuricheti 1 Agenda: 1. Performance Tuning Overview 2. Identify Bottlenecks 3. Optimizing at different levels : Target Source Mapping Session System 2 3 Performance Tuning Overview: 4 What is

More information

FUJITSU Cloud Service K5 - API Management Service Description

FUJITSU Cloud Service K5 - API Management Service Description FUJITSU Cloud Service K5 - API Management Service Description March 22, 2018 1. API Management Service Overview API Management Service is built on Apigee Edge, an integrated API platform product provided

More information

P6 Compression Server White Paper Release 8.2 December 2011 Copyright Oracle Primavera P6 Compression Server White Paper Copyright 2005, 2011, Oracle and/or its affiliates. All rights reserved. Oracle

More information

What every DBA needs to know about JDBC connection pools Bridging the language barrier between DBA and Middleware Administrators

What every DBA needs to know about JDBC connection pools Bridging the language barrier between DBA and Middleware Administrators Presented at What every DBA needs to know about JDBC connection pools Bridging the language barrier between DBA and Middleware Administrators Jacco H. Landlust Platform Architect Director Oracle Consulting

More information

1 Copyright 2011, Oracle and/or its affiliates. All rights reserved.

1 Copyright 2011, Oracle and/or its affiliates. All rights reserved. 1 Copyright 2011, Oracle and/or its affiliates. All rights Getting Started with Oracle and.net Christian Shay Principal Product Manager, Oracle 2 Copyright 2011, Oracle and/or its affiliates. All rights

More information

Comparing SQL and NOSQL databases

Comparing SQL and NOSQL databases COSC 6397 Big Data Analytics Data Formats (II) HBase Edgar Gabriel Spring 2014 Comparing SQL and NOSQL databases Types Development History Data Storage Model SQL One type (SQL database) with minor variations

More information

Short Summary of DB2 V4 Through V6 Changes

Short Summary of DB2 V4 Through V6 Changes IN THIS CHAPTER DB2 Version 6 Features DB2 Version 5 Features DB2 Version 4 Features Short Summary of DB2 V4 Through V6 Changes This appendix provides short checklists of features for the most recent versions

More information

NoSQL systems: introduction and data models. Riccardo Torlone Università Roma Tre

NoSQL systems: introduction and data models. Riccardo Torlone Università Roma Tre NoSQL systems: introduction and data models Riccardo Torlone Università Roma Tre Leveraging the NoSQL boom 2 Why NoSQL? In the last fourty years relational databases have been the default choice for serious

More information

Primavera Compression Server 5.0 Service Pack 1 Concept and Performance Results

Primavera Compression Server 5.0 Service Pack 1 Concept and Performance Results - 1 - Primavera Compression Server 5.0 Service Pack 1 Concept and Performance Results 1. Business Problem The current Project Management application is a fat client. By fat client we mean that most of

More information

Equitrac Office and Express DCE High Availability White Paper

Equitrac Office and Express DCE High Availability White Paper Office and Express DCE High Availability White Paper 2 Summary............................................................... 3 Introduction............................................................

More information

Cloud-Native Applications. Copyright 2017 Pivotal Software, Inc. All rights Reserved. Version 1.0

Cloud-Native Applications. Copyright 2017 Pivotal Software, Inc. All rights Reserved. Version 1.0 Cloud-Native Applications Copyright 2017 Pivotal Software, Inc. All rights Reserved. Version 1.0 Cloud-Native Characteristics Lean Form a hypothesis, build just enough to validate or disprove it. Learn

More information

Worldwide Production Distributed Data Management at the LHC. Brian Bockelman MSST 2010, 4 May 2010

Worldwide Production Distributed Data Management at the LHC. Brian Bockelman MSST 2010, 4 May 2010 Worldwide Production Distributed Data Management at the LHC Brian Bockelman MSST 2010, 4 May 2010 At the LHC http://op-webtools.web.cern.ch/opwebtools/vistar/vistars.php?usr=lhc1 Gratuitous detector pictures:

More information

DEPLOYMENT GUIDE A10 THUNDER ADC FOR EPIC SYSTEMS

DEPLOYMENT GUIDE A10 THUNDER ADC FOR EPIC SYSTEMS DEPLOYMENT GUIDE A10 THUNDER ADC FOR EPIC SYSTEMS OVERVIEW This document shows how an A10 Thunder Series device can be deployed with Epic Electronic Medical Record system. The tested solution is based

More information

Manual Trigger Sql Server 2008 Insert Multiple Rows At Once

Manual Trigger Sql Server 2008 Insert Multiple Rows At Once Manual Trigger Sql Server 2008 Insert Multiple Rows At Once Adding SQL Trigger to update field on INSERT (multiple rows) However, if there are multiple records inserted (as in the user creates several

More information

Scientific data processing at global scale The LHC Computing Grid. fabio hernandez

Scientific data processing at global scale The LHC Computing Grid. fabio hernandez Scientific data processing at global scale The LHC Computing Grid Chengdu (China), July 5th 2011 Who I am 2 Computing science background Working in the field of computing for high-energy physics since

More information

Course 40045A: Microsoft SQL Server for Oracle DBAs

Course 40045A: Microsoft SQL Server for Oracle DBAs Skip to main content Course 40045A: Microsoft SQL Server for Oracle DBAs - Course details Course Outline Module 1: Database and Instance This module provides an understanding of the two major components

More information

Lecture 9: MIMD Architectures

Lecture 9: MIMD Architectures Lecture 9: MIMD Architectures Introduction and classification Symmetric multiprocessors NUMA architecture Clusters Zebo Peng, IDA, LiTH 1 Introduction A set of general purpose processors is connected together.

More information

Microservices Beyond the Hype. SATURN San Diego May 3, 2016 Paulo Merson

Microservices Beyond the Hype. SATURN San Diego May 3, 2016 Paulo Merson Microservices Beyond the Hype SATURN San Diego May 3, 2016 Paulo Merson Our goal Try to define microservice Discuss what you gain and what you lose with microservices 2 Defining Microservice Unfortunately

More information

Introduction. CS3026 Operating Systems Lecture 01

Introduction. CS3026 Operating Systems Lecture 01 Introduction CS3026 Operating Systems Lecture 01 One or more CPUs Device controllers (I/O modules) Memory Bus Operating system? Computer System What is an Operating System An Operating System is a program

More information

Understanding Impact of J2EE Applications On Relational Databases. Dennis Leung, VP Development Oracle9iAS TopLink Oracle Corporation

Understanding Impact of J2EE Applications On Relational Databases. Dennis Leung, VP Development Oracle9iAS TopLink Oracle Corporation Understanding Impact of J2EE Applications On Relational Databases Dennis Leung, VP Development Oracle9iAS TopLink Oracle Corporation J2EE Apps and Relational Data J2EE is one of leading technologies used

More information

ArcGIS Server Components: An Introduction to Server IT

ArcGIS Server Components: An Introduction to Server IT ArcGIS Server Components: An Introduction to Server IT Outline Web Adaptors & Web Server Web Server Certificates Portal Security Settings SQL Server & Management Studio Platform Illustrated: Windows 2012

More information

Oracle Enterprise Manager. 1 Before You Install. System Monitoring Plug-in for Oracle Unified Directory User's Guide Release 1.0

Oracle Enterprise Manager. 1 Before You Install. System Monitoring Plug-in for Oracle Unified Directory User's Guide Release 1.0 Oracle Enterprise Manager System Monitoring Plug-in for Oracle Unified Directory User's Guide Release 1.0 E24476-01 October 2011 The System Monitoring Plug-In for Oracle Unified Directory extends Oracle

More information

Active Endpoints. ActiveVOS Platform Architecture Active Endpoints

Active Endpoints. ActiveVOS Platform Architecture Active Endpoints Active Endpoints ActiveVOS Platform Architecture ActiveVOS Unique process automation platforms to develop, integrate, and deploy business process applications quickly User Experience Easy to learn, use

More information

Comparative evaluation of software tools accessing relational databases from a (real) grid environments

Comparative evaluation of software tools accessing relational databases from a (real) grid environments Comparative evaluation of software tools accessing relational databases from a (real) grid environments Giacinto Donvito, Guido Cuscela, Massimiliano Missiato, Vicenzo Spinoso, Giorgio Maggi INFN-Bari

More information

Database System Concepts and Architecture

Database System Concepts and Architecture CHAPTER 2 Database System Concepts and Architecture Copyright 2017 Ramez Elmasri and Shamkant B. Navathe Slide 2-2 Outline Data Models and Their Categories History of Data Models Schemas, Instances, and

More information