CREAM Computing Element Overview
|
|
- Edward Moore
- 5 years ago
- Views:
Transcription
1 Co-ordination & Harmonisation of Advanced e-infrastructures for Research and Education Data Sharing Research Infrastructures Grant Agreement n CREAM Computing Element Overview Bruce Becker, Coordinator SAGrid (Meraka Institute) Africa Arabia ROC (South) All Hands meeting
2 Outline CREAM-CE Outline Functionality Integration with the batch system Sandbox management Security, Authn/Authz Information provider Accounting system integration Where to get CREAM-CE Installation guide Configuration guide Some customisations 2
3 CREAM CE at a glance A lightweight web-service interface to the LRMS Extension of the Java-Axis servlet running inside the Apache Tomcat container Modular design : CREAM service (tomcat container) BLAH services (Bupdater, Bnotifier) BLAH (Batch Local Ascii Helper) provides integration between queue/scheduer and usage information, and CREAM Resource BDII Torque services (main service usually TORQUE, MAUI, munge) Other services (Mysql, LB locallogger, Globus GridFTP) 3
4 CREAM CE functionality Job Management Direct to CREAM CE or via WMS Using generic CLI client or C++/Java/Python API Possibility of direct staging of input sandbox files glite JDL compliance (with CREAM-specific extensions) Support for batch and parallel jobs (automatic forwarding of job requirements to the batch system) Manual and automatic proxy delegation Job Cancellation, Info, List, Suspend/Resume Output retrieval Possibility (for admin) to disable new submissions Self limiting CREAM behaviour Integration with APEL and DGAS accounting systems 4
5 CREAM CE Architecture 5
6 Integration with the batch system Integration with the batch system is done with BLAH responsible for : interaction with the batch system for job management operations (submit, cancel, etc) Notification of CREAM about job status changes via the BLParser component about state changes Supported batch sytems : TORQUE Sun Grid Engine LSF (Platform Computing) 6
7 Sandbox Management Input Sandbox (ISB): files needed for the job to execute properly Client data push and/or server data pull supported Client data push: files staged from the client (UI) node Server data pull: files retrieved from remote server Output Sandbox (OSB): files produced by the job Server data push and/or client data pull supported Server data push: files uploaded to remote server Client data pull: files retrieved on the client (UI) node by the end user 7
8 Security Authentication implemented via Trustmanager Authorisation implemented by gja (grid Java Authorisation Framework) or ARGUS CREAM CE can have remote administrators: An admin can manage jobs submitted by other users and perform some priveliged operations (enable/disable submissions) DN of the CREAM admin must be listed in the admin list : /etc/grid-security/admin-list 8
9 Authorisation For CREAM-CE service : grid/group mapfiles are read to check if the submitting user is authorised or not glexec is used to check which local user is mapped to the grid user All operations done on behalf of this user are done using sudo with localid returned by glexec For gridftp: Authorisation is managed by LCAS (Local Centre Authorisation Service) Grid/group mapfiles are read to check if the submitting user is authorised or not LCMAPS is used to check which local user is mapped to that grid user To use gjaf, set YAIM variable: USE_ARGUS=no 9
10 Authorisation with ARGUS ARGUS central authorisation service All authorisation policies are defined on the ARGUS box Easy way to maintain control over credentials in a distributed infrastructure Usually 1 ARGUS box per grid. CREAM contacts ARGUS to check if the user is authorised In the case of success, ARGUS returns the local mapped user All operations are done on behalf of the user using sudo To enable ARGUS on the sute, use USE_ARGUS=yes Other relevant YAIM variables : ARGUS_PEPD_ENDPOINTS endpoint of the ARGUS box (e.g. CREAM_PEPC_RESOURCE_ID the ID of the CREAM CE in the ARGUS box 10
11 CREAM GRIS CREAM CE runs a resource BDII publishing information about the CE Both GLUE1 and GLUE2 schema are supported Some information is dynamic and produced by plugins/providers GIP generic information provider see scripts in /var/lib/bdii/gip/plugin and /var/lib/bdii/gip/provider: created by YAIM at configuration time. To query the resource bdii : ldapsearch -h <cream-ce host> -p b <basedn> e.g.: ldapsearch -h grid-ce.chpc.ac.za -p b mds-vo-name=resource,o=grid (GLUE 1) ldapsearch -h grid-ce.chpc.ac.za -p b o=glue (GLUE2) 11
12 Accounting system integration CREAM CE supports natively both DGAS and APEL accounting systems Integration is implemented by files (filled by BLAH) which are parsed by APEL/DGAS sensors Accounting log files are stored in /var/log/cream/accounting 12
13 Example accounting output [.] "timestamp= :58:24" "userdn=/c=it/o=infn/ou=personal Certificate/L=ZA-ITHEMBALABS/CN=Sean Hamilton Thomas Murray" "userfqan=/alice/role=lcgadmin/capability=null" "userfqan=/alice/role=null/capability=null" "userfqan=/alice/lcg1/role=null/capability=null" "ceid=grid-ce.chpc.ac.za:8443/cream-pbs-alice" "jobid=cream " "lrmsid= grid-ce.chpc.ac.za" "localuser=60142" "clientid=cream_ " "timestamp= :58:28" "userdn=/c=it/o=infn/ou=personal Certificate/L=ZA-ITHEMBALABS/CN=Sean Hamilton Thomas Murray" "userfqan=/alice/role=lcgadmin/capability=null" "userfqan=/alice/role=null/capability=null" "userfqan=/alice/lcg1/role=null/capability=null" "ceid=grid-ce.chpc.ac.za:8443/cream-pbs-alice" "jobid=cream " "lrmsid= grid-ce.chpc.ac.za" "localuser=60142" "clientid=cream_ " 13
14 Where to get CREAM CREAM is distributed with EMI - select the relevant repos : SL5/SL6 (x86_64) OS Repos rpm -Uvh rpm -Uvh 64.rpm EPEL rpm -Uvh rpm -Uvh ch.rpm CA repo rpm -Uvh noarch.rpm EMI Repos Rpm -Uvh rch.rpm DISABLE DAG 14
15 Installation prerequisites Host certificate is required in /etc/grid-security/host[cert,key].pem Check: selinux is DISABLED cron and logrotate are installed ntp is installed, configured and running yum install -y ntp chkconfig level 2345 ntpd on ntpdate tick.meraka.csir.co.za /etc/init.d/ntpd restart 15
16 Install metapackages Install required metapackages: yum install yum-protectbase ca-policy-egi-core xml-commons-apis emi-cream-ce nfs-utils Depending on which LRMS you have, install the CREAM module: SGE: yum install emi-sge-utils LSF: yum install emi-lsf-utils TORQUE: If CE is not TORQUE server, or if you have your own version of TORQUE: yum install emi-torque-utils If CE is also TORQUE server: yum install emi-torque-utils emi-torque-server 16
17 Munge configuration Since EMI-1, munge is used for inter-node authentication by TORQUE Verify that munge has been correctly installed (from EPEL repo) #rpm -q munge munge-libs munge el5 munge-libs el5 One one host (e.g. batch server), generate munge key: /usr/sbin/create-munge-key (gets created in /etc/munge permissions 400) Copythe key to every host in tnhe clusters, djusting permissions #chown munge:munge /etc/munge/munge/key Start munge daemon on each WN /etc/init.d/munge start chkconfig munge on 17
18 CREAM CE Configuration Use the examples distributed by YAIM as starting points for configuration files: mkdir -p /opt/glite/yaim/etc/<site-name> cp -r /opt/glite/yaim/examples/* /opt/glite/yaim/etc/<site-name> Files to edit: /opt/glite/yaim/etc/chpc/ -- edgusers.conf -- groups.conf -- site-info.def -- users.conf -- wn-list.conf -- services -- glite-creamce 18
19 Edit the site-info.def file For reference: General configuration variables: BDII_HOST WNLIST GROUPS_CONF USERS_CONF MSQL_PASSWORD Site configuration variables SITE_NAME SITE_ SITE_LAT SITE_LONG Batch-server configuration variables BATCH_LOG_DIR (for Torque/PBS, this must be set to the directory "containing" the server_logs directory; usually /var/torque) BATCH_SERVER JOB_MANAGER BATCH_VERSION WN_LIST 19
20 Edit the CE specific variables LRMS-specific variables MUNGE_KEY_FILE=/etc/munge/munge.key CONFIG_MAUI=yes CREAM-CE variables in services/glite-creamce CEMON_HOST=$CE_HOST CREAM_DB_USER=cream CREAM_DB_PASWORD=SecurePassword BLPARSER_HOST=$CE_HOST BLPARSER_WITH_UPDATER_NOTIFIER=true Full list at nfiguration_variables#torque_server 20
21 Configure the CREAM CE service If CREAM CE is the same host as TORQUE, configure the profiles creamce TORQUE_server TORQUE_utils Else only creamce TORQUE_utils Verify the that the site-info.def file is vaild: /opt/glite/yaim/bin/yaim -v -s /opt/glite/yaim/etc/<site-name>/site-info.def -n creamce -n TORQUE_server -n TORQUE_utils Configure the services /opt/glite/yaim/bin/yaim -s site-info.def -n creamce -n TORQUE_server -n TORQUE_utils 21
22 Checks to be done Verifying the configuration status Check via browser: Check log file to see if CREAM service started succesfully: grep CREAM started! connection with BLParser /var/log/cream/glite-cream-ce.log Test gridftp (with proxy): uberftp grid-ce.chpc.ac.za 220 grid-ce.chpc.ac.za GridFTP Server 6.14 (gcc64, ) [Globus Toolkit 5.2.1] ready. 230 User sagrid014 logged in. UberFTP> Try service-info: glite-ce-service-info grid-ce.chpc.ac.za Interface Version = [2.1] Service Version = [1.13] EMI3 Tutorial Description = [CREAM 2] Started at = [Tue Feb 26 17:20: ] Submission enabled = [YES] Status Service Property = [RUNNING] Check submissinon enabled: = [cemon_url]->[na] glite-ce-allowed-submission grid-ce.chpc.ac.za Job Submission to this CREAM CE is enabled 22
23 Check job submission from UI Cat test.jdl [ ] Executable= /bin/hostname ; Arguments= -f ; StdOutput="output"; StdError ="output"; OutputSandbox="output"; OutputSandboxBaseDestURI="gsiftp://cream-ce.core.wits.ac.za/tmp"; glite-ce-job-submit -a -r cream-ce.core.wits.ac.za:8443/cream-pbs-sagrid test_cream.jdl -bash-3.2$ glite-ce-job-status ****** JobID=[ Status = [REALLY-RUNNING] -bash-3.2$ glite-ce-job-status ****** JobID=[ Status = [DONE-OK] ExitCode = [0] 23
24 Which log files to look at? CREAM log file: /var/log/cream/glite-ce-cream.log /var/log/cream/glite-ce-bupdater.log /var/log/cream/glite-ce-bnotifier.log Tomcat log file: /usr/share/tomcat5/logs/trustmanager.log /usr/share/tomcat5/logs/catalina.out Gridftp log file: /var/log/globus-gridftp.log /var/log/gridftp-session.log 24
25 Processes in execution Check that the following processes are all running : Tomcat Multiple blahpd Bupdater, bnotifier (for new BLAH parser) Logd,interlogd (for LB localogger) Slapd (GRIS) Gridftp mysql 25
26 CREAM CE customisation 1: local disks for Sandboxes By default, sandboxes are created in the mapped user's home directory e.g. /home/sagrid014/home_creamxxx Usually NFS-mounted For performance reasons at large sites, you want to be able to use the local disks instead : NFS/autofs mounts can present a large network overhead or bottlneck Specify some directory on the local disk which will be used for sandboxes: e.g. /scratch This customisation has to be done in the CREAM jobwrapper template - See /etc/glite-ce-cream/jobwrapper.tpl /var/lib/tomcat5/webapps/ce-cream/web-inf/jobwrapper.tpl in EMI-1 26
27 References CREAM-CE System Administrator's guide CREAM-CE Known Issues CREAM User Guide CREAM Troublshotting guide Service Reference card, including network requirements EMI-2 product pages (CREAM) ue-module-1 (CREAM Torque module) Special thanks to Giuseppe Larocca for slides from EMI training in Taipei 27
YAIM Overview. Bruce Becker Meraka Institute. Co-ordination & Harmonisation of Advanced e-infrastructures for Research and Education Data Sharing
Co-ordination & Harmonisation of Advanced e-infrastructures for Research and Education Data Sharing Research Infrastructures Grant Agreement n. 306819 YAIM Overview Bruce Becker Meraka Institute Outline
More informationCE+WN+siteBDII Installation and configuration
The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) CE+WN+siteBDII Installation and configuration Andrea Cortellese (andrea.cortellese@ct.infn.it) National Institute of Nuclear
More informationGrid services. Enabling Grids for E-sciencE. Dusan Vudragovic Scientific Computing Laboratory Institute of Physics Belgrade, Serbia
Grid services Dusan Vudragovic dusan@phy.bg.ac.yu Scientific Computing Laboratory Institute of Physics Belgrade, Serbia Sep. 19, 2008 www.eu-egee.org Set of basic Grid services Job submission/management
More informationEMI Deployment Planning. C. Aiftimiei D. Dongiovanni INFN
EMI Deployment Planning C. Aiftimiei D. Dongiovanni INFN Outline Migrating to EMI: WHY What's new: EMI Overview Products, Platforms, Repos, Dependencies, Support / Release Cycle Migrating to EMI: HOW Admin
More informationYAIM: The glite configuration tool
YAIM: The glite configuration tool Sara Bertocco INFN Padova EMI is partially funded by the European Commission under Grant Agreement RI-261611 28 August 2012 GridKa School YAIM documentation Guide for
More informationglite Grid Services Overview
The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) glite Grid Services Overview Antonio Calanducci INFN Catania Joint GISELA/EPIKH School for Grid Site Administrators Valparaiso,
More informationEMI Componets Installation And Configuration
EMI Componets Installation And Configuration Sara Bertocco INFN Padova - GridKA School EMI is partially funded by the European Commission under Grant Agreement RI-261611 Tour in the EMI site EMI site:
More informationEUROPEAN MIDDLEWARE INITIATIVE
EUROPEAN MIDDLEWARE INITIATIVE VOMS CORE AND WMS SECURITY ASSESSMENT EMI DOCUMENT Document identifier: EMI-DOC-SA2- VOMS_WMS_Security_Assessment_v1.0.doc Activity: Lead Partner: Document status: Document
More informationRelease 0.1. Paolo Andreetto
CREAM G uidedocumentation Release 0.1 Paolo Andreetto Dec 15, 2017 Contents 1 CREAM Functional Description 3 2 CREAM installation 5 2.1 Requirements............................................... 5 2.2
More information30 Nov Dec Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy
Advanced School in High Performance and GRID Computing Concepts and Applications, ICTP, Trieste, Italy Why the Grid? Science is becoming increasingly digital and needs to deal with increasing amounts of
More informationArchitecture Proposal
Nordic Testbed for Wide Area Computing and Data Handling NORDUGRID-TECH-1 19/02/2002 Architecture Proposal M.Ellert, A.Konstantinov, B.Kónya, O.Smirnova, A.Wäänänen Introduction The document describes
More informationArgus Vulnerability Assessment *1
Argus Vulnerability Assessment *1 Manuel Brugnoli and Elisa Heymann Universitat Autònoma de Barcelona June, 2011 Introduction Argus is the glite Authorization Service. It is intended to provide consistent
More informationWLCG Lightweight Sites
WLCG Lightweight Sites Mayank Sharma (IT-DI-LCG) 3/7/18 Document reference 2 WLCG Sites Grid is a diverse environment (Various flavors of CE/Batch/WN/ +various preferred tools by admins for configuration/maintenance)
More informationTroubleshooting Grid authentication from the client side
Troubleshooting Grid authentication from the client side By Adriaan van der Zee RP1 presentation 2009-02-04 Contents The Grid @NIKHEF The project Grid components and interactions X.509 certificates, proxies
More informationg-eclipse A Framework for Accessing Grid Infrastructures Nicholas Loulloudes Trainer, University of Cyprus (loulloudes.n_at_cs.ucy.ac.
g-eclipse A Framework for Accessing Grid Infrastructures Trainer, University of Cyprus (loulloudes.n_at_cs.ucy.ac.cy) EGEE Training the Trainers May 6 th, 2009 Outline Grid Reality The Problem g-eclipse
More informationThe ARC Information System
The ARC Information System Overview of a GLUE2 compliant production system Florido Paganelli, Lund University EGI Community Forum 2012, Munich, 26-30 March 2012 Outline Existing solutions ARC key concepts
More informationThe glite middleware. Presented by John White EGEE-II JRA1 Dep. Manager On behalf of JRA1 Enabling Grids for E-sciencE
The glite middleware Presented by John White EGEE-II JRA1 Dep. Manager On behalf of JRA1 John.White@cern.ch www.eu-egee.org EGEE and glite are registered trademarks Outline glite distributions Software
More informationGrid Authentication and Authorisation Issues. Ákos Frohner at CERN
Grid Authentication and Authorisation Issues Ákos Frohner at CERN Overview Setting the scene: requirements Old style authorisation: DN based gridmap-files Overview of the EDG components VO user management:
More informationI Tier-3 di CMS-Italia: stato e prospettive. Hassen Riahi Claudio Grandi Workshop CCR GRID 2011
I Tier-3 di CMS-Italia: stato e prospettive Claudio Grandi Workshop CCR GRID 2011 Outline INFN Perugia Tier-3 R&D Computing centre: activities, storage and batch system CMS services: bottlenecks and workarounds
More informationSetup Desktop Grids and Bridges. Tutorial. Robert Lovas, MTA SZTAKI
Setup Desktop Grids and Bridges Tutorial Robert Lovas, MTA SZTAKI Outline of the SZDG installation process 1. Installing the base operating system 2. Basic configuration of the operating system 3. Installing
More informationLCG-2 and glite Architecture and components
LCG-2 and glite Architecture and components Author E.Slabospitskaya www.eu-egee.org Outline Enabling Grids for E-sciencE What are LCG-2 and glite? glite Architecture Release 1.0 review What is glite?.
More informationGrid Architectural Models
Grid Architectural Models Computational Grids - A computational Grid aggregates the processing power from a distributed collection of systems - This type of Grid is primarily composed of low powered computers
More informationISTITUTO NAZIONALE DI FISICA NUCLEARE Sezione di Padova
ISTITUTO NAZIONALE DI FISICA NUCLEARE Sezione di Padova INFN/TC-09/03 May 05, 2009 DESIGN AND IMPLEMENTATION OF THE GLITE CREAM JOB MANAGEMENT SERVICE Cristina Aiftimiei, Paolo Andreetto, Sara Bertocco,
More informationGlobus Toolkit Manoj Soni SENG, CDAC. 20 th & 21 th Nov 2008 GGOA Workshop 08 Bangalore
Globus Toolkit 4.0.7 Manoj Soni SENG, CDAC 1 What is Globus Toolkit? The Globus Toolkit is an open source software toolkit used for building Grid systems and applications. It is being developed by the
More informationGridbus Portlets -- USER GUIDE -- GRIDBUS PORTLETS 1 1. GETTING STARTED 2 2. AUTHENTICATION 3 3. WORKING WITH PROJECTS 4
Gridbus Portlets -- USER GUIDE -- www.gridbus.org/broker GRIDBUS PORTLETS 1 1. GETTING STARTED 2 1.1. PREREQUISITES: 2 1.2. INSTALLATION: 2 2. AUTHENTICATION 3 3. WORKING WITH PROJECTS 4 3.1. CREATING
More informationglite UI Installation
The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) glite UI Installation Antonio Calanducci INFN Catania Joint GISELA/EPIKH School for Grid Site Administrators Valparaiso, Chile,
More informationInformation and monitoring
Information and monitoring Information is essential Application database Certificate Certificate Authorised users directory Certificate Certificate Grid tools Researcher Certificate Policies Information
More informationArchitecture of the WMS
Architecture of the WMS Dr. Giuliano Taffoni INFORMATION SYSTEMS UNIT Outline This presentation will cover the following arguments: Overview of WMS Architecture Job Description Language Overview WMProxy
More informationEGI-InSPIRE. ARC-CE IPv6 TESTBED. Barbara Krašovec, Jure Kranjc ARNES. EGI-InSPIRE RI
EGI-InSPIRE ARC-CE IPv6 TESTBED Barbara Krašovec, Jure Kranjc ARNES Why ARC-CE over IPv6? - IPv4 exhaustion - On Friday 14 th, RIPE NCC has announced that the last /8 is being distributed from available
More informationglite Middleware Usage
glite Middleware Usage Dusan Vudragovic dusan@phy.bg.ac.yu Scientific Computing Laboratory Institute of Physics Belgrade, Serbia Nov. 18, 2008 www.eu-egee.org EGEE and glite are registered trademarks Usage
More informationOverview of HEP software & LCG from the openlab perspective
Overview of HEP software & LCG from the openlab perspective Andreas Unterkircher, CERN openlab February 2005 Andreas Unterkircher 1 Contents 1. Opencluster overview 2. High Energy Physics (HEP) software
More informationThe EU DataGrid Fabric Management
The EU DataGrid Fabric Management The European DataGrid Project Team http://www.eudatagrid.org DataGrid is a project funded by the European Union Grid Tutorial 4/3/2004 n 1 EDG Tutorial Overview Workload
More informationThe glite middleware. Ariel Garcia KIT
The glite middleware Ariel Garcia KIT Overview Background The glite subsystems overview Security Information system Job management Data management Some (my) answers to your questions and random rumblings
More informationLinux Administration
Linux Administration This course will cover all aspects of Linux Certification. At the end of the course delegates will have the skills required to administer a Linux System. It is designed for professionals
More informationSouth African Science Gateways
Co-ordination & Harmonisation of Advanced e-infrastructures for Research and Education Data Sharing Research Infrastructures Grant Agreement n. 306819 South African Science Gateways Bruce Becker, Coordinator,
More informationMyProxy Server Installation
MyProxy Server Installation Emidio Giorgio INFN First Latin American Workshop for Grid Administrators 21-25 November 2005 www.eu-egee.org Outline Why MyProxy? Proxy Renewal mechanism. Remote authentication
More informationEdinburgh (ECDF) Update
Edinburgh (ECDF) Update Wahid Bhimji On behalf of the ECDF Team HepSysMan,10 th June 2010 Edinburgh Setup Hardware upgrades Progress in last year Current Issues June-10 Hepsysman Wahid Bhimji - ECDF 1
More informationA QUICK INTRODUCTION TO VOMS
A QUICK INTRODUCTION TO VOMS 1.INTRODUCTION Aim of this write-up is to give an overview on the current VOMS affaire : a description of the mechanism currently used on the LCG/gLite middleware, the relationships
More informationAdvanced Job Submission on the Grid
Advanced Job Submission on the Grid Antun Balaz Scientific Computing Laboratory Institute of Physics Belgrade http://www.scl.rs/ 30 Nov 11 Dec 2009 www.eu-egee.org Scope User Interface Submit job Workload
More informationLayered Architecture
The Globus Toolkit : Introdution Dr Simon See Sun APSTC 09 June 2003 Jie Song, Grid Computing Specialist, Sun APSTC 2 Globus Toolkit TM An open source software toolkit addressing key technical problems
More informationLesson 6: Portlet for job submission
Lesson 6: Portlet for job submission Mario Torrisi University of Catania - Italy (mario.torrisi@ct.infn.it) Sci-GaIA Winter School This project has received funding from the European Union s Horizon 2020
More informationGrid Scheduling Architectures with Globus
Grid Scheduling Architectures with Workshop on Scheduling WS 07 Cetraro, Italy July 28, 2007 Ignacio Martin Llorente Distributed Systems Architecture Group Universidad Complutense de Madrid 1/38 Contents
More informationGlobus Toolkit 4 Execution Management. Alexandra Jimborean International School of Informatics Hagenberg, 2009
Globus Toolkit 4 Execution Management Alexandra Jimborean International School of Informatics Hagenberg, 2009 2 Agenda of the day Introduction to Globus Toolkit and GRAM Zoom In WS GRAM Usage Guide Architecture
More informationIntroduction to Programming and Computing for Scientists
Oxana Smirnova (Lund University) Programming for Scientists Lecture 4 1 / 44 Introduction to Programming and Computing for Scientists Oxana Smirnova Lund University Lecture 4: Distributed computing Most
More informationGrid Infrastructure For Collaborative High Performance Scientific Computing
Computing For Nation Development, February 08 09, 2008 Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi Grid Infrastructure For Collaborative High Performance Scientific
More informationTroubleshooting Grid authentication from the client side
System and Network Engineering RP1 Troubleshooting Grid authentication from the client side Adriaan van der Zee 2009-02-05 Abstract This report, the result of a four-week research project, discusses the
More informationThe University of Oxford campus grid, expansion and integrating new partners. Dr. David Wallom Technical Manager
The University of Oxford campus grid, expansion and integrating new partners Dr. David Wallom Technical Manager Outline Overview of OxGrid Self designed components Users Resources, adding new local or
More informationIntegrating the IEKP Linux cluster as a Tier-2/3 prototype centre into the LHC Computing Grid
Institut für Experimentelle Kernphysik, Universität Karlsruhe (TH) IEKP KA/2006 3 Integrating the IEKP Linux cluster as a Tier-2/3 prototype centre into the LHC Computing Grid Volker Büge 1,2, Ulrich Felzmann
More informationGROWL Scripts and Web Services
GROWL Scripts and Web Services Grid Technology Group E-Science Centre r.j.allan@dl.ac.uk GROWL Collaborative project (JISC VRE I programme) between CCLRC Daresbury Laboratory and the Universities of Cambridge
More informationJob submission and management through web services: the experience with the CREAM service
Journal of Physics: Conference Series Job submission and management through web services: the experience with the CREAM service To cite this article: C Aiftimiei et al 2008 J. Phys.: Conf. Ser. 119 062004
More informationParallel Computing in EGI
Parallel Computing in EGI V. Šipková, M. Dobrucký, and P. Slížik Ústav informatiky, Slovenská akadémia vied 845 07 Bratislava, Dúbravská cesta 9 http://www.ui.sav.sk/ {Viera.Sipkova, Miroslav.Dobrucky,
More informationCloud Computing. Up until now
Cloud Computing Lecture 4 and 5 Grid: 2012-2013 Introduction. Up until now Definition of Cloud Computing. Grid Computing: Schedulers: Condor SGE 1 Summary Core Grid: Toolkit Condor-G Grid: Conceptual Architecture
More informationEUROPEAN MIDDLEWARE INITIATIVE
EUROPEAN MIDDLEWARE INITIATIVE MSA1.2.1 - EMI REF ERENCE RELE AS ES EC MILESTONE: MS18 Document identifier: EMI_MS18_v1.0.doc Activity: Lead Partner: Document status: Document link: SA1 INFN Final http://cdsweb.cern.ch/record/1277546?ln=en
More informationOverview of WMS/LB API
Overview of WMS/LB API Emidio Giorgio www.eu-egee.org Outline API vs. CLI Overview of WMProxy API Overview of Logging and Bookkeeping API Usage examples API vs. CLI Users applications can be integrated
More informationUnit 2: Manage Files Graphically with Nautilus Objective: Manage files graphically and access remote systems with Nautilus
Linux system administrator-i Unit 1: Get Started with the GNOME Graphical Desktop Objective: Get started with GNOME and edit text files with gedit Unit 2: Manage Files Graphically with Nautilus Objective:
More informationEGEE and Interoperation
EGEE and Interoperation Laurence Field CERN-IT-GD ISGC 2008 www.eu-egee.org EGEE and glite are registered trademarks Overview The grid problem definition GLite and EGEE The interoperability problem The
More informationTesting SLURM open source batch system for a Tierl/Tier2 HEP computing facility
Journal of Physics: Conference Series OPEN ACCESS Testing SLURM open source batch system for a Tierl/Tier2 HEP computing facility Recent citations - A new Self-Adaptive dispatching System for local clusters
More informationIvane Javakhishvili Tbilisi State University High Energy Physics Institute HEPI TSU
Ivane Javakhishvili Tbilisi State University High Energy Physics Institute HEPI TSU Grid cluster at the Institute of High Energy Physics of TSU Authors: Arnold Shakhbatyan Prof. Zurab Modebadze Co-authors:
More informationQosCosGrid Middleware
Domain-oriented services and resources of Polish Infrastructure for Supporting Computational Science in the European Research Space PLGrid Plus QosCosGrid Middleware Domain-oriented services and resources
More informationInstallation and Administration
Introduction to GT3 Background The Grid Problem The Globus Approach OGSA & OGSI Globus Toolkit GT3 Architecture and Functionality: The Latest Refinement of the Globus Toolkit Core Base Services User-Defined
More informationInternational Collaboration to Extend and Advance Grid Education. glite WMS Workload Management System
International Collaboration to Extend and Advance Grid Education glite WMS Workload Management System Marco Pappalardo Consorzio COMETA & INFN Catania, Italy ITIS Ferraris, Acireale, Tutorial GRID per
More informationBookkeeping and submission tools prototype. L. Tomassetti on behalf of distributed computing group
Bookkeeping and submission tools prototype L. Tomassetti on behalf of distributed computing group Outline General Overview Bookkeeping database Submission tools (for simulation productions) Framework Design
More informationDataGrid. Document identifier: Date: 24/11/2003. Work package: Partner: Document status. Deliverable identifier:
DataGrid WMS GUI USER G UIDE Document identifier: Work package: Partner: WP1 Datamat SpA Document status Deliverable identifier: Abstract: This document provides a description of all functionalities provided
More informationOn the employment of LCG GRID middleware
On the employment of LCG GRID middleware Luben Boyanov, Plamena Nenkova Abstract: This paper describes the functionalities and operation of the LCG GRID middleware. An overview of the development of GRID
More informationR-GMA (Relational Grid Monitoring Architecture) for monitoring applications
R-GMA (Relational Grid Monitoring Architecture) for monitoring applications www.eu-egee.org egee EGEE-II INFSO-RI-031688 Acknowledgements Slides are taken/derived from the GILDA team Steve Fisher (RAL,
More informationGergely Sipos MTA SZTAKI
Application development on EGEE with P-GRADE Portal Gergely Sipos MTA SZTAKI sipos@sztaki.hu EGEE Training and Induction EGEE Application Porting Support www.lpds.sztaki.hu/gasuc www.portal.p-grade.hu
More informationIntroduction to Programming and Computing for Scientists
Oxana Smirnova (Lund University) Programming for Scientists Lecture 4 1 / 44 Introduction to Programming and Computing for Scientists Oxana Smirnova Lund University Lecture 4: Distributed computing, security
More informationVMs at a Tier-1 site. EGEE 09, Sander Klous, Nikhef
VMs at a Tier-1 site EGEE 09, 21-09-2009 Sander Klous, Nikhef Contents Introduction Who are we? Motivation Why are we interested in VMs? What are we going to do with VMs? Status How do we approach this
More informationProblemi di schedulazione distribuita su Grid
Problemi di schedulazione distribuita su Grid Ivan Porro Università degli Studi di Genova, DIST, Laboratorio BioLab pivan@unige.it 010-3532789 Riadattato da materiale realizzato da INFN Catania per il
More informationDESY. Andreas Gellrich DESY DESY,
Grid @ DESY Andreas Gellrich DESY DESY, Legacy Trivially, computing requirements must always be related to the technical abilities at a certain time Until not long ago: (at least in HEP ) Computing was
More informationLHC COMPUTING GRID INSTALLING THE RELEASE. Document identifier: Date: April 6, Document status:
LHC COMPUTING GRID INSTALLING THE RELEASE Document identifier: EDMS id: Version: n/a v2.4.0 Date: April 6, 2005 Section: Document status: gis final Author(s): GRID Deployment Group ()
More informationCREAM-WMS Integration
-WMS Integration Luigi Zangrando (zangrando@pd.infn.it) Moreno Marzolla (marzolla@pd.infn.it) JRA1 IT-CZ Cluster Meeting, Rome, 12 13/9/2005 1 Preliminaries We had a meeting in PD between Francesco G.,
More informationGrid Compute Resources and Job Management
Grid Compute Resources and Job Management How do we access the grid? Command line with tools that you'll use Specialised applications Ex: Write a program to process images that sends data to run on the
More informationGT 4.2.0: Community Scheduler Framework (CSF) System Administrator's Guide
GT 4.2.0: Community Scheduler Framework (CSF) System Administrator's Guide GT 4.2.0: Community Scheduler Framework (CSF) System Administrator's Guide Introduction This guide contains advanced configuration
More informationGrid Documentation Documentation
Grid Documentation Documentation Release 1.0 Grid Support Nov 06, 2018 Contents 1 General 3 2 Basics 9 3 Advanced topics 25 4 Best practices 81 5 Service implementation 115 6 Tutorials
More informationGlobus GTK and Grid Services
Globus GTK and Grid Services Michael Rokitka SUNY@Buffalo CSE510B 9/2007 OGSA The Open Grid Services Architecture What are some key requirements of Grid computing? Interoperability: Critical due to nature
More informationA Simplified Access to Grid Resources for Virtual Research Communities
Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA A Simplified Access to Grid Resources for Virtual Research Communities Roberto BARBERA (1-3), Marco FARGETTA (3,*) and Riccardo ROTONDO (2) (1) Department
More informationUniversità degli Studi di Ferrara
Università degli Studi di Ferrara Dottorato di ricerca in Matematica e Informatica Ciclo XXIII Coordinatore: Prof. Luisa Zanghirati Grid accounting for computing and storage resources towards standardization
More informationImplementing GRID interoperability
AFS & Kerberos Best Practices Workshop University of Michigan, Ann Arbor June 12-16 2006 Implementing GRID interoperability G. Bracco, P. D'Angelo, L. Giammarino*, S.Migliori, A. Quintiliani, C. Scio**,
More informationIntegration of Cloud and Grid Middleware at DGRZR
D- of International Symposium on Computing 2010 Stefan Freitag Robotics Research Institute Dortmund University of Technology March 12, 2010 Overview D- 1 D- Resource Center Ruhr 2 Clouds in the German
More informationConfigure Sensu and other Actions to Register Clients
Configure Sensu and other Actions to Register Clients Contents Introduction Prerequisites Requirements Components Used Background Information Configure Install Epel Repository Install Erlang Install Redis,
More informationHortonworks Technical Preview for Apache Falcon
Architecting the Future of Big Data Hortonworks Technical Preview for Apache Falcon Released: 11/20/2013 Architecting the Future of Big Data 2013 Hortonworks Inc. All Rights Reserved. Welcome to Hortonworks
More informationALICE Grid/Analysis Tutorial Exercise-Solutions
WLCG Asia Workshop, 2.12.2006 ALICE Grid/Analysis Tutorial Exercise-Solutions Andreas-Joachim Peters CERN www.eu-egee.org cern.ch/lcg http://cern.ch/arda EGEE is a project funded by the European Union
More informationIntroduction to Grid Infrastructures
Introduction to Grid Infrastructures Stefano Cozzini 1 and Alessandro Costantini 2 1 CNR-INFM DEMOCRITOS National Simulation Center, Trieste, Italy 2 Department of Chemistry, Università di Perugia, Perugia,
More informationDay 1 : August (Thursday) An overview of Globus Toolkit 2.4
An Overview of Grid Computing Workshop Day 1 : August 05 2004 (Thursday) An overview of Globus Toolkit 2.4 By CDAC Experts Contact :vcvrao@cdacindia.com; betatest@cdacindia.com URL : http://www.cs.umn.edu/~vcvrao
More informationARC integration for CMS
ARC integration for CMS ARC integration for CMS Erik Edelmann 2, Laurence Field 3, Jaime Frey 4, Michael Grønager 2, Kalle Happonen 1, Daniel Johansson 2, Josva Kleist 2, Jukka Klem 1, Jesper Koivumäki
More informationAutomated Installation Guide for CentOS (PHP 7.x)
Automated Installation Guide for CentOS (PHP 7.x) *Note: This script will not work on shared hosting, only works on CentOS dedicated server only. Prerequisites: Hardware: RAM: minimum 2 GB Processor: Core2duo
More informationGrid Computing Fall 2005 Lecture 5: Grid Architecture and Globus. Gabrielle Allen
Grid Computing 7700 Fall 2005 Lecture 5: Grid Architecture and Globus Gabrielle Allen allen@bit.csc.lsu.edu http://www.cct.lsu.edu/~gallen Concrete Example I have a source file Main.F on machine A, an
More informationEGI-InSPIRE. GridCertLib Shibboleth authentication for X.509 certificates and Grid proxies. Sergio Maffioletti
EGI-InSPIRE GridCertLib Shibboleth authentication for X.509 certificates and Grid proxies Sergio Maffioletti Grid Computing Competence Centre, University of Zurich http://www.gc3.uzh.ch/
More informationHow to build Scientific Gateways with Vine Toolkit and Liferay/GridSphere framework
How to build Scientific Gateways with Vine Toolkit and Liferay/GridSphere framework Piotr Dziubecki, Piotr Grabowski, Michał Krysiński, Tomasz Kuczyński, Dawid Szejnfeld, Dominik Tarnawczyk, Gosia Wolniewicz
More informationCondor-G: HTCondor for grid submission. Jaime Frey (UW-Madison), Jeff Dost (UCSD)
Condor-G: HTCondor for grid submission Jaime Frey (UW-Madison), Jeff Dost (UCSD) Acknowledgement These slides are heavily based on the presentation Jaime Frey gave at UCSD in Feb 2011 http://www.t2.ucsd.edu/twiki2/bin/view/main/glideinfactory1111
More informationThe LHC Computing Grid
The LHC Computing Grid Gergely Debreczeni (CERN IT/Grid Deployment Group) The data factory of LHC 40 million collisions in each second After on-line triggers and selections, only 100 3-4 MB/event requires
More informationCheck the FQDN of your server by executing following two commands in the terminal.
LDAP or Lightweight Directory Access Protocol, is a protocol designed to manage and access related information in a centralized, hierarchical file and directory structure. An LDAP server is a non-relational
More informationIntroducing the HTCondor-CE
Introducing the HTCondor-CE CHEP 2015 Presented by Edgar Fajardo 1 Introduction In summer 2012, OSG performed an internal review of major software components, looking for strategic weaknesses. One highlighted
More informationDr. Giuliano Taffoni INAF - OATS
Query Element Demo The Grid Query Element for glite Dr. Giuliano Taffoni INAF - OATS Overview What is a G-DSE? Use and Admin a DB: the Query Element; Upcoming Features; Working on QE People: Edgardo Ambrosi
More informationMolecular dynamics simulations in the MolDynGrid Virtual Laboratory by means of ARC between Grid and Cloud
Molecular dynamics simulations in the MolDynGrid Virtual Laboratory by means of ARC between Grid and Cloud Andrii Salnikov * NorduGrid 2016 * manf@grid.org.ua MolDynGrid Virtual Laboratory Has been established
More informationPreparing for the Prime Home Installation
Prerequisites and Verifications, page 1 Verify Default OS Configurations, page 1 Additional OS Configurations, page 7 Prerequisites and Verifications Before installing Prime Home, verify that the servers
More informationGRID COMPANION GUIDE
Companion Subject: GRID COMPANION Author(s): Miguel Cárdenas Montes, Antonio Gómez Iglesias, Francisco Castejón, Adrian Jackson, Joachim Hein Distribution: Public 1.Introduction Here you will find the
More informationOpenNebula 4.4 Quickstart CentOS 6 and ESX 5.x. OpenNebula Project
OpenNebula 4.4 Quickstart CentOS 6 and ESX 5.x OpenNebula Project December 17, 2013 Copyright 2013 OpenNebula Project, C12G Labs. All rights reserved. Although the information in this document has been
More informationGanga The Job Submission Tool. WeiLong Ueng
Ganga The Job Submission Tool WeiLong Ueng wlueng@twgrid.org Objectives This tutorial gives users to understand Why require Ganga in Grid environment What advantages of Ganga The Architecture of Ganga
More information