EGI-InSPIRE EGI Operations and Best Practices Tiziana Ferrari/EGI.eu CHAIN-REDS Workshop ISGC 2013 CHAIN-REDS workshop, ISGC, March 2013 1
EGI Infrastructure EGI Infrastructure is the composition of multiple Resource Infrastructures Infrastructure providers (National Grid Infrastructures, EIROs, other federated RIs): operating the local resources centres, and integrated globally with other providers through a set of coordination activities (EGI.eu) Resource centres (sites) which contribute resources (to EGI and other infrastructures), the smallest operational domain CHAIN-REDS workshop, ISGC, March 2013 2
Capacity and yearly CPU Usage Metrics Capacity CPU wall clock time CPU cores (EGI and integrated resource providers) Disk/Tape (PB) Total normalized CPU wall clock time consumed grid (Billion HEP-SPEC 06 hours) Value 372,612 (315 resource production centres) 180/167 15.5 (02-2012/02-2013) 31.4 (01-2010/02-2013) Job/year (Million) 528.4 Jobs % of total norm. CPU wall time consumed Average Job/day (Million) High-Energy Physics 93.78% Astronomy and Astrophysics 2.78 % Life Sciences 1.31% Remaining disciplines 2.13% 1.67 (2.25 including local computation) CHAIN-REDS workshop, ISGC, March 2013 3
VO and user Statistics Metric Values (March 2013) Registered VOs 237 national and international VOs Registered users 22085 High: CPU time > 1 Year/Week Active VOs Medium: CPU time > 1 Month/Week, < 1 Year/Week Low: CPU > 1 Day/Week, < 1 Month/Week 25 17 12 CHAIN-REDS workshop, ISGC, March 2013 4
Infrastructure providers Integrated infrastructure providers sharing policies, procedures, tools, QoS agreements and part of the same operations structure Members of the EGI collaboration (EGI Council/EGI- InSPIRE) External providers Austria and Ukraine, Latin America, Canada, Asia Pacific Peer providers own operations tools and procedures, compatible policies, loose operations collaboration with EGI Open Science Grid CHAIN-REDS workshop, ISGC, March 2013 5
Operations Architecture CHAIN-REDS workshop, ISGC, March 2013 6
EGI providers Integrated EGI-InSPIRE Partners and EGI Council Members Internal/External RPs being integrated External RP Peer RP CHAIN-REDS workshop, ISGC, March 2013 7
Integration Operations of an integrated infrastructure are fully delegated to the infrastructure operators (integrated) resource infrastructures responsible of providing operations support, services, tools to the local community and local resource centre administrators (integrated) resource infrastructures are free to choose the grid middleware/s to be deployed One requirement: services must be discoverable (GLUE 1.3/2.0) Examples of integrated middleware stacks: (ex) glite, ARC, UNICORE, dcache, Globus, QCG, desktop grid CHAIN-REDS workshop, ISGC, March 2013 8
EGI Integration Policies and procedures Adoption of compatible policies (security, accounting, service operations) https://wiki.egi.eu/wiki/spg:documents Adoption of the same set of operational procedures (site certification/decommissioning, service level management, incident handling, vulnerability issue handling etc.) https://wiki.egi.eu/wiki/operations_procedures CHAIN-REDS workshop, ISGC, March 2013 9
EGI Integration registry and monitoring GOCDB is the EGI service and contact registry Provides infrastructure topology information for monitoring and accounting Will be able to support multiple registries for different infrastructures (even not integrated), services will be able to be part of different infrastructures ongoing development Services are monitored by the (distributed) Service Availability Monitoring infrastructure CHAIN-REDS workshop, ISGC, March 2013 10
EGI Integration support (international) users and service operators of an integrated infrastructure share the same support services 1 st level operations support Resource Infrastructure 2 nd level and 3 rd level EGI and technology providers 1 entry point: GGUS Integrated Resource Infrastructures can still deploy their own helpdesk, to be integrated with GGUS CHAIN-REDS workshop, ISGC, March 2013 11
Requirements EGI Integration Quality of Service Minimum service availability provided by certified Resource Centres Resource Centre Operational Level Agreement https://documents.egi.eu/document/31 Resource Centres can be part of a test infrastructure if not ready for certification Minimum service availability provided by Resource Infrastructures Resource Provider Operational Level Agreement https://documents.egi.eu/document/463 CHAIN-REDS workshop, ISGC, March 2013 12
EGI Integration Governance Collaboration between EGI and an integrated Resource Infrastructure: regulated by a MoU Examples: https:///community/resourceproviders/index.html Template: https://documents.egi.eu/document/215 Integrated Resource Infrastructure are member of the EGI Operations Management Board Technical roadmapping Approval of new policies and procedures, of changes to the OLAs Requirement gathering 1 interface towards the technology providers http://go.egi.eu/omb CHAIN-REDS workshop, ISGC, March 2013 13
Conclusions EGI infrastructure EGI integrated resource providers retain the responsibility of running operations locally Benefits of an integrated infrastructure for the user Compatible procedures, QoS guarantees, one support interface, one security framework CHAIN-REDS workshop, ISGC, March 2013 14