Management & Operations Excellence Scott Roots Business Development Director EMEA Operational Excellence 2015 Uptime Institute, LLC
The global Data Center Authority 1000+ Certifications in over 80 Countries For over 20 years, Uptime Institute offers independent, technology neutral, and senior-level expertise, advisory, and Certification at both the executive and technical level. Unbiased & Vendor Neutral 1 2015 Uptime Institute, LLC
Introduction Uptime Institute simplifies the process of planning, designing, building and operating a data center. WE DELIVER BUSINESS VALUE BY REDUCING THE RISK OF INFRASTRUCTURE OUTAGES 2 2015 Uptime Institute, LLC
Downtime happens According to 2015 Survey Data: Nearly 50% of enterprise IT organizations experienced a business-impacting data center outage in their own data center during the previous 12 months. In 2016, one third of enterprise IT organizations experienced an outage from a colocation provider in the previous 12-month period. In 2017 25% of respondants experienced an outage. Members of the Uptime Institute network had half as many. 3 2015 Uptime Institute, LLC
Outages due to human error? 4 2015 Uptime Institute, LLC
Failure does not keep to 9-5 5 2015 Uptime Institute, LLC
Nor does it stick to Monday to Friday 6 2015 Uptime Institute, LLC
Downtime Costs Money Industry ball park estimate is now around $9,000 per MINUTE!!! 1 Minute - $9,000 10 Minutes - $90,000 1 Hour - $540,000 6 hours - $3,240,000 7 2015 Uptime Institute, LLC
Is all news, really good News? 8 2015 Uptime Institute, LLC
No standard Industry existed to help data center executives assess operations Management and Operations program provides: Common language/vocabulary of data center operations Focus of data center management Guidance for resource allocation and requirements 9 2015 Uptime Institute, LLC
Data center operations reviews Uptime Institute has conducted Operational Sustainability and M&O Assessments since 2010 based upon decades of site operations knowledge and experience: Operational Sustainability Certifications: Tier + Gold, Silver, or Bronze Management & Operations Stamp of Approval 10 2015 Uptime Institute, LLC
A score-based system for evaluating operations Better match site resiliency to IT workload criticality Standardize Quality of Service to end users Evaluate vendor performance Provide transparency to C- level stakeholders 11 2015 Uptime Institute, LLC
M&O candidate attributes: Operational data center 24 x 7 uptime requirement Significant cost of downtime Not a Tier Certified Constructed Facility Commitment to excellence in operations 12 2015 Uptime Institute, LLC
Who is Ultimately in Charge? 13 2015 Uptime Institute, LLC
The Corporate Dilemma 14 2015 Uptime Institute, LLC
Operational Sustainability Elements Impact to Operations Management & Operations Building Characteristics Site Location Change Opportunity 15 2015 Uptime Institute, LLC
Pathway to Excellence 16 2015 Uptime Institute, LLC
How to measure effectively 17 2015 Uptime Institute, LLC
Proactive Anticipate issues or risk Look for continued improvement 18 2015 Uptime Institute, LLC
Practiced Formal disciplined approach available Procedures in place and followed 19 2015 Uptime Institute, LLC
Informed Site operations are known and available Knowledge is owned and maintained by Organization, NOT individuals 20 2015 Uptime Institute, LLC
5 Key Areas where problems can be 21 2015 Uptime Institute, LLC
Staffing and Organization Inadequate Staffing Levels Excessive Overtime Roles and Responsibilities not documented No Escalation Process High personnel turnover DC staff not integrated 22 2015 Uptime Institute, LLC
Maintenance No formal housekeeping policy No lists of equipment No Lists of PM activities Critical Data missing Lack of quality control Deferred maintenance exceeds 10% 23 2015 Uptime Institute, LLC
Management No predictive maintenance program Roles and responsibilities not listed Missing contracts Poor communications No Briefings for visitors Poor failure analysis 24 2015 Uptime Institute, LLC
Operations Missing site policies Reference library not up to date Load management settings not listed No process for forecasting future use Cooling set points not listed No Lifecycle plan 25 2015 Uptime Institute, LLC
Training Undocumented On the Job training program Training requirement per role not listed No list of required qualification No process for forecasting future use No formal training program Not enough experience with specific equipment 26 2015 Uptime Institute, LLC
Tour of the not so good 27 2015 Uptime Institute, LLC
Waterproofing. 28 2015 Uptime Institute, LLC
Priorities? 29 2015 Uptime Institute, LLC
Some Tough Questions What is the trend of deferred maintenance? Where are your formal training plans? Do Overtime hours exceed 10%? Is your personnel turnover rate changing? Do you analyze trends of events and incidents? Do you have frequent last minute deployments? 30 2015 Uptime Institute, LLC
Small issues can add up 31 2015 Uptime Institute, LLC
Bad habits can creep up on you Cabling badly, incorrectly or not even labelled Equipment not uniquely labeled/asset tagged Loads out of balance Capacities not tracked or managed Deferred maintenance exceeds 10% Housekeeping, if it looks like a mess, it is a mess! Common theme? None of these happen overnight! 32 2015 Uptime Institute, LLC
Framework Define Concepts Availability Req Corp Policies Capabilities Create Asset list Unique asset label program Create MOP Plan Develop MMS Calculate Staffing Req Maintenance Frequency Predictive Maintenance Plan Create Maintenance Schedule Track Completed Activity Internal Staff Asset Replacement Plan Staff/Vendor Responsibilities Detail MOP, SOP, EOP Trades, Skills, Licenses External/Vendor Staff Create Training Plan Induction Training Staff Training Vendor Training Housekeeping Finance Metrics Measure Capacity Create Policies Load management Life Cycle Planning Site Policies Reference Library Incident Tracking Manage Contracts 33 2015 Uptime Institute, LLC
At the Portfolio Level Additional Questions Can you easily replace any key member of you DC team? Are you protected against poor operations practices migrating from older sites to higher criticality data centers? Do you have sites that operate in isolation, ignoring global corporate standards? Do you even have corporate global standards? If you outsource any aspect of your data center operations, how do you avoid losing responsibility and accountability? Do you manage an outsourcing contract.or direct an expert team? 34 2015 Uptime Institute, LLC
Summary The value of industry best practices is in the process of continuous improvement Discovery leads to learning Learning leads to change Change leads to improvement Regular reviews leads to discovery Crises can be avoided 35 2015 Uptime Institute, LLC
Questions? Scott Roots, Uptime Institute sroots@uptimeinstitute.com 1000 DATA CENTRES CERTIFIED IN 80 + COUNTRIES 2015 Uptime Institute, LLC