At a Glance. Introducing Security Metrics

Transcription

1 At a Glance PART I Introducing Security Metrics 1 What Is a Security Metric? Designing Effective Security Metrics Understanding Data Case Study 1: In Search of Enterprise Metrics Part II Implementing Security Metrics 4 The Security Process Management Framework Analyzing Security Metrics Data Designing the Security Measurement Project Case Study 2: Normalizing Tool Data in a Security Posture Assessment ix

2 x IT Security Metrics Part III Exploring Security Measurement Projects 7 Measuring Security Operations Measuring Compliance and Conformance Measuring Security Cost and Value Measuring People, Organizations, and Culture Case Study 3: Web Application Vulnerabilities Part IV Beyond Security Metrics 11 The Security Improvement Program Learning Security: Different Contexts for Security Process Management Case Study 4: Getting Management Buy-in for the Security Metrics Program Index

3 Contents Foreword... xix Acknowledgments... xxi Introduction... xxiii Part I Introducing Security Metrics 1 What Is a Security Metric?... 3 Metrics and Measurement... 5 Metrics Are a Result... 6 Measurement Is an Activity... 6 Security Metrics Today... 8 Risk... 8 Security Vulnerability and Incident Statistics Annualized Loss Expectancy Return on Investment Total Cost of Ownership The Dissatisfying State of Security Metrics: Lessons from Other Industries Insurance Manufacturing Design xi

4 xii IT Security Metrics Reassessing Our Ideas About Security Metrics Thinking Locally Thinking Analytically Thinking Ahead Summary Further Reading Designing Effective Security Metrics Choosing Good Metrics Defining Metrics and Measurement Nothing Either Good or Bad, but Thinking Makes It So What Do You Want to Know? Observe! GQM for Better Security Metrics What Is GQM? Setting Goals Asking Questions Assigning Metrics Putting It All Together The Metrics Catalog More Security Uses for GQM Measuring Security Operations Measuring Compliance to a Regulation or Standard Measuring People and Culture Applying GQM to Your Own Security Measurements Summary Further Reading Understanding Data What Are Data? Definitions of Data Data Types Data Sources for Security Metrics System Data Process Data Documentary Data People Data We Have Metrics and Data Now What? Summary Further Reading... 72

5 Contents xiii Case Study 1: In Search of Enterprise Metrics Scenario One: Our New Vulnerability Management Program Scenario Two: Who s on First? Scenario Three: The Value of a Slide Scenario Four: The Monitoring Program Scenario Five: What Cost, the Truth? Summary Part II Implementing Security Metrics 4 The Security Process Management Framework Managing Security as a Business Process Defining a Business Process Security Processes Process Management over Time The SPM Framework Security Metrics Security Measurement Projects The Security Improvement Program Security Process Management Before You Begin SPM Getting Buy-in: Where s the Forest? The Security Research Program Summary Further Reading Analyzing Security Metrics Data The Most Important Step Reasons for Analysis What Do You Want to Accomplish? Preparing for Data Analysis Analysis Tools and Techniques Descriptive Statistics Inferential Statistics Other Statistical Techniques Qualitative and Mixed Method Analysis Summary Further Reading

6 xiv IT Security Metrics 6 Designing the Security Measurement Project Before the Project Begins Project Prerequisites Deciding on a Project Type Tying Projects Together Getting Buy-in and Resources Phase One: Build a Project Plan and Assemble the Team The Project Plan The Project Team Phase Two: Gather the Metrics Data Collecting Metrics Data Storing and Protecting Metrics Data Phase Three: Analyze the Metrics Data and Build Conclusions Phase Four: Present the Results Textual Presentations Visual Presentations Disseminating the Results Phase Five: Reuse the Results Project Management Tools Summary Further Reading Case Study 2: Normalizing Tool Data in a Security Posture Assessment Background: Overview of the SPA Service SPA Tools Data Structures Objectives of the Case Study Methodology Challenges Summary Part III Exploring Security Measurement Projects 7 Measuring Security Operations Sample Metrics for Security Operations Sample Measurement Projects for Security Operations SMP: General Risk Assessment SMP: Internal Vulnerability Assessment SMP: Inferential Analysis Summary Further Reading

7 Contents xv 8 Measuring Compliance and Conformance The Challenges of Measuring Compliance Confusion Among Related Standards Auditing or Measuring? Confusion Across Multiple Frameworks Sample Measurement Projects for Compliance and Conformance Creating a Rationalized Common Control Framework Mapping Assessments to Compliance Frameworks Analyzing the Readability of Security Policy Documents Summary Further Reading Measuring Security Cost and Value Sample Measurement Projects for Compliance and Conformance Measuring the Likelihood of Reported Personally Identifiable Information (PII) Disclosures Measuring the Cost Benefits of Outsourcing a Security Incident Monitoring Process Measuring the Cost of Security Processes The Importance of Data to Measuring Cost and Value Summary Further Reading Measuring People, Organizations, and Culture Sample Measurement Projects for People, Organizations, and Culture Measuring the Security Orientation of Company Stakeholders An Ethnography of Physical Security Practices Summary Further Reading Case Study 3: Web Application Vulnerabilities Source Data and Normalization Outcomes, Timelines, Resources Initial Reporting with Dirty Data Ambiguous Data Determining Which Source to Use Working with Stakeholders to Perform Data Cleansing

8 xvi IT Security Metrics Follow-up with Reports and Discussions with Stakeholders Lesson Learned: Fix the Process, and Then Automate Lesson Learned: Don t Wait for Perfect Data Before Reporting Summary Part IV Beyond Security Metrics 11 The Security Improvement Program Moving from Projects to Programs Managing Security Measurement with a Security Improvement Program Governance of Security Measurement The SIP: It s Still about the Data Requirements for a SIP Before You Begin Documenting Your Security Measurement Projects Sharing Your Security Measurement Results Collaborating Across Projects and Over Time Measuring the SIP Security Improvement Is Habit Forming Is the SIP Working? Is Security Improving? Case Study: A SIP for Insider Threat Measurement Summary Further Reading Learning Security: Different Contexts for Security Process Management Organizational Learning Three Learning Styles for IT Security Metrics Standardized Testing: Measurement in ISO/IEC The School of Life: Basili s Experience Factory Mindfulness: Karl Weick and the High-Reliability Organization Final Thoughts Summary Further Reading

9 Contents xvii Case Study 4: Getting Management Buy-in for the Security Metrics Program The CISO Hacked My Computer What Is Buy-in? Corporations vs. Higher Ed: Who s Crazier? Higher Education Case Study Project Overview Themes Findings Key Points Influence and Organizational Change Conclusion Index