Dysfunctional Testing

Transcription

1 Dysfunctional Testing

2 Chief Security Advocate, Cisco Systems Chris Core CSDL Team Member Cisco Security Black Belt (CSBB), CISSP Co-Founded the Cisco Security Awareness Program with Tony Vargas and Lisa McDonald in 2012

3 Tony - Co-Founder & CEO - Security Together, LLC - CSSLP, CISSP-ISSAP - (ISC) 2 President s Award Winner Chair, Application Security Advisory Council - (ISC) 2 - Cisco Security Champion of the Year 2013 (Inaugural) - Cisco CIIP Mentorship Award Winner Cisco Security Champion (2012) - Cisco Security Black Belt - Cisco Client Partnership Award Winner - Co-Founder, Chairman & President - (ISC) 2 Sacramento Chapter

4 Agenda Define Dysfunctional Testing Tools and Vehicles for Dysfunctional Testing The Process of Dysfunctional Testing Dysfunctional Call to Action (ISC) 2 e-symposium 4

5 Trusting What We Test VS

6 Product Testing Check features for functionality Features that improve product or meet customer need Management does not prioritize security, but manages by bug counts Hard to drive security in a group

7 Customers expect both functional and application security testing

8 Functional Test Process

9

10 Functional Test - Does the feature or product work according to the written specification?

11 Definition: Dysfunctional Test Test cases that behave outside social norms What the developer never expected or imagined Creating a test case to uncover a malfunctioning part or element Ted Not performing testing normally

12 Functional Testing: The process of checking if something works as designed

13 Functional Testing: The process of checking if something works as designed Dysfunctional Testing is the opposite

14 Dysfunctional Testing is thinking differently about how testers approach their products

15 Four Keys to Effective Dysfunctional Test 1. Goal oriented what will be achieved? 1. Full product is in scope 1. Simulate realistic compromise patterns if production pieces are in scope 1. Break testing into iterations Source: Zane DevOps Security Presentation; concept modified for dysfunctional.

16 Dysfunctional is Not A call to turn all testers into white hat hackers (not possible) Just a negative test negative test is constrained to unexpected input

17 A Search for Motivation Misuse case pinpointing the bad in the good Agile user stories Functional Spec Anti-functional Spec Threat Modeling Mindset DevOps Rugged Software

18 A pattern of #dysfunction in your test process is good; a virtual hunt for vulnerabilities.

19 Tools of Dysfunction Testers brain! Teamwork Codenomicon for fuzz testing Dynamic / Static Analysis Root cause analysis

20 Dysfunctional Testing Vehicles - Agile - DevOps - Hack-a-Thon - Internal Bug Bounty

21 Dysfunctional is Difficult It can t be done by one person -- requires a community Persistence High level of knowledge Knowledge ++ Experience ++ Industry Relationships

22 Six Steps of Dysfunctional Testing 1. Build a Tool Chest 2. Search for Motivation 3. Compute the Attack Surface 4. Plan the Attack 5. Execute the Attack 6. Document and Present Results/Demonstration to Others

23 1. Building Your Tool Chest + =

24 Kali tools for product security testing 1.Nmap 2.Nikto 3.W3AF 4.ZAP 5. SQLmap 6.Wireshark 7.Metasploit 8.Your Brain

25 2. Search for Motivation Not traditional skim and approve; hunt for attack vectors in the artifacts Review the artifacts with YOUR SECURITY HAT ON (critically) Functional specifications (design, software, hardware) Test Plans Agile User Stories

26 Being Critical for Security Ask questions about the ideas and information presented in the document Take Notes: document your questions to ensure you define answers or mark as unanswered Make judgments about the validity or relevance of the document to the security features and architecture of the product

27 Writing an Attack User Story As an attacker, I want to break <function> so that I can receive this <benefit>. As an attacker, I want to break the web administrative interface so that I can dump a list of passwords. As an attacker, I want to break the DNS daemon so that I can get elevated privilege on the product.

28 3. Attack Surface Information gathering before an attack Attack Surface -- collection of all entry points that could be used to attack the product (software or hardware)

29 4. Planning the Attack Refine the attack user stories by adding prioritization Figure of Merit {High, Medium, or Low} Probability of success for an attack Impact of an attack Ease of detecting the attack and attacker Tool Available to Assist? Write test procedures to ensure repeatability

30 5. Executing the Attack Automated tools? Simulate the attack manually? Netcat Browser Tools to create ip packets Ruby or perl script

31 6. Documenting Results Create an official defect / bug; the issue must be tracked Identified issues must be understandable by the developer stand alone Share how the bug was found, and the issue becomes a learning tool for others

32 Dysfunctional Call to Action Will you commit? A Commitment to Break Your Product

33 Conclusion Release a portion of the testers from the binding of functional test Functional test is required; security testing is mandatory Expand your mind and tool sets Dysfunctional testing is not hard, the skills can be learned Dysfunctional test is good for products

34 Q & A Tony Vargas Chris Romeo