Quo pertentas, OSS? How Open Source can benefit from well-crafted Tests

Size: px

Start display at page:

Download "Quo pertentas, OSS? How Open Source can benefit from well-crafted Tests"

Damon Nichols
5 years ago
Views:

1 Quo pertentas, OSS? How Open Source can benefit from well-crafted Tests Björn Kimminich Web: v1.0

2 Let s start with some code

3 and a corresponding unit test!

4 It passes with flying colors

5 and achieves 100% code coverage!

6 Nothing could possibly go wrong!

7 How about adding another test?

8 Oops!

9 Finding Bugs in Open Source Software

commit rights Occasionally during Hackathons Hard to organize properly

10 Pair Programming Peer Review Committer Review Code Reviews Infeasible with remote development Developers review each other Not everyone has commit rights Occasionally during Hackathons Hard to organize properly Senior developers review contributions before merge into master Cartoon: Geek & Poke

Static Code Analysis Popular Open Source Tools FindBugs CheckStyle PMD Sonar Find code smells and potential programming errors but miss a lot as

11 Static Code Analysis Popular Open Source Tools FindBugs CheckStyle PMD Sonar Find code smells and potential programming errors but miss a lot as well or produce false positives Some commercial Tools might be more powerful but are typically not affordable for OSS projects Cartoon: Geek & Poke

12 Testing Unit Tests Penetration Tests Integration Tests Test Types Load/Stress Tests GUI Tests Manual Tests Cartoon: Geek & Poke

13 Best vs. Bad Practices for Testing

14 Test Pyramid Manual Tests GUI Tests Integration Tests Unit Tests Source: WatirMelon

15 Test Ice-Cream Cone Manual Tests GUI Tests Integratio n Tests Unit Tests Source: WatirMelon

16 Happy Path Testing Photo: Tortured Mind Photography

17 Testing Border & Exceptional Cases

18 No Assertions

19 API Tests

20 Scenario Tests with BDD

21 Benefits of well-crafted Tests for OSS

22 Maintainability++ A suite of automated regression tests helps finding defects resulting from code changes New contributors do not have to fear touching old code neither do long-time committers after a longer vacation! Cartoon: Geek & Poke

Documentation++ External and Javadoc documentation tends to rot quickly and becomes obsolete or even misleading Tests that get outdated tend to break, so they have to be fixed resulting in updated

23 Documentation++ External and Javadoc documentation tends to rot quickly and becomes obsolete or even misleading Tests that get outdated tend to break, so they have to be fixed resulting in updated documentation Well-written tests document the intended behavior of a class or component Even if the production code is hard to understand, a good test can help to fill this gap Cartoon: Geek & Poke

Specification++ Writing tests before the production code is even better than just documenting existing code Consequent TDD / BDD will let the Tests become

24 Specification++ Writing tests before the production code is even better than just documenting existing code Consequent TDD / BDD will let the Tests become the actual specification of the program's intended behavior Failing tests indicate that the specification is not met yet (or any more) Cartoon: Geek & Poke

25 Contribution++ Well maintained, documented and tested projects are safer and more fun to contribute to Nobody likes working on an untested piece of unreadable code (especially in their free time) Cartoon: Geek & Poke

26 Truck Factor++ How many project contributors could be fatally hit by a truck before the project perishes? The lower the number, the more volatile the project as it relies on individual experts The number can be increased by spreading knowledge and lowering entry barriers Cartoon: Geek & Poke

27 Introducing Unit Tests to OWASP ZAP

28 OWASP Zed Attack Proxy (ZAP) Easy-to-use integrated penetrationtesting tool Locates vulnerabilities in web applications Helps building secure apps OWASP Flagship Project Programmed in Java with javax.swing UI

How to contribute to ZAP? Develop core features https://code.google.com/p/zaproxy/ Develop addons https://code.google.com/p/zap-extensions/ Help with translation https://crowdin.

29 How to contribute to ZAP? Develop core features Develop addons Help with translation Promote ZAP

30 ZAP Truck Factor 2 Source: Ohloh

31 Starting from zero Unit Tests Some JUnitbased Integration tests No Unit Tests

32 Separate Test Project

33 ZAPs first Unit Test

34 Adding some more Show Cases

35 Separation into Test Suites

36 Providing Test Guidelines Code Quality Code Coverage Test Libraries Naming Conventions Behavior Driven Development Test Suites Types of Tests

37 Pull vs. Push Pull Push Photos: One Man Think Tank

38 Measure Code Coverage

39 Move Tests close to Production Code

40 Instant execution from IDE

41 Run all Tests during Continuous Build...

42 ...and let it fail when any tests fail!

43 Future: Adding a GUI Testing Framework ZAP is very UI heavy which makes a lot of the code hard or impossible to unit test Right now there are no GUI Tests in place for ZAP Several free UI Testing Frameworks exist for Java Swing unfortunately none is actively maintained any more

44 Testing is a crucial part of Software Development Conclusion Good Tests are the better documentation Tests can make a difference between a prospering and a dead-end OSS project

45 Thank you! Björn Kimminich Web: Background Image: Eikira

3 Continuous Integration 3. Automated system finding bugs is better than people

3 Continuous Integration 3. Automated system finding bugs is better than people This presentation is based upon a 3 day course I took from Jared Richardson. The examples and most of the tools presented are Java-centric, but there are equivalent tools for other languages or you can