Software Engineering 2 (SWT2) Chapter 7: BDD, and Testing (in Rails)

Transcription

1 Software Engineering 2 (SWT2) Chapter 7: BDD, and Testing (in Rails)

2 Agenda 2 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Testing Tests & Hints for Successful Test Design Outlook

3 Agenda 3 untzuntz.wordpress.com/ 2007/12/11/algorithms/

4 1. Collecting User Stories 4 Start game Submit guess Win game Loose game Play again Request hint Save score Here and the following: * [Chelimsky et al.: The Rspec Book, 2010] * [Rappin: Rails Test Prescriptions, 2010]

5 2. Planning the First Release 5 Focus: just play the game Selecting User Stories

6 2. Planning the First Release 6 Relevant Selected User Stories Start game Submit guess Win game? Play again? Request hint? Save score? Selected User Stories Start game The secret code?! Submit guess

7 A few Words about User Stories 7 User Stories are a planning tool Attributes Have business value Be testable Be small enough to be implemented in one iteration

8 BDD Cycle Recap 8 [Chelimsky et al.: The Rspec Book, 2010]

9 Start Game Feature 9

10 Submit Guess Feature 10

11 Scenario Outlines 11

12 Scenario Outlines 12 Adopted from FIT

13 Test::Unit vs. RSpec 13 Test::Unit comes with Ruby

14 Test::Unit vs. RSpec 14 RSpec has syntactical sugar in it We ll use RSpec ;) rspec-vs-testunit/

15 BDD Cycle Recap 15 [Chelimsky et al.: The Rspec Book, 2010]

16 Agenda 16 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Goals of Automated Testing The Case for BDD Writing Software that Matters Building Blocks of Tests and BDD Testing Tests & Hints for Successful Test Design Outlook

17 Goals of Automated Developer Testing 17 Feature 1: Website registration Developer 1 (no TDD/BDD) Minute 5: working registration page Minute 8: feature is tested (3 times) Developer 2: with TDD/BDD (almost no browser testing) Minute 5: working test Minute 10: working implementation Minute 10.30: feature is tested (3 times)

18 Goals of Automated Developer Testing 18 Feature 2: Special case for feature 1 Developer 1 (no TDD/BDD) Minute 11: implemented Minute 14: tested (3 times) Minute 17: refactoring ready Minute 19: tested feature 1 Minute 21: tested feature 2 Minute 22: committed Developer 2: with TDD/BDD (almost no browser testing) Minute 12.30: test ready Minute 15.30: implemented Minute 16.00: tested (3 times) Minute 19: refactoring ready Minute 19.10: tested Minute 20.10: committed

19 Goals of Automated Developer Testing 19 Finding errors faster Better code (correct, robust, maintainable) Automated developer testing is frequently faster Easier to add new features Easier to modify existing features BUT Tests might have bugs Test environment!= production environment Code changes break tests we ll cover a bit of this in this lecture

20 Agenda 20 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Goals of Automated Testing The Case for BDD Writing Software that Matters Building Blocks of Tests and BDD Testing Tests & Hints for Successful Test Design Outlook

21 How Traditional Projects Fail 21 Delivering late Delivering over budget Delivering the wrong thing Unstable in production Costly to maintain

22 Why Traditional Projects Fail 22 Smart people trying to do good work Stakeholders are well intended Process in traditional projects Planning Analysis Design Code Test Deploy Much effort for Documents for formalized hand-offs Templates Review committees

23 Why Traditional Projects Fail 23 The later we find a defect, the more expensive to fix it front-loading a software development process makes sense Really?! Reality shows Project plans are wonderful Adjustments/assumptions are made during analysis, design, code Re-planning takes place Example: testing phase Tester raises a defect Programmer claims he followed the specification Architect blames business analyst etc. exponential cost

24 Why Traditional Projects Fail 24 People are afraid of making changes Unofficial changes are carried out Documents get out of sync... Again, why do we do that!? To minimize the risk of finding a defect to late

25 A Self-Fulfilling Prophecy 25 We conduct the front-loaded process to minimize exponential costs of change Project plan Requirements spec High-level design documents Low-level design documents This process causes the exponential costs of change! A self-fulfilling prophecy Makes sense for a bridge, ship, or a building Software (and Lego) are EASY to change!

26 A Self-Fulfilling Prophecy 26 Many teams work very successful with traditional approaches High degree of communication and collaboration is needed Behavior-driven development an Agile methodology

27 Behavior-driven Development 27 People ask Why do so many software projects fail? Why are we consistently bad at delivering software? Why do larger teams suffer more? What can be done? A series of lightweight methods Focus: delivering working software to users

28 The Agile Manifesto 28 We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more.

29 How Agile Methods Address Project Risks 29 No longer late or over budget Tiny iterations Easy to calculate budget High-priority requirements first No longer delivering the wrong thing Strong stakeholder communication Short feedback cycles

30 How Agile Methods Address Project Risks 30 No longer unstable in production Delivering each iteration High degree of automation No longer costly to maintain Maintenance mode since Sprint 2 Maintenance of multiple versions during development

31 The Cost of Going Agile 31 Outcome-based planning / no complete detailed project plan Streaming requirements / a new requirements process Evolving design / no complete upfront design flexible Changing existing code / need for refactoring

32 The Cost of Going Agile 32 Frequent code integration / continuous integration Continual regression testing / add n th feature; test n-1 features Frequent production releases / organizational challenges Co-located team / keep momentum

33 Agenda 33 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Goals of Automated Testing The Case for BDD Writing Software that Matters Building Blocks of Tests and BDD Testing Tests & Hints for Successful Test Design Outlook

34 Writing Software that Matters 34 BDD is about implementing an application by describing its behavior from the perspective of its stakeholders Principles of BDD 1. Enough is enough 2. Deliver stakeholder value 3. It s all behavior

35 Maximum BDD Pyramid 35 Vision Goals Epics Use Case Feature User Stories Scenarios Scenario Steps Test Cases

36 Vision 36 All Stakeholders, one statement Core stakeholders vs. incidental stakeholders Example: improve Supply Chain; understand customers better Core stakeholders have to define the vision Incidental stakeholders help understand what is possible at what cost with what likelihood

37 Goals 37 Goals have to be identified Examples Easier ordering process Better access to suppliers information Goals should be SMART Specific (when is it done?) Measurable (was the objective reached?) Achievable (reduce unrealistic expectations) Relevant (not every exception is relevant) Timeboxed (certain investment in time)

38 Epics 38 Huge themes / feature sets are described as an epic Too high level to start coding Useful for conversations Examples Reporting Customer registration

39 Use Case Features 39 Describe the behavior we will implement in software Can be traced back to a stakeholder Warning: do not directly start at this level Is it a waterfall process? Yes, we think about goals to be achieved No, we just do enough Explain the value/context of a feature to stakeholders not too much detail Features deliver value to stakeholders

40 User Stories 40 Stories are demonstrable functionality 1 feature 1..n User Stories Stories should be vertical A token for a conversation

41 User Stories 41 Story content Title Narrative Description, reason, benefit As a <stakeholder>, I want <feature> so that <benefit> In order to <benefit>, a <stakeholder> wants to <feature> Acceptance criteria Stories in, features out

42 Scenarios, Scenario Steps, Test Cases 42 1 User Story 1..n scenarios Each scenario describes one aspect of a User Story Describe high-level behavior 1 scenario m scenario steps + step implementation Given When Then 1 scenario step 0..i tests (e.g., in RSpec) Describe low-level behavior

43 Agenda 43 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Test Data Test Doubles Setup and Teardown Model Tests View Tests Controller Tests Routing Tests Outgoing Mail Tests Helper Tests Integration and Acceptance Tests Testing Tests & Hints for Successful Test Design Outlook

44 Agenda 44 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Test Data Test Doubles Setup and Teardown Model Tests View Tests Controller Tests Routing Tests Outgoing Mail Tests Helper Tests Integration and Acceptance Tests Testing Tests & Hints for Successful Test Design Outlook

45 Test Data Overview 45 Fixtures Fixed state at the beginning of a test Assertions can be made against this state Factories Blueprint for models Used to generate test data locally in the test

46 Fixtures 46 test/fixtures/users.yml

47 Fixtures, 1:n relationship 47 test/fixtures/companies.yml test/fixtures/users.yml

48 Fixtures, n:m relationship 48 test/fixtures/users.yml

49 Fixtures are ERB Files 49

50 Loading Fixtures 50

51 Why Fixtures are a Pain 51 Fixtures are global Only ONE set of data Every test has to deal with ALL test data Fixtures are spread out Own directory One file per model data for one test is spread out over many files Tracing relationships is a pain

52 Why Fixtures are a Pain 52 Fixtures are distant A test fails It is unclear which data is used How are values computed? assert_equal(users(:ernie).age + users(:bert).age), 20) Fixtures are brittle Tests rely on this data Tests break when data is changed Data requirements may be incompatible

53 Fixing Fixtures with Factories 53 Test data should be Local (defined as closely as possible to the test) Compact (easy and quick to generate; even complex data sets) Robust (independent to other tests) Data factories

54 Data Factories 54 Blueprint for sample instances Rails tool support FixtureReplacement ( factory_girl ( Machinist ( ObjectDaddy ( Similar structure Syntax for creating the factory blueprint API for creating new objects We ll use factory_girl ;)

55 Defining Factories 55

56 Defining Factories 56 One most simplistic factory <model_name> for each class Put in test/factories.rb spec/factories.rb test/factories/*.rb spec/factories/*.rb with * = <model_name>

57 Using Factories 57 Build strategies: build, create standard, attributes_for, stub

58 Attributes 58

59 Associations 59

60 Associations 60

61 Inheritance 61

62 Sequences for Unique Values 62

63 Sequences for Unique Values 63

64 Callbacks 64 after_build - called after a factory is built (via Factory.build) after_create - called after a factory is saved (via Factory.create) after_stub - called after a factory is stubbed (via Factory.stub)

65 Agenda 65 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Test Data Test Doubles Introduction Stubs in Detail Mocks in Detail Setup and Teardown Model Tests View Tests Controller Tests Testing Tests Outlook

66 Isolation of Test Cases 66 Tests should be independent New bug in a model only tests related to this model should fail How to achieve this? Don t share complex test data Don t use complex objects Steve Freeman, Nat Pryce: Growing Object-Oriented Software, Guided by Tests

67 Test Doubles 67 Fake objects used in place of real ones Purpose: automated testing Used when real object is unavailable real object is difficult to access or trigger following a strategy to re-create an application state limiting scope of the test to the object/method currently under test

68 Verifying Behavior During a Test 68 Usually: test system state AFTER a test With test doubles: test system behavior!

69 Stubs vs. Mocks 69 Stub (passive) Returns a predetermined value for a method call Does not actually call the method thing.stubs(:name).returns( Fred ) Mock (more aggressive) In addition: set an assertion If expectation is not met test failure

70 Why to have Mocks? 70 Makes sense? Makes more sense?

71 Ruby Test Double Frameworks 71 FlexMock ( Mocha ( DoubleRuby ( Rspec ( We ll use Mocha ;)

72 Agenda 72 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Test Data Test Doubles Introduction Stubs in Detail Mocks in Detail Setup and Teardown Model Tests View Tests Controller Tests Testing Tests Outlook

73 Stubs 73 Replacement for one or many parts of an object Normal method call is not happening Returns a predefined value if called You can only call stubby.name or stubby.weight Else: error Or: stub_everything(...) nil

74 Stubbing Instances 74

75 Stubbing Classes 75 A specific instance is returned Database is not touched find cannot be verified anymore BUT Tests based on find can be isolated just test the logic that is under test

76 Multiple Return Values 76

77 Stub Returns and Raises 77

78 Examples 78

79 Hints for any_instance 79 No guarantee that find returns the exact object you expect any_instance is valid only for instances created after you declared the stub (not for fixture data)

80 Stubs with Parameters 80

81 Stubs with Parameters 81

82 with() Descriptor 82

83 instance_of(), Not 83

84 any_of() 84

85 regexp_matches(), 85

86 Agenda 86 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Test Data Test Doubles Introduction Stubs in Detail Mocks in Detail Setup and Teardown Model Tests View Tests Controller Tests Testing Tests Outlook

87 Mocks 87 Mock = Stub + attitude Demands that mock parameters are called

88 Mocks 88

89 Mocks 89

90 Stub Modifiers are Valid for Mocks 90

91 How often wants a Mock to be Called? 91 By default: one

92 Mock Objects and Behavior-Driven Development 92 Example of a controller test vs.

93 Advantages and Disadvantages 93 Disadvantages Mismatch between mocked model and real model Data type Semantic integration tests Risk to test predefined data (non-sense) Tests might depend on internal structures of mocked object brittle while refactoring Advantages The test is focused on behavior Speed Isolation of tests (failure in model does not affect controller test)

94 Test Double Dos & Don ts 94 You replace an object because it is hard to create in a test environment use a stub minimize number of mocked methods #mocks possibility to run out of sync with real implementation & #mocks test too large? Poor object-oriented design? Don t assert a value you set by a test double (false positives)

95 Agenda 95 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Test Data Test Doubles Setup and Teardown Model Tests View Tests Controller Tests Testing Tests Outlook

96 Setup and Teardown RSpec 96

97 Setup and Teardown RSpec 97

98 Agenda 98 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Model Tests View Tests Controller Tests Routing Tests Outgoing Mail Tests Helper Tests Integration and Acceptance Tests Testing Tests & Hints for Successful Test Design Outlook

99 Model Tests 99 A Rails model accesses data through an ORM implements business logic is fat Model tests Model tests in Rails = Test::Unit + test data + setup/teardown + test logic + additional assertions Easiest tests to write

100 Hints for Model Tests 100 Tests should cover ~100% of the model code Do not test framework functionality like belongs_to Test your validations How many tests? Let tests drive the code perfect fit What comes out? One test for the happy-path case One test for each branch Corner cases (nil, wrong values, ) if appropriate Keep each test small!

101 How many Assertions per Test? 101 If 1 call to a model many changes: #Assertions clarity and cohesion & #Assertions test independece Use context & describe and have 1 assertion per test

102 Test Run 102

103 Example Rspec Tests 103

104 Example Rspec Tests 104

105 RSpec 105 Remainder: we use RSpec

106 Autotest 106 Automate testing with Autotest ( Run autotest rails Integrate with Growl Use FSEvent

107 Agenda 107 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Model Tests View Tests Controller Tests Routing Tests Outgoing Mail Tests Helper Tests Integration and Acceptance Tests Testing Tests & Hints for Successful Test Design Outlook

108 View Tests 108 A Rails view Has only minimal logic Does never call the database Presents the data given by the controller Challenges for view tests Time-intensive How to test look & feel? Brittle w.r.t. re-designs What to do?

109 View Tests 109 Specify and verify logical and semantic structure Goals Validate that view layer runs without error Data gathered by the controller is presented as expected Validate security-based output (e.g., for admins) Do not Validate HTML markup Look & feel

110 Most Basic View Tests 110

111 Keys to Meaningful View Tests 111 Test views semantically with use DOM IDs / CSS classes Do not test actual text Side-effect: you validate your HTML

112 assign() 112

113 should have_selector() 113

114 Agenda 114 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Model Tests View Tests Controller Tests Routing Tests Outgoing Mail Tests Helper Tests Integration and Acceptance Tests Testing Tests & Hints for Successful Test Design Outlook

115 Controller Tests 115 A Rails controller Is skinny Calls the ORM Calls the model Passes data to the view Goal of controller tests Simulate a request Verify the result Subclass of ActionController::TestCase ( and ActiveSupport:TestCase (

116 Controller Tests important variables controller request response Variables for session session[:key] controller variables assigns[:key] flash flash[:key]

117 Controller Tests 117 Methods for get post put delete xhr (Ajax)

118 What to test? 118 Remember: model functionality is tested in model tests Controller tests Verify that user requests trigger model/oer calls that data is forwarded to view Handling of invalid user requests Verifying security roles / role-based access control

119 Structure of a Controller Test 119 Setup Prepare data Prepare request/session Send request to controller Validate controller response

120 Simple Controller Test 120

121 Simple Controller Test 121

122 Structure of a Request 122 <http_verb> :<method>, <parameters>, <session_data>, <text_flash> format.js

123 Some More Examples 123

124 Testing File Uploads 124

125 Background on Controller Tests 125 Controller method is called directly Routes are NOT evaluated Real request parameters are always strings

126 Background on Controller Tests 126 By default, views are not rendered

127 Testing the Controller Response 127 HTTP status code Correct template Assertion methods response.should redirect_to( ) response.should be_success be_redirect response.should render_template( )

128 Agenda 128 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Model Tests View Tests Controller Tests Routing Tests Outgoing Mail Tests Helper Tests Integration and Acceptance Tests Testing Tests & Hints for Successful Test Design Outlook

129 Route Tests 129 route_for params_from

130 Agenda 130 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Model Tests View Tests Controller Tests Routing Tests Outgoing Mail Tests Helper Tests Integration and Acceptance Tests Testing Tests & Hints for Successful Test Design Outlook

131 Outgoing Mail Tests 131 What to validate? Application sends mail when expected content is what you expect Enable testing Uncomment line 26 in config/environments/test.rb config.action_mailer.delivery_method = :test In mail test setup: ActionMailer::Base.deliveries.clear

132 First Steps in Outgoing Mail Tests 132

133 Validating Content 133 Applied to the body of each in ActionMailer::Base.deliveries Only text/html s

134 Complete Outgoing Mail Test 134 More insights about mail spec at

135 Mail Tests in RSpec & Cucumber spec ( Installation

136 Cucumber Mail Scenarios 136 Clear the queue (done automatically by _spec) Execute steps that sends an Check the user received an/no/[0-9] s Open the Inspect the contents Interact with the (e.g. click links)

137 Cucumber Mail Scenarios 137 Given a clear queue no s have been sent When... Then I they address should receive an no \d+ s Then I they address should have an no \d+ s Then I they address should receive an no \d+ s with subject subject Then I they address should receive an with the following body:

138 Cucumber Mail Scenarios 138 When I they address opens the last recent one When I they address opens the with subject subject When I they address opens the with text text Then I they should see text" in the subject Then I they should see \regex/ in the subject Then I they should see text" in the body Then I they should see \regex/ in the body

139 Cucumber Mail Scenarios 139 Then I they should see the delivered from text Then I they should see header_text" in the header_name Then I they should see \regex/ in the header_name" header Then I should see it is a multi-part Then I they should see text" in the html part body Then I they should see text" in the text part body

140 Cucumber Mail Scenarios 140 Then I they should see an no \d+ attachments with the Then there should be an no \d+ attachments named filename Then attachment \d+ should be named filename Then there should be an no \d+ attachments of type content_type Then attachment (\d+) should be of type content_type Then all attachments should not be blank Then show me a list of attachments

141 Cucumber Mail Scenarios 141 When I they follow link" in the When I they click the first link in the # Debugging, Rails and OSx ATM since Viewer uses RAILS_ROOT and OSx's 'open' command. Then save and open current Then save and open all text s Then save and open all html s Then save and open all raw s

142 Cucumber Example Scenario 142

143 RSpec Example Test 143

144 RSpec Example Test 2 144

145 RSpec Example Test 2 145

146 Agenda 146 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Model Tests View Tests Controller Tests Routing Tests Outgoing Mail Tests Helper Tests Integration and Acceptance Tests Testing Tests & Hints for Successful Test Design Outlook

147 Helper Tests 147 Helpers are filled with the rest Used as mediator between views and models or views and controllers (Complex) view logic is moved to helpers

148 Helper Tests 148

149 Agenda 149 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Model Tests View Tests Controller Tests Routing Tests Outgoing Mail Tests Helper Tests Integration and Acceptance Tests Testing Tests & Hints for Successful Test Design Outlook

150 Integration Tests 150 Written by a developer for a developer Test communication of controllers via sessions/cookies Verify end-to-end behavior Make controller calls Verify that everything is okay Similar to controller tests, BUT Not tied to one controller 1..n sessions for different users

151 Methods 151 url_for(object_instance) via_redirect post_via_redirect follow_redirect! https! https!(false) host!(epic.hpi.uni-potsdam.de) Set initial state: session[:user_id] = 3

152 Test::Unit 152

153 Multiple Session Example with Test::Unit 153

154 Webrat & Capybara 154 Webrat and Capybara are very similar We ll use Capybara DSL for browsing the Internet Acceptance testing Capybara is case-sensitive (due to Xpath backend)

155 Finding Elements 155 Finding elements label DOM ID form field name Phone Number phone user[phone]

156 10 Capybara Methods 156 attach_file(field_locator, path, content_type = nil) check(field_locator) choose(field_locator) click_button(value) click_link(text_or_title_or_id, options = {}) fill_in(field_locator, options = {}) save_and_open_page() select(option_text, options = {}) uncheck(field_locator) visit(url = nil, http_method = :get, data = {})

157 Clarity 157

158 Cucumber 158 Features Gherkin Title Narrative Scenarios Title Steps cucumber command Parses steps Map them to step definitions Step definition Written in Ruby (or other languages) implements a step

159 Gherkin 159 Feature Background Scenario Scenario Outline Scenarios Given When Then And / But #

160 Predefined Steps 160 Given I am on the homepage <RESTresource>s page When I go to (.+) When I press button (within selector ) When I follow link (within selector ) When I fill in field" with value (within selector ) When I fill in value" for field (within selector ) When I fill in the following: Account Number 5002 Expiry date Note Nice guy

161 Predefined Steps 161 When I select value" from field (within selector ) When I check uncheck field (within selector ) When I choose field (within selector ) When I attach the file path" to field (within selector ) Then I should see JSON: Then I should (not) see text (within selector") Then I should (not) see \regex/ (within selector )

162 Predefined Steps 162 Then the field" field (within selector ) should (not) contain value Then the label" checkbox (within selector ) should (not) be checked Then I should be on the <RESTresource>s page Then I should have the following query string: Then show me the page

163 Declarative vs. Imperative 163 Scenario: transfer money (declarative) Given I have $100 in checking And I have $20 in savings When I transfer $15 from checking to savings Then I should have $85 in checking And I should have $35 in savings [RSpec Book] Scenario: transfer money (imperative) Given I have $100 in checking And I have $20 in savings When I go to the transfer form And I select "Checking" from "Source Account And I select "Savings" from "Target Account And I fill in "Amount" with "15 And I press "Execute Transfer Then I should see that I have $85 in checking And I should see that I have $35 in savings

164 Organizing Features 164./features/*.feature./features/<epic>/*.feature cucumber features Cucumber features/<epic>

165 Tagging 165 Passing Pending Work in progress

166 Tags @piw AND: cucumber OR: cucumber --tags NOT: cucumber Run certain scenarios / features Only run in certain environments Relate epics

167 Calling Steps From Steps 167 When /I transfer (.*) from (.*) to (.*)/ do amount, source, target When "I select #{source} as the source account When "I select #{target} as the target account When "I set #{amount} as the amount When "I click transfer end When /I transfer (.*) from (.*) to (.*)/ do amount, source, target steps %Q{ } end When I select #{source} as the source account And I select #{target} as the target account And I set #{amount} as the amount And I click transfer

168 Background 168 Feature: invite friends Background: Logged in Given I am logged in as "Aslak And the following people exist: name friend? David yes Vidkun no Scenario: Invite someone who is already a friend Scenario: Invite someone who is not a friend Scenario: Invite someone who is not a friend who doesn't have an account

169 Multi-Line Text 169 Scenario: pending implementation Given a file named "example_without_block_spec.rb" with: """ describe "an example" do it "has not yet been implemented end """ When I run "spec example_without_block_spec.rb Then the exit code should be 0 And the stdout should include """ Pending: an example has not yet been implemented \(Not Yet Implemented\).\/example_without_block_spec.rb:2 Finished in ([\d\.]*) seconds 1 example, 0 failures, 1 pending """

170 Cucumber Configuration 170 config/cucumber.yml Define profiles wip: features cucumber p wip Define default format

171 Capybara and Ajax 171 Capybara uses Celerity ( or Culerity ( To run JS without a browser

172 Capybara and Ajax 172

173 Agenda 173 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Testing Tests & Hints for Successful Test Design Outlook

174 Testing Tests 174 Test coverage Fault seeding Mutation testing

175 Tests Coverage 175 Most commonly used metric for evaluating test suite quality Test coverage = executed code during test suite run / all code * loc / 100 loc = 85% test coverage 1. Absence of code coverage indicates a potential problem 2. Existence of code coverage means very little 3. In combination with good testing practices, coverage might say something about test suite reach

176 How to Measure Coverage? 176 Most useful approaches Line coverage Branch coverage Tool Rcov ( Uses line coverage 100% code coverage although 1 branch wasn t executed

177 RCov swt2-ci/swt2-ci/job/hassocrm/57/ rcov/?

178 RCov 178

179 How Much is Enough? % code coverage says nothing 0% says much Almost 100% is a side effect of BDD

180 5 Habits of Highly Successful Tests 180 Independence of external test data of other tests (or test order) Repeatability Same results each test run Problems date (Timecop) random numbers (try to avoid them)

181 5 Habits of Highly Successful Tests 181 Clarity Test purpose should be immediately understandable Readability How does the test fit into the larger test suite? Worst case:

182 5 Habits of Highly Successful Tests 182 Clarity Better: Debugging is harder than coding Tests should be simple

183 5 Habits of Highly Successful Tests 183 Conciseness Use the minimum amount of code and objects Clear beats concise Writing the minimum amount of tests tests will be faster

184 5 Habits of Highly Successful Tests 184 Robustness Tests the logic as intended Code is correct tests passes Code is wrong test does not pass Example: view testing vs.

185 5 Habits of Highly Successful Tests 185 Robustness But be aware of false positives

186 Troubleshooting 186 Reproduce the error What has changed? Isolate the failure thing.inspect (p thing) Add assertions/prints to your test Rails.logger.error save_and_open_page Explain to someone else

187 Fault Seeding 187 Introduce a fault into your program Run tests Minimum 1 test should fail Warning: do not leave the fault in the software!

188 Mutation Testing 188 Mutant: Slightly modified version of the program under test, differing from it by a small, syntactic change To create mutants, replace: if if not = < Test Cases should pass on should fail on Program Mutants

189 Mutation Testing 189 Ruby tool: Heckle ( 1. Your tests should pass 2. You run Heckle to change your code 3. Test(s) should fail 4. Write tests for surviving mutants if useful

190 Agenda 190 Behavior-Driven Development of MasterMind Why Behavior-driven Design (BDD)? Building Blocks of Tests and BDD Testing Tests & Hints for Successful Test Design Outlook

191 Outlook 191 Agile Scrum in depth Project Feedback Agile Product Management

192 192 Thank you for your attention!