Introduction to Automated Acceptance Testing

Transcription

1 Introduction to Automated Acceptance Testing Micah Martin, 8th Light, Inc.

2 What are Acceptance Tests?

3 What are Acceptance Tests? ATs vs UTs Acceptance Tests Unit Tests Written by Customer Test Features Written by developers Test Code

4 What are Acceptance Tests? Criteria for Completion Provided by the Customer Given to developers Specify stories Criteria for Done-Done

5 What are Acceptance Tests? Communication Forces customers, testers, and developers to work together creation implementation execution

6 What are Acceptance Tests? Feedback Are a source of data A tool for project management

7 Data From Acceptance Total AT's Failing AT's Passing AT's

8 Acceptance Tests: A Critical Piece of Agile

9 Acceptance Tests: A Critical Piece of Agile Traditional Approach 1 May 1 Jul 1 Sep 1 Nov Analysis Design Implementation Object Mentor DFD ERD DD ST

10 Acceptance Tests: A Critical Piece of Agile The Agile Aproach May 1 Jul 1 Sep 1 Nov 1 Analysis Design Implementation

11 Acceptance Tests: A Critical Piece of Agile The Agile Approach Failing Iteration Acceptance Tests Passing Acceptance Tests

12 Acceptance Tests: A Critical Piece of Agile The Agile Approach Failing Iteration Acceptance TDD Tests Passing Acceptance Tests

13 Acceptance Tests: A Critical Piece of Agile Criteria for DONE Every story must have at least one Acceptance Test A story is not DONE until it passes it s Acceptance Tests

14 Acceptance Tests: A Critical Piece of Agile Manual Acceptance Tests

15 Acceptance Tests: A Critical Piece of Agile Manual Acceptance Tests Manual Acceptance Tests

16 Acceptance Tests: A Critical Piece of Agile Manual Acceptance Tests Manual Acceptance Tests

17 Acceptance Tests Are Automated

18 Acceptance Tests: A Critical Piece of Agile The Button How often would you press it? When would you press it? Who would press it? Testers, Developers, Managers, Customers, Spectators, etc.

19 Acceptance Tests: A Critical Piece of Agile Automated Tests Require: High level, yet, precise language Unambiguous detail Examples Interaction with the system

20

21 Criteria for DONE

22 Criteria for DONE +

23 Criteria for DONE + Automated

24 Criteria for DONE + Automated Executable Specification

25 Acceptance Tests: A Critical Piece of Agile Executable Specification A new paradigm for testing Puts quality first Removes ambiguity from requirements

26 Who Writes Acceptance Tests?

27 Who Writes Acceptance Tests? The Customer Yeah right! Then who? The Customer Role Stake holder Business Analyst Quality Assurance Product Owner

28 Who Writes Acceptance Tests? Tests Get Technical The Customer may need technical help to write tests Developers are technical Pair test authoring

29 Who Writes Acceptance Tests? Business Rules Get Fuzzy Sometimes developers need help understanding tests Customers know business rules Pair test implementation

30 Who Writes Acceptance Tests? Business Decisions are Hard To Make Sometimes, the customer will be challenged to make decisions Stakeholders can help. Developers can help. Do you see a pattern of communication?

31 Exercise #1

32 Exercise #1 The Login Test Write a test plan, in plain text, for the business rules of logging in. Web application User credentials are stored in relational database Successful login redirects to Welcome page

33 Writing Good Acceptance Tests

34 Writing Good Acceptance Tests Login Test Possibilities 1. Direct browser to URL for login page

35 Writing Good Acceptance Tests Login Test Possibilities 1. Direct browser to URL for login page

36 Writing Good Acceptance Tests Login Test Possibilities 1. Direct browser to URL for login page 1. Enter the username wallace

37 Writing Good Acceptance Tests Login Test Possibilities 1. Direct browser to URL for login page 1. Enter the username wallace

38 Writing Good Acceptance Tests Login Test Possibilities 1. Direct browser to URL for login page 1. Enter the username wallace Build a Testable Environment First

39 Writing Good Acceptance Tests BOC Build Operate Check

40 Writing Good Acceptance Tests Login Test Possibilities 1. Add some users to the system

41 Writing Good Acceptance Tests Login Test Possibilities 1. Add some users to the system

42 Writing Good Acceptance Tests Login Test Possibilities 1. Add some users to the system 3. Enter a value into the username field

43 Writing Good Acceptance Tests Login Test Possibilities 1. Add some users to the system 3. Enter a value into the username field

44 Writing Good Acceptance Tests Login Test Possibilities 1. Add some users to the system 3. Enter a value into the username field Be Specific

45 Writing Good Acceptance Tests Tests are Examples Use concrete examples Specify concrete behavior No ambiguity allowed

46 Writing Good Acceptance Tests Login Test Possibilities 1. Insert into User table values ( wallace, ilikecheeze )

47 Writing Good Acceptance Tests Login Test Possibilities 1. Insert into User table values ( wallace, ilikecheeze )

48 Writing Good Acceptance Tests Login Test Possibilities 1. Insert into User table values ( wallace, ilikecheeze ) 2. Open a browser to the URL localhost/myapp

49 Writing Good Acceptance Tests Login Test Possibilities 1. Insert into User table values ( wallace, ilikecheeze ) 2. Open a browser to the URL localhost/myapp

50 Writing Good Acceptance Tests Login Test Possibilities 1. Insert into User table values ( wallace, ilikecheeze ) 2. Open a browser to the URL localhost/myapp Avoid Implementation Details

51 Writing Good Acceptance Tests Avoid Implementation Details Tests UI System Database

52 Writing Good Acceptance Tests Login Test: Possible Solution Add user to system: ( wallace, ilikecheeze ) Process login with username wallace and password blah Check login failed Process login with username wallace and password ilikecheeze Check login succeeded

53 Tools

54 Tools Commercial Tools WinRunner Silk Robot Are not suitable for Acceptance Testing in an Agile environment

55 Tools Open Source Options FIT FitNesse Among the few tools that support Test Driven Development

56 Tools FIT Framework for Integrated Tests Created by Ward Cunningham Open Source The most accepted solution for agile acceptance testing

57 Tools FitNesse Environment build around FIT Makes everything easier Created by Object Mentor, Inc. Open Source

58 FIT FitNesse - Tests written in HTML - Tests are executed on the command line - Tables are executed - Non-table markup is ignored - Tables map to Fixtures - Fixtures are code that is aware of the system - Supplies foundational Fixtures - Implementations ported to many languages - Stand alone web server - Is a wiki - Tests written in wiki text - Tests are executed from within the wiki - Translates tests into HTML - Uses FIT to execute tests - Supports test suites - Supports variables in tests - Supports test refactoring - Written in Java - Supports FIT implementations in any language

59 Thinking in Tables

60 Thinking in Tables Only Tables Execute Ignored Executed

61 Thinking in Tables Foundational Table Structure Name of Fixture Interaction with Application Table structure depends on type of Fixture

62 Thinking in Tables 3 Foundation Fixtures Column Fixture Row Fixture Action Fixture

63 Thinking in Tables Column Fixture package eg; // Copyright (c) 2002 Cunningham & Cunningham // Released under the terms of the GNU Genera Public import fit.columnfixture; public class Division extends ColumnFixture { public float numerator; public float denominator; public float quotient() { return numerator / denominator; } }

64 Thinking in Tables Row Fixture package fitnesse.fixtures; Analogous to comparing against rows in a database table import fit.rowfixture; public class EmployeePayRecordsRowFixture extends RowFixture { public Object[] query() throws Exception { EmployeePayRecord[] records = new EmployeePayRecord[2]; records[0] = new EmployeePayRecord(1, 1000); records[1] = new EmployeePayRecord(2,2000); return records; } public Class gettargetclass() { return EmployeePayRecord.class; } } public class EmployeePayRecord { public int id; private double salary; public EmployeePayRecord(int id, double salary){ this.id = id; this.salary = salary; } public double pay() { return salary; } }

65 Thinking in Tables Action Fixture Think GUI window public class CountFixture extends Fixture { private int counter = 0; public void count() { counter++; } Counter Window Counter: Counter: 6 Count } public int counter() { return counter; }

66 Exercise #2

67 Exercise #2 Login With Tables Write the Login test using tables Keep in mind what we learned from the previous exercise BOC Examples Avoid Implementation Details

68 Exercise #2 Possible Solution

69 FitLibrary Extension to FIT Written by Rick Mugridge Adds some handy Fixtures

70 FitLibrary FitLibrary Fixtures ArrayFixture for ordered lists SetFixture for unordered lists Supports Graphics Tree structures Nested Tables

71 FitLibrary DoFixture Broken tables Text Highly readable Flexibility

72 Wiki

73 Wiki What is it? A collaborative web site Editable by any Created by Ward Cunningham Every project should have one

74 Wiki Creating Tests Use Wiki syntax to create a page with test tables Label the page as a Test Page Use a page name of the form Test Turn on the Test property Make sure your Fixtures are in the classpath Use!path widget Mechanics!path values are concatenated Java command to start FitServer is executed Testable HTML is passed to FitServer FitServer runs the tests Results are passed back to FitNesse

75 Wiki Creating Suites There are 2 ways to make Suites Set the Suite property Create a page with the Suite property Created test pages inside this page When the suite is executed, all child test pages will be included in the suite execution Use the!see widget!see <name of test page> All included tests pages will be included in the suite execution Run a Suite by clicking the Suite button

76 New Advancements

77 New Advancements Slim New Extension to FitNesse Alternative to FIT More Flexible Integration Written by Uncle Bob

78 New Advancements RSpec/Cucumber BDD version of Acceptance Testing Pure Text Very Readable (Given, When, Then) Dan North, David Chelimsky, Aslak Hellesoy Works in Java (JRuby),.NET (Iron Ruby), C (Ruby)

79 The End. Micah Martin, 8th Light, Inc.