Testing => Good? Automated Testing => Better?

Survival techniques for your acceptance tests of web applications Julian Harty Senior Test Engineer 2009 Google Inc 1 Introduction Testing => Good? Automated Testing => Better? 2 1

Introduction: Focus Unit Tests => Micro Acceptance Tests => Macro 3 Introduction: Web applications Uses: Web browser Client: AJAX Server: Your choice Protocol: HTTP UI: Rich 4 2

Aims and Drivers Of our tests 5 Goals for Acceptance Tests 1Safety rails 2Early warning 3 Good enough 4Code your assumptions 5Fail early 6 3

Types of tests Large (Acceptance) Medium Small (Unit) Quantity of tests 10/20/70 split Resources & Duration See: GTAC 2008: The value of small tests http://www.youtube.com/watch?v=mpg2i_6nkug GTAC 2008: Taming the Beast - How to Test an AJAX Application http://www.youtube.com/watch?v=5jjrtbfzwgk 7 Cost benefit analysis Return on investment Over the project lifetime Opportunity cost During my involvement 8 4

Test automation challenges 9 Test automation challenges: Window of opportunity Tests need to detect relevant states / changes Possible sources of events Input: Click a button Clock: Calendar reminder for 1 minute Server: new email message time event wait reaction too late Window of opportunity When the system is in the correct state 10 5

Test automation challenges: Polling Regular polling by test code Check / Sleep loop Look for expected behaviour Checking for problems helps tests to fail fast time event wait reaction too late Polling Polling Polling Polling Polling Polling Timeout 11 Test automation challenges: heavyweight tests Time taken for an acceptance test Open browser Login Navigate Do stuff Logout Close browser Preparation Body of the test Cleanup 12 6

Test automation challenges: What and How Separate what from how Encapsulate how in one place Consider the PageObject pattern Represents a web page Presents services to callers Encapsulates nitty-gritty details PageObject pattern: http://code.google.com/p/webdriver/wiki/pageobjects 13 How to create acceptance tests 2009 Google Inc 14 7

2 audiences Users Model activities Send a message Order a book Decompose activities into actions Open browser Login with valid account Select [Compose Message] Enter a Subject Enter message Select [Send] Developers Homogenous language and tools Java -> JUnit Python -> PyUnit Add tests to their codebase Get them to review your test code Reduce effort required to a minimum 15 Given, When, Then // Given I have a valid user account and am at the login page, // When I enter the account details and select the Enter button, // Then I expect the inbox to be displayed with the most recent email selected. Given, When, Then - credited to: Joe Walnes & Dan North 16 8

Coding the tests Create a skeleton of the program methods Have the skeleton reviewed by developers and users Use Interfaces in Java Use the PageObject pattern Write tests for support code Some tests may include JavaScript e.g. to rotate the UI for iphone applications PageObject pattern: http://code.google.com/p/webdriver/wiki/pageobjects 17 Navigation options Brittle xpath /html/body/div/div[5]/div[3]/div/div[2]/div class id < class= contacts-entry-name roster /> < id = first-contact /> Multiple items Unique 18 9

Selenium IDE screenshot Simple recorder Doesn t capture everything Brittle scripts 19 Firebug screenshot Firebug: http://getfirebug.com/ 20 10

Practical tests Bugs come from many sources They need to be tested Regression tests detect any that reappear Code bug reports as acceptance tests Make sure the new acceptance test fails first Fix the application Re-run the acceptance tests Antony Marcano pioneered the idea of creating acceptance tests in place of bug reports http://www.testingreflections.com 21 Example WebDriver script // Create a new instance of webdriver WebDriver driver = new HtmlUnitDriver(); // Go to the Google home page, following any redirects driver.get("http://www.google.com"); // Now find the element with the name "q" WebElement onebox = driver.findelement(by.name("q")); // And type the word "webdriver" into it onebox.sendkeys("webdriver"); // Now submit the form onebox.submit(); // And check the title of the destination page contains "WebDriver" asserttrue("title should contain webdriver, but contains: " + driver.gettitle(), driver.gettitle().contains("webdriver")); 22 11

Increasing the velocity Turn up the volume! 2009 Google Inc 23 Fast, automatic feedback Run tests after changes are submitted Use build scripts Continuous builds Highly visual displays Browser toolbar 24 12

Writing better tests, faster Fluid test implementation Add id and class elements to relevant links Take advantage of keyboard shortcuts Add debug panels to the application Use good test design patterns Use good test automation tools e.g. WebDriver 25 Vary the tests Keep good records Finding near-by bugs Data driven tests Use randomisation 26 13

Vary the browsers 27 Emulating mobile browsers private static final String IPHONE_USER_AGENT_V1_1 = "Mozilla/5.0 (iphone; U; CPU like Mac OS X; en) AppleWebKit/420.1 " + "(KHTML; like Gecko) Version/3.0 Mobile/3B48b Safari/419.3"; /** * Returns the WebDriver instance with settings to emulate an iphone V1.1 */ public static WebDriver createwebdriverforiphonev1_1() { final String emptystring = ""; FirefoxProfile profile = new FirefoxProfile(); // blank out headers that would otherwise confuse the web server. profile.setpreference("general.appversion.override", ""); profile.setpreference("general.description.override", ""); profile.setpreference("general.platform.override", ""); profile.setpreference("general.vendor.override",""); profile.setpreference("general.vendorsub.override",""); profile.setpreference("general.appname.override", "iphone"); profile.setpreference( "general.useragent.override", IPHONE_USER_AGENT_V1_1); WebDriver webdriver = new FirefoxDriver(profile); return webdriver; } Http Headers online http://www.pycopia.net/webtools/headers 28 14

Robust tests 1 Operate correctly when unimportant things change 2 Ductile, not brittle 3 Contain Guard conditions to ensure your assumptions still hold 4 Fail visibly and early e.g. at compile time 29 Informative tests Screenshots Record evidence Debug traces Logs System.out.println( Made it here ); 001 I Starting 1 user test. 001 W Image(logo.png) missing 001 E Login failed! 001 W Aborting login test. 001 I Starting 2 user test. Good error messages: say what s expected Useful, but not always in sync 30 15

Bit-rot Half-life, shared by Radioactive elements Our tests! Our tests are likely to suffer from bit-rot Their usefulness declines They start to break Active maintenance vital to cure bit-rot and maintain potency Bit rot http://en.wikipedia.org/wiki/bit_rot 31 Wrap-up Over to you? 32 16

Gaining skills Apprenticeship & Mentoring Pairing with developers & practitioners Dive-in Try out examples Experiment Learn the underlying technologies Practice often http://code.google.com/p/webdriver/wiki/gettingstarted AJAX 33 Peripheral topics Accessibility and Testability need ways to Interrogate Interact with Interpret the contents of web applications Accessibility Testability Find ways to improve both Fire Vox, a screen reader for Firefox. http://firevox.clcworld.net/about.html 34 17

Thank You! Q&A 35 Further information (1) Pragmatic Test Automation guide http://www.pragprog.com/titles/auto/pragmatic-project-automation JUnit in Action http://www.manning.com/tahchiev/ (2nd edition, early access) http://www.manning.com/massol/ (1st edition) JUnit Recipes http://www.manning.com/rainsberger/ AJAX resources http://bulletproofajax.com/ An incredibly good book on how to write good AJAX code. It starts with the basics and builds reliably and clearly from good foundations. The DOM manipulation code is relevant for implementing your acceptance tests in tools such as WebDriver. Another good book is Building a web site with Ajax http://www.dmcinsights.com/ajax/ again a book that starts simple and builds a simple application step by step. 36 18

Further information (2) GTAC 2008: The value of small tests http://www.youtube.com/watch?v=mpg2i_6nkug GTAC 2008: Taming the Beast - How to Test an AJAX Application http://www.youtube.com/watch?v=5jjrtbfzwgk Acceptance tests are more A+S than T+G (Antony Marcano) http://www.testingreflections.com/node/view/6704 A+S => Activities + Specific T+G => Tasks + General A little abstraction when testing software with Selenium-RC and Java (Alan Richardson) http://www.eviltester.com/index.php/2008/03/08/a-little-abstractionwhen-testing-software-with-selenium-rc-and-java/ How will you know? A generalised model for User Acceptance Testing (UAT) (Alan Richardson) http://www.eviltester.com/index.php/2008/03/09/ageneralised-model-for-user-acceptance-testing-uat/ 37 19