Maja Schreiner 9th Lean, Agile & Scrum Conference 2017
Senior Test Master @ Swisscom, Switzerland maja.schreiner@gmail.com testmotion.wordpress.com Twitter: majaschreiner
process of executing many different types of tests both manual and automated continually throughout the delivery process as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.
Appium freely distributed open source mobile application UI testing framework allows hybrid, native, and web app testing cross-platform app testing by using the single API for both Android and ios platform test scripts BDD testing tools: Jbehave SpecFlow Cucumber Taurus An abstraction layer over JMeter (or Grinder or Gatling or Selenium) https://www.blazemeter.com/blog/how-automate-jmeter-and-selenium-testing Selenium
Before the Sprint High-level keyword-driven automated tests writen with requirements During the Sprint Unit Tests Acceptance tests (incl. User scenarios) part of the DoD After the Sprint User and usage monitoring in production can deliver lots of results
It is vital that developers own the acceptance tests. -> an essential part of the Continuous Delivery feedback loop. Implement What the system under test should do, not How it does it. Do focus on short feedback loops (roughly 5 minutes for commit stage tests and 45 minutes for acceptance tests)
A unique way of involving crowd i.e. the real users/testers, into software testing under real world conditions.
Fast feedback user-centric software no confirmation biases end users live in the whole world.
Test coverage scaling testing resources saving on a budget
3 Scrum development teams 1 tester Decision for crowd sourced testing: time frame / deadline / resources scaling reasons
Prepare -> Run -> Analyze
Quality of test results Developers rejecting lots of bugs Distrusting CT Coordination & communication time needed
Decide where to start Continuously adjust your strategy Stick to timeboxed time Track improvements (process, results) Build trust Add value
Input Test scope / products Test plan (regression) Known issues list Internal release notes Sprint review video demo & presentation Roadmap All other needed information Output -> test results including featuring screenshots, video recordings, and crash reports -> test report (dashboard: results summary per Risk, Devices, Browsers, technical or functional etc.)
Know your testing goal & purpose What How When do you want to test (every day, middle of sprint, end of sprint) Integrate developers in the test cycle and even in the weekly sync calls The whole team approach Foster the communication with crowdsourced testers Star and reward the good testers Keep the best testers in every cycle Invite new testers fresh point of view
exploratory tests add lots of value and definitely more than regression: testers are more motivated and creative generally, more bugs found sometime regression is done "too fast" and some bugs get overlooked and missed Don t expect CT to solve all your testing, QA or software development problems Don t forget to test internally as well Beware the costs
Pros for crowdsourced testing Specialized skill sets Scale capacity to meet variable demand Increases fluidity of workforce Keeps fixed cost off balance sheet Good for functional end-user testing of mobile and web apps Contra for crowdsourced testing Not so applicable for the applications which require extensive business know-how
Why and how Strategy Challenges How to succeed What to keep in mind
Please contact me for any further questions: Maja Schreiner, Senior Test Master, Swisscom, Switzerland maja.schreiner@gmail.com testmotion.wordpress.com Twitter: majaschreiner