A CONFUSED TESTER IN AGILE WORLD QA A LIABILITY OR AN ASSET THIS IS A WORK OF FACTS & FINDINGS BASED ON TRUE STORIES OF ONE & MANY TESTERS!! J Presented By Ashish Kumar,
A STORY OF TESTING. WHAT S AHEAD FROM THE MIND OF A CONFUSED TESTER. FEW CASE STUDIES. CHALLENGES IDENTIFIED. SURVEY STUDIES. GLOBAL RESPONSES. SOLUTION APPROACH. PRINCIPLES AND PRACTICES. CONCLUSION & RECAP. Q & A.
A STORY OF TESTING IN AGILE
HAVE YOU HEARD ANY OF THESE?? YOU DON T NEED A DEDICATED SOFTWARE TESTING TEAM ON YOUR AGILE TEAMS IF WE HAVE BDD,ATDD,TDD,UI AUTOMATION, UNIT TEST >> WHAT IS THE NEED OF MANUAL TESTING?? WE WANT 100% AUTOMATION IN THIS PROJECT TESTING IS BECOMING BOTTLENECK AND REASON OF SPRINT FAILURE REPEATING REGRESSION IS A BIG TASK AND AN OVERHEAD MICROSOFT HAS NO TESTERS NOT EVEN GOOGLE, FACEBOOK AND CISCO
IN A MOBILE-FIRST AND CLOUD-FIRST WORLD. THE EFFORT, KNOWN AS AGILE SOFTWARE DEVELOPMENT, IS DESIGNED TO LOWER COSTS AND HONE OPERATIONS AS THE COMPANY FOCUSES ON BUILDING CLOUD AND MOBILE SOFTWARE, SAY ANALYSTS MR. NADELLA TOLD BLOOMBERG THAT IT MAKES MORE SENSE TO HAVE DEVELOPERS TEST & FIX BUGS INSTEAD OF SEPARATE TEAM OF TESTERS TO BUILD CLOUD SOFTWARE. SUCH AN APPROACH, A DEPARTURE FROM THE COMPANY S TRADITIONAL PRACTICE OF DIVIDING ENGINEERING TEAMS. WOULD MAKE MICROSOFT MORE EFFICIENT, ENABLING IT TO CUT COSTS WHILE BUILDING SOFTWARE FASTER, EXPERTS SAY. 15K+ DEVELOPERS /4K+ PROJECTS UNDER ACTIVE DEVELOPMENT/50% CODE CHANGES PER MONTH. 5500+ SUBMISSION PER DAY ON AVERAGE 20+ SUSTAINED CODE CHANGES/MIN WITH 60+PEAKS 75+ MILLION TEST CASES RUN PER DAY. DEVELOPERS OWN TESTING AND DEVELOPERS OWN QUALITY. GOOGLE HAVE PEOPLE WHO COULD CODE AND WANTED TO APPLY THAT SKILL TO THE DEVELOPMENT OF TOOLS, INFRASTRUCTURE, AND TEST AUTOMATION. DEVELOPER SKILLS AND A TESTER MINDSET. GOOGLE PERFORMS A GREAT DEAL OF MANUAL TESTING, BOTH SCRIPTED AND EXPLORATORY, BUT EVEN THIS TESTING IS DONE UNDER THE WATCHFUL EYE OF AUTOMATION. Source: Wall Street Journal: http://blogs.wsj.com/cio/2014/07/15/microsog plots agile development course as talk on job cuts loom/ Mico J Tools for Continuous Integration at Google Scale https://www.youtube.com/watch?v=kh2_sb1a6la&feature=youtu.be How Google test Software :James W, Jason A, Jeff C
FROM THE MIND OF A CONFUSED TESTER
IS QA PART OF THE DEVELOPMENT TEAM? CAN WE FIT QA IN THE SAME ITERATION AS DEVELOPMENT? SHOULD I FOCUS ON MANUAL OR AUTOMATION HOW CAN WE SCALE AGILE QA? WHO DOES QA? DOES QA COSTS MORE IN AGILE AS PRODUCT SEEMS TO CHANGE FROM SPRINT TO SPRINT? DO WE NEED TEST PLAN? ARE STORY ACCEPTANCE TESTS ENOUGH? WHEN DO WE KNOW TESTING IS DONE? WHO DEFINES TEST CASES? DO WE NEED TO TRACK BUGS?
TEST ENGINEERING @ GOOGLE ITS NOT QA
QA AND AGILE ARE INEXTRICABLY INTERTWINED. BUT QUITE OFTEN IN AGILE ORGANIZATIONS, THE ART OF QA IS NOT WELL UNDERSTOOD. THE VERY ESSENCE OF AGILE DEVELOPMENT IS DELIVERING QUALITY WORKING SOFTWARE FREQUENTLY. IN AGILE PROJECTS, QA SHOULD BE EMBEDDED IN THE SCRUM TEAMS BECAUSE TESTING AND QUALITY IS NOT AN AFTERTHOUGHT. QUALITY SHOULD BE BAKED IN RIGHT FROM THE START.
Source : https://www.linkedin.com/pulse/agile-just-tech-elite-david-akka
Case 1 Project Description QA Roles QA Approach Challenges Type: Enhancement and Maintenance Project ; Domain : Core banking Team Size: 40 ; With Agile : < 5 years No testers on Team 1. Whole Team Approach over Testing Departments and Independent Testing 2. Developers perform Automation and Cross developed verification. 3. TDD 4. Developers Develops Unit test Case > Story Development > Functional Automation Test Case> Exploratory testing > Done 1.Hiring testers who can code features is difficult; finding feature developers who can test is even more difficult. 2. Maintenance is a BIG challenge 3. Non-functional testing during sprint is a challenge One Query Why Should we pay more for Manual testing The 'whole team' approach has helped in instilling sense of Inclusiveness within the team. It has also helped in reducing delays & improved the overall team efficiency. It is been a paradigm shift for many.
Case 2 Project Description QA Roles QA Approach Type: Development Project ; Domain : Finance With Agile : < 3+ years 1. Cross-Functional Team 2. Functional Tester Performing both the task of Manual validation and Automation 1. ATDD. 2. Team together works on test scenarios > Dev develop the stories QA Develop test case > QA automate the test scenario Developers pitch in for help > BA Validates > Done 3. Whole Team Approach, Developers also supports QA to perform Automation and Verification Challenges One Query 1. Shortened time for testing 2. Sub-standard delivery of few stories towards the end of sprint 3. Spill Over 4. Testing backlog creation Why should we duplicate the effort by having separate roles as manual and automation testers. We believe in spirit of agile, it was difficult to break the shackles of mindset and create an effective whole team approach. But it works wonder for us although we have lot of scope for improvements
Case 3 Project Description Type: Mission Critical Products; With Agile : 5+ years. More inline with DAD approach QA Roles Manual testers as Part of scrum team ( Work as product experts) Automation testers distributed among different teams Field Engineers along with PO does UAT, Regulatory Testing etc.. QA Approach Component to verify :- Hardware, Firmware, Application Software Application :- Automate, Interface :- Automate, Portion of Firmware and H/W :- Automate Unit and Integration Testing by developers. System Integration and System Testing by QA Because of complex integration and system dependency, dedicated hardening sprints at the end Challenges Sometime there is lag in automation. Risk based testing as all configurations can t be testing before release Work load is uneven for Manual Test team. One Query Can we make the non-functional test also a part of sprint, if yes how? Quality is everyone responsibility. Agile has made it true. It's not Developers or QA who is owning but right from customer everyone is building Quality in the product
Case 4 Project Description Type: Development Project Team Size: 45 ; With Agile : 1~2 years QA Roles 1. Separate Testing team / Vendor for QA QA Approach Challenges 1. Development Sprint and QA sprint are separate. 2. Both the Sprint have different sprint goals and deliverables. 3. QA Sprint always lag by one dev sprint 4. They work on current sprint test scenarios and verifying previous sprint deliverables. 5. All the QA activities Functional and Non-functional are taken care in QA sprint. 1. Teams working in Silos 2. The approach is very much waterfall 3. Defects and issues found is QA sprint are part of product backlog. One Query Why do we need to release sub standard builds in every sprint. We Effectively synchronized QA activity on a distributed development model with dedicated QA-Dev pairing. Did not reduce QA s Importance to unit test dev s substandard build.
A Sneak Peak into the past.. https://www.scrumalliance.org/community/articles/2015/june/a-confused-tester-in-agile-world
LETS IDENTIFY SOME MORE CHALLENGES
CHALLENGES IDENTIFIED CHANGING REQUIREMENTS / LAST MINUTE CHANGES NOT ENOUGH INFORMATION ON THE STORY CONTINUOUS TESTING TECHNICAL SKILLS / TEST AUTOMATION MULTIPLE BROWSERS / MULTIPLE DEVICES COMMUNICATION :: TO PRODUCE AND COMMUNICATE RELEVANT INFORMATION PROMPTLY FEAR TO LOSE IDENTITY COLLABORATION :: TO MAKE TESTING, DEVELOPMENT AND BUSINESS COLLABORATE HOW TO KEEP UP WITH THE PACE OF THE DEVELOPMENT? HOW TO TEST EARLY BUT NOT DO ANTICIPATORY TEST DESIGN?
STUDY OF AGILE PRACTICES IMPLEMENTATION IN DISTRIBUTED SOFTWARE DEVELOPMENT A REFERENCE This is a research conducted by Manjunath M S Rao, Vijay Wade and M M Jha. This Paper presents the results of a systematic study of implementation of agile practices, which covers the summary of most effectively implemented practices, most widely recommended practices and least implemented practices in Global Software Engineering (GSE). The findings are based on the survey data collated from 22 agile practitioners from 14 different software organizations spread across the globe. 1 to 2 14% 2 to 5 years 57%
FEW SAMPLE SURVEY QUESTIONS 1. ARE RELEASE BACKLOGS BUILT WITH THE INVOLVEMENT OF THE RELEVANT STAKEHOLDERS ( PRODUCT OWNER, SALES/MKT, PRODUCT MANAGER, ARCHITECTS, BUSINESS ANALYSTS, SYSTEM TESTING. ETC..)? 2. IS THE TEAM CROSS FUNCTIONAL AND INDEPENDENT TO DELIVER A FUNCTIONAL SOFTWARE(STORY) WITHIN A SPRINT? 3. ARE RISKS AND ISSUES GETTING TRACKED WITHIN SPRINTS? 4. HAVE YOU IMPLEMENTED XP PRACTICES LIKE TEST DRIVEN DEVELOPMENT, PAIR PROGRAMMING ETC? PLEASE PROVIDE DETAILS IN REMARKS 5. DO YOU HAVE A SETUP TO HANDLE CONTINUOUS INTEGRATION AND DELIVERY TO MAINTAIN THE PACE OF DELIVERY? 6. ARE YOU USING AUTOMATION TO OPTIMIZE EFFORT AND TO IMPROVE PRODUCT QUALITY? 7. IS THE DELIVERABLE AT THE END OF THE SPRINT / ITERATION READY TO BE SHIPPED?( IS THERE A SEPARATE TESTING PHASE OR DELIVERABLE FROM EACH SPRINT IS READY TO SHIP?)
8 8 6
IS QA AN ASSET ON YOUR TEAM?? J J Response 1.Always 2.Some+me 3.Not-Done 4.NA Recommenda0ons 1.Strongly recommended 2. Recommended 3.Not recommended 4.Fine Tune
RESPONSE AND RECOMMENDATION Fine Tune 14% Sometimes 36% Always 64% Strongly Recommended 45% Recommended 41%
Source: http://cartoontester.blogspot.in/2010_01_01_archive.html
If Time permits - Appendix
PRINCIPLES AND PRACTICES TESTING MOVES THE PROJECT FORWARD TESTING IS NOT A PHASE ON AGILE TEAMS, TESTING IS A WAY OF LIFE. CONTINUOUS TESTING IS THE ONLY WAY TO ENSURE CONTINUOUS PROGRESS. EVERYONE TESTS WHOLE TEAM APPROACH COLLABORATION SHORTENING FEEDBACK LOOPS KEEP THE CODE CLEAN LIGHTWEIGHT DOCUMENTATION TEST-LAST V. TEST-DRIVEN
Source: Quality Tree Software, Inc
Source: Quality Tree Software, Inc
Source: Quality Tree Software, Inc
Q1: In Agile Methodology, Some think that with focus more on complete automation and continuous integration, the software testing job will become less important or obsolete. What's your Take on This?? Source: https://www.linkedin.com/groups/961927/961927-6145911947582808065?trk=hb_ntf_commented_on_group_discussion_you_created#commentid_6145995769821302784 https://www.linkedin.com/groups/49087/49087-6145911731643260933?trk=hb_ntf_liked_group_discussion_you_created https://www.linkedin.com/groups/99444/99444-6145911085917564931?trk=hb_ntf_liked_group_discussion_you_created
Q2 : Contrary to the synchronous activities of traditional waterfall project, agile expects development action to be performed in order they are needed more of asynchronous we can say. Now what are the different ways for a testing professional to engage EFFECTIVELY during a sprint before any feature has been built? Source; https://www.linkedin.com/groups/55636/55636-6146759619357265922?trk=hb_ntf_liked_group_discussion_you_created
Q3 : How may of you think that S/W quality is compromised due to shortened period allotted for testing with major emphasis on development. Also if you can please add reasons for this & correction measures. Source; https://www.linkedin.com/groups/55636/55636-6146757659057020932?trk=hb_ntf_liked_group_discussion_you_created
Q4: During My Pursuit of identifying the approach different teams are following for testing in an agile environment I came across some following cases, (The details given are brief)
REFERENCES https://www.scrumalliance.org/community/articles/2015/june/a-confused-tester-in-agile-world
Life is a sprint. If you start on the right note, you enjoy it till the end, and win at the last. @A_ashish_kumar ashish.ashok.kumar@gmail.com