Testing. in A Large scale agile Development Environment

Transcription

1 Testing in A Large scale agile Development Environment

2 Content/agenda 1. Introduction 2. Agile on a large scale 3. Testing in a large scale agile environment 4. Conclusion Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 2

3 Introduction Ericsson: The leading vendor for mobile network infrastructure Tough competition on the telecommunication market: Number of competitors Merging of traditional Circuit Switched Technology and Packet Switched Technology towards all IP Network Motivation: Increase the feedback loops towards design Delivery of high quality products to the customer Being flexible to adapt faster to the market s demand Early and Frequent Delivery Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 3

4 AGILE ON A LARGE SCALE Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 4

5 Agile Manifesto for Agile Software Development We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more. Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 5

6 ScRUM IN A NUTSHELL One team collaborating to one feature Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 8

7 large scale Chief Product Owner: 1 Number of Product owners: 5 Number of Scrum teams: 8 Size of the teams: 6-8 Number of Scrum Masters: 8 Number of people: around 70 Number of locations: 4 Number of Releases: 2 Number of manhours: ~ Number of User Stories: ~700 Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 9

8 LARGE SCALE Our legacy: The Ericsson Mobile Soft Switch PLEX statements ASA instructions corresponding to approximately Intel instructions 2196 SW blocks Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 10

9 TESTING IN A LARGE SCALE AGILE ENVIRONMENT Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 11

10 Ways of working Cross functional team of 7 +/- 1 SW developers: Virtual teams Synchronized sprints inside a feature Common Backlog Common ceremonies to keep everybody informed Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 12

11 Ways of Working Backlog Setup strictly based on functional areas US1 US2 Application Transport US3 US4 Internet Network US5 Team A Team B Team C Team D Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 13

12 Ways of Working Backlog Setup strictly based on functional areas Risk of dependencies between User Stories Difficulties to test from an end to end perspective More planning needed Prone to generate delays US1 US2 US4 Application Transport Internet Network US3 US5 Team A Team B Team C Team D Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 14

13 Ways of Working Backlog Setup with end to end focus and functional areas Application US1 US2 Transport US3 US4 US5 Internet Network Team A Team B Team C Team D Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 15

14 Ways of Working Backlog Setup with end to end focus and functional areas Testing possible from end to end perspective as the User Story is concluded Less planning needed Reduce dependencies due to functional areas Coordination might increase Competence build through training on the job and mix of competences US2 US1 Application Transport Internet US5 US4 US3 Network Team A Team B Team C Team D Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 16

15 Ways of Working Setup of virtual teams Team A Team B Team C Team D Virtual Test Team LSV team System Architecture Team Scrum of Scrum Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 17

16 Ways of Working Setup of virtual teams Team A Team B Team C Team D Virtual Test Team LSV team System Architecture Team Scrum of Scrum Overview over the activities with higher priority Testing, Architecture, Software Delivery, Organizational topics Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 18

17 LLV track LLV track LLV track LLV track Test Overview Feature n Feature 3 Feature 2 Feature 1 LSV N N+1 M Note: Simplified view, details on following slides Rebase mandatory Rebase optional LSV Delivery Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 19 Ericsson AB 2009 Ericsson Internal X (X) Date

18 Supporting the Team Testing activities Automated & Manual Business facing Manual Functional Tests Examples & Integration Story Tests Function Test Tests Unit System Module Tests Q2 Q1 Exploratory Testing Function Usability Testing (User Integration Acceptance Tests) Q3 Q4 Performance & Load Testing Security Verification Testing ility Testing Critique Product Automated Technology facing Tools Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 20 1 : taken from Crispin/Gregory: Agile Testing, Addison Wesley, 2009

19 Testing activities Q1 Kind of tests are Unit tests testing small pieces of code System Module Tests test that units work together correctly Q2 Kind of tests are Functional & Integration tests positive and deterministic negative tests, if possible to automate User Story Acceptance Tests Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 21

20 Testing activities Q3 Kind of tests Exploratory Testing feedback into stories Feature Test and Regression Test with Background Traffic (early) Characteristics Q4 Kind of tests Dimensioned Load on LLV/LSV High Load/Overload on LSV (most likely) Parafunctional tests (Performance, stress) Interoperability, Security Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 22

21 TEST Detailed view Detailed view on Testing Timeframe Team D Team C Feature2 Team B Team A Feature1 LSV 0 Main Unit Test Functional Test Integration Test Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 24

22 Continuous integration a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily leading to multiple integrations per day. Each integration is verified by an automated build (incl. test) to detect its errors as quickly as possible Martin Fowler Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 25

23 CI Workflow FCS FIP TF Sprint 1 Sprint 2 Sprint 3 Sprint n GEOFF AutoRT preparation: S1 TCs AutoRT preparation: S2 TCs AutoRT preparation: S3 TCs AutoLT preparation: e2e feature TCs AutoLT preparation: e2e feature TCs CI Server Dump building Dump building Dump building Dump building AutoLT execution (appropriate suite) AutoLT execution (appropriate suite) AutoLT execution (appropriate suite) AutoLT execution (appropriate suite) AutoRT execution S1 TCs AutoRT execution S1+S2 TCs AutoRT execution S1+ +S(n-1) TCs F1 F2 MAx Week MAx Week MAx Week MAx Week F4 Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 26

24 Test TOOLs & Environment CSCF MSC-S Blade Cluster MSC-S Dual Blade 60 HLR MME M- MGW M- MGW MME BSC RNC RNC BSC Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 28

25 Test TOOLs & Environment Unit Test: Ericsson proprietary tool testing the interfaces Function Test: Manual: using a simulated environment simulating all nodes but with the possibility to load the real MSC software Automated: using a proprietary tool with an open source framework based on TTCN-3 Regression Test: Based on the automated test cases from the Function Test Running on a Continuous Integration Server (Jenkins) Feature load (moderate/high/overload): Proprietary tool based on TTCN-3 generating load on the real environment Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 29

26 CONCLUSION/WRAP-UP Challenges Coordination Maintenance of a high reliability Working high quality Legacy Documentation Test tool & environment Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 30

27 Conclusion Benefits Eliminate waste Early indication of Software Quality Early corrective actions possible Reduce test time enabling earlier release of product Enabling of Continuous Integration as first step towards a possible Continuous Delivery Early customer involvement Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 31

28 Conclusion Manifesto for Agile Software Development We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more. Testing in large scale Agile Development Ericsson Internal Ericsson AB Page 32

29