Test Suite Knowledge sharing - Solution Manager 7.2. By: Endre Rasmussen At: Coop Norge Date:

Transcription

1 Test Suite Knowledge sharing - Solution Manager 7.2 By: Endre Rasmussen At: Coop Norge Date:

2 Expectations Experience Sharing and Demonstration of Test Suite, Solution Manager 7.2 by Coop Coop will share their experience of using Test Suite in Solution Manager 7.2. Learn and understand how Solution Documentation is the foundation to successful use of Test Suite. Gain insight and build experience in test planning, reporting, execution and defect handling with integration to ChaRM.

3 Agenda Facts about Coop SolMan usage in Coop High level concept Solution Documentation Experience and learning points Test Suite Experience and learning points Demo

4 Facts about Coop 29,4 % marked share in grocery 1350 stores 45 billion in turnover 1,5 million members employees SAP landscape ECC Retail, ECC Industry, ECC slag, SCM, CRM, BW (HANA/ABAP), CAR (HANA/ABAP), SLT, Enterprise Portal, NWDI Three tier system landscape + sandbox >4000 users Solution Manager 7.2 SP6 Data from March 2017

5 High-level concept in Coop One Solution with integration to other modules Test Suite Link to both manual and automated (CBTA) test scripts Business Process Change Analyzer (BPCA) Scope and Effort Analyzer (SEA) - pilot Business Process Operation Business Process Monitoring (BPMon) Job Monitoring Data Volume Management Custom Code Management Code Inspector (integration to ChaRM) CCLM Strategic roadmap in progress Change Control Management Integration to ChaRM documents Process structure and document change control Business Process Improvement - PILOT Measure ROI based on usages statistic in the production system Identify critical business processes based on most used objects Identify improvement areas based on not used objects (lack of training etc.) Create Solution reports for audit and new projects

6 Solution Documentation

7 Solution Documentation Our «Single Source of Truth» Solution: 1 Solution, 75 logical component groups, 2 process groups E2E Processes: 5 Scenarios, 9 Processes, 159 Process steps, 7 diagrams Modular Processes: 144 Scenarios, 458 Processes, 2395 process steps, 785 executables, 214 diagrams, 4385 test cases (only 1500 assigned to processes) Statement: Process modelling is the foundation for control and measure of our solution, and a common visual language between IT and Business

8 Solution Documentation Coop Experience and learning points General recommendations - Recommend to upgrade to at least SP05 with possibility to define business process variants - Identify the processes responsible to ensure ownership and continues maintenance - Recommend numbering the defined processes as part of naming standard, for easier reporting and search - Define needed customer attributes from start to simplify the search and reporting (via SOLMAN_SETUP) - Use collaboration diagram in order to allow modelling of interfaces to third parties - Define how the process can be measured and controlled (input to BPMon) General experience - Very good search and filtering functionality - Experience that IE browser is slow to work with, Chrome is better but does not have the edit online function for document handling - User training is needed in order to navigate in a good and efficient way

9 Solution Documentation Coop Experience and learning points Relevant for Test Suite - Decide whether to create and assign test cases on step or process level - Create a manual test case template for consistency (expected result, actual result etc) - Recommend to ensure test cases reflects the modelled processes, including interfaces - Recommend to assign custom attributes to test cases, for easy test planning (example: test phase relevance, criticality, relevant for test automation) - Note that test cases not possible to be created for interfaces in interfaces library. This makes it hard to report and sign off testing of critical interfaces Modell process Create test case based on template Test Case reflects the process modell

10 Test Suite

11 Test Suite Experience taken from SAP upgrade project in Coop Scope: Upgrade all SAP systems (3xECC, BW, CRM, SCM, Portal, TREX, Java Hub, NDWI) Timeline: 6 months (April October 2017) Facts: 38 systems upgraded, 130 involved people, 1766 manual test cases executed, 159 errors detected and corrected TEST PHASE PACKAGES TEST CASES NO IN PROGRESS ERRORS OK DEFECTS COMPLETION (%) RESULT SANDBOX (ST) DEVELOPMENT (DT) USER ACCEPTANCE TESTING (UAT) PRODUCTION VERIFICATION (PRODT) TOTAL OF ALL TEST PHASES One of the key success factors was the preparation work, identify and secure test coverage of critical business processes and related interfaces

12 Test Suite High level Test flow Test Manager Planning Execute Defect handling Tester Developer Run BPCA to scope test Creation of Test Plan, Test Package and Test Case and Tester assignment Creation of Test Reports Test Execution via Tester Worklist Test Note creation Test Defect creation Test defect handling in ITSM Using Defect Correction in ChaRM (close link to test defect)

13 Test Manager Test Suite - Test Planning Experience and Learning points Experience Nice integration with Solution Documentation, structure is maintained and well known Note that changes in test case (SolDoc) will not be visible in existing test packages, new test package needs to be created and assigned to tester Missing mass update function of tester in test packages for Test Manager, comes with SP05 Good reporting possibilities (drill down in all levels) BPCA analyse is complex and requires detailed investigation in order to understand the result No experience in Test Case sequence, possible but not used Learning points Decide on which level in structure the test plan and test packages should be created for easy reporting and follow up Example of report:

14 Tester Test Suite - Test Execution Experience and Learning points Experience Tester recognize the structure in tester worklist from the process structure, which is good Too many pop ups/different windows when working with manual execution. Solved with Focus Build Test Cases opens in a separate window, not embedded as in 7.1. Solved with Focus Build Many refresh buttons under Tester Worklist. Needs to be refreshed manually Start Test Execution button makes sense if you have test case on process step level (Coops decision to have test case on process level). Solved with Focus Build No experience in TBOM recording, decided not to use refer to statement above Learning points Decide the use of Test Note and Test Result, and method (check in check out or upload new version) Evaluate the available Focused Build solutions offered by SAP related to Test Suite (Simplified test execution etc) NEWS: Focused Build for New manual test solution with grid approach fully embedded in Test Suite etc link (slide 27-45)

15 Developer Test Suite - Defect Handling Experience and Learning points Experience Nice follow up document integration between Test Defect (Incident) and Defect Correction (ChaRM), text and field entries can be copied over (SP05) No interlink when processing Test Defect and Defect Correction. Developer needed to work on both Test Defect and Defect Correction document when updating status etc No direct link to test notes from Test Defect, only link to test case in SOLDOC (where actual result is blank) Our method was to attach test note to Test Defect Note that Testers must go into ITSM in order to reply and confirm reported Test Defects (direct link from tester worklist or mail notification) Learning points Analyse the standard Defect Handling workflow and evaluate if customizing is needed Upgrade to SP06 in order to get the status interlinked between Test Defect and Defect Correction which can cause manual work and continual progress* *Shipped with SP06: Status profiles for Test Defect and Defect Correction interlinked link (slide 24)

16 Demo SolMan 7.2 SP6