Software Test Process Assessment Methodology - II

Transcription

1 Journal of Engineering Computers & pplied Sciences(JECS) ISSN No: Software Test Process ssessment Methodology - II Dr.S. K. Sudarsanam, Professor, VIT Business School Chennai, VIT University, Chennai, India bstract This paper provides a methodology for rating the process areas of TPI. TPI (Test Process Improvement) model TPI is Sogeti and Capgemini's highly-effective approach to test process assessment and improvement, providing an objective insight into the maturity of an organisation s entire test processes. The TPI model has 4 key area groups and 20 key areas. This paper provides a quantitative method of measuring the process compliance for each of TPI key areas. Keywords: TPI, Test Process improvement (TPI), Key reas, Levels, Checkpoints and Test process Rating. Introduction TPI Test Process Improvement is a tried and tested structured assessment of an organization s testing maturity with a view to improving its overall testing and Q effectiveness and efficiency. are given. The implementation of improvement activities carried out using the suggestions would enable the organizations to reach the desired level of maturity for each of the key areas of TPI. TPI (Pol, 1995) offers insight to the maturity of the test processes within the organization. TPI also offers guidelines for Software process improvements (similar to CMMI, TMMI, ISO and IEEE Testing standards). It reveals the strengths and weaknesses of existing test processes within an organization and provides structured methodology and key areas for test process improvement. There are broadly three key area groups in TPI. They are (LTIO) Life Cycle related areas, Technology related areas, Infrastructure and tools related areas and Organization related areas. Levels : There are four maturity levels, B, C and D. Each of the key area is assessed against the maturity level with being the lowest level of maturity and D being the hightest level of maturity. Checkpoints are used to determine the level of maturity for each of the key area. There are four maturity scales : dhoc (0), Controlled (1-5), Efficient (6-10) and Optimizing (11-13). Improvement Suggestions : In the TPI model, for every maturity level of each key area improvement suggestions There are 21 Key reas in the TPI model covering all the aspects of the test process. Each of the key area has different maturity levels starting from which is covered in all key areas and ending with D which is covered in only few key areas. Under each maturity level, there are checkpoints for each of the key area. Higher maturity levels automatically imply the compliance of lower maturity levels (there may be some exceptions). No. of checkpoints Checkpoint compliance Key reas B C D B C D Test Strategy Test Life-Cycle Model Moment of involvement Estimating and Planning Test Design Techniques Static Test Techniques Open ccess Journals Blue Ocean Research Journals 20

2 Journal of Engineering Computers & pplied Sciences(JECS) ISSN No: Metrics Test utomation Test Environment Office Environment 3 3 Commitment and motivation Test functions and training Scope of methodology Communication Reporting Defect Management Testware Management Test Process Management Evaluation Low-level testing Integration Testing The table shows the Key areas and the no. of checkpoints in the maturity levels, B, C and D. The checkpoint compliance shows the no. of checkpoints which are satisfied by a Software Testing organization in each of the key test process areas. TPI Rating nd Compliance: Currently the TPI compliance is reported as maturity level compliance in the scale of 0-13 for each of the key area. (as in the table below): Open ccess Journals Blue Ocean Research Journals 21

3 Journal of Engineering Computers & pplied Sciences(JECS) ISSN No: new rating methodology is proposed in addition to the current methodology which clearly gives the areas of non-compliance and areas of improvement. a) For each of the Key area, the compliance levels of checklists for each of the maturity levels are listed out. b) For the organization which wants to get TPI compliance assessed, for each of the key area the compliance of the checklists for each of the maturity levels (,B,C and D) are assessed c) For example, if an organization satisfies 3 out 3 checklists in Level, then its compliance is 3 in Level under that particular process are d) The percentage of compliance under each level (,B, C and D) is calculated for each of the key areas ratio of checkpoint compliance to number of checkpoints e) Overall percentage of compliance is also calculated for each of the key areas. Comparison Of The Current TPI Compliance Reporting With New Proposed Rating Method: TPI process audit was carried for a testing organization for 21 key areas. Each of the key area is assessed for compliance of checklists of, B, C and D and the level of compliance for the organization are reported in the table below: Key reas No. of checkpoints Checkpoint compliance B C D B C D Test Strategy Test Life-Cycle Model Moment of involvement Estimating and Planning Test Design Techniques Static Test Techniques Metrics Test utomation Test Environment Office Environment 3 3 Commitment and motivation Test functions and training Scope of methodology Communication Reporting Defect Management Testware Management Test Process Management Evaluation Low-level testing Integration Testing Open ccess Journals Blue Ocean Research Journals 22

4 Journal of Engineering Computers & pplied Sciences(JECS) ISSN No: The table below shows the compliance reporting suggested by TPI model Scale Key rea Controlled Efficient Optimizing Test Strategy B Test Life-Cycle B Moment of involvement Estimating and Planning B Test Design Techniques B Static Test Techniques Metrics B C Test utomation B C Test Environment B C Office Commitment and motivation B C Test functions and training B C Scope of methodology B C Communication B Reporting B C Defect B C Testware Management B C Test Process Management B C Evaluation Low-level testing Integration Testing B C Overall Level of Compliance Compliance Key reas B C D Test Strategy 0.00% 40.00% 0.00% 0.00% 8.70% Test Life-Cycle Model % % % Moment of involvement % 0.00% 0.00% 0.00% 25.00% Estimating and Planning % % % Test Design Techniques 50.00% % 0.00% 66.67% Static Test Techniques 50.00% 0.00% 33.33% Metrics % 50.00% 50.00% 0.00% 55.56% Test utomation % 83.33% % 92.86% Test Environment 75.00% 66.67% 0.00% 66.67% Office Environment % % Commitment and motivation % % % % Test functions and training % % % % Scope of % % 66.67% 0.00% 66.67% th d l Communication % 75.00% 0.00% 69.23% Reporting % % 0.00% 0.00% 50.00% Defect Management % % % % Testware Management % 71.43% % 83.33% Test Process Management % % 66.67% 88.89% Evaluation % 0.00% 0.00% 40.00% Low-level % 0.00% 0.00% 20.00% t ti Integration Testing % 57.14% 66.67% 73.33% Open ccess Journals Blue Ocean Research Journals 23

5 Journal of Engineering Computers & pplied Sciences(JECS) ISSN No: The table above provides the new TPI reporting model suggested The new model clearly provides the percentage of compliance for each key area for each maturity level and also the overall compliance level for the key area. This would help organizations identify easily the areas of improvement and can use the suggestions for improvement to implement process assets and attain a higher level of maturity for those areas where the compliance is low (less than 70%). Conclusion This paper provides a new process compliance rating mechanism which can be used as a reference and guideline for carrying out TPI process maturity assessments for Software Test processes in IT organizations cutting across all domains of business. lso, this can be used by Quality Management teams and Software Engineering Process groups (SEPG) to carry out internal assessment of their organizations to find out the current level of maturity for each of the key test process areas. Compared with the rating mechanism outlined by TPI, the compliance rating appraisal suggested in the previous section provides the rating compliance percentage for each of the key process areas and would help clearly identify the areas of TPI compliance and areas for improvement. This would help all the stakeholders involved to be better equipped with their understanding of the TPI assessment guidelines, maturity levels, checklists of each level for each area, rating scale and scoring methods. References [1]. TMMi Foundation, TMMI Framework and Levels, 2009, [2]. TPI Home Pages, Sogeti Nederland B.V., [ ] [3]. TMP Home Pages, TMap - Sogeti Nederland B.V., 2004, [ ] [4]. Koomen T and Pol M., Test Process Improvement: practical step-by-step guide to structured testing, CM Press, London, England, 1999 [5]. Koomen T et al, TMap Next a result driven testing, UTN Publishers, Netherlands, Nov 2009, ISBN-13: Open ccess Journals Blue Ocean Research Journals 24