Standard CMMI SM Appraisal Method for Process Improvement (SCAMPI Version ) Lockheed Martin Management & Data Systems King of Prussia, Pennsylvania SM CMM Integration and CMMI are service marks of Carnegie Mellon University
Presentation Agenda Scars of a SCAMPI Appraisal Introduction CMMI Artifact Development The On-Site Period Some Metrics Lessons Learned: Team Afterthoughts
Objectives of Appraisal The goals of the process improvement program and this appraisal were to: Formally recognize the mature practices that are being used within this organization: for freely sharing results, methodology and experiences with other units and organizations. Determine the process maturity of M&DS under the CMMI model. Identify the strengths, weaknesses, and potential improvement opportunities.
The M&DS Team CA - 740 CO 700 PA -,860 WMA -,80 AZ - 480 4
Overview of Appraisal The scope of the SCAMPI was the entire M&DS organization. Representatives from all regions and lines of business were included. The primary focus programs represented 60% of the full scale development work across all of M&DS in 00. An additional 0% were covered through various participation levels and contributing benchmarks. 5
Overview of Appraisal SCAMPI Appraisal Team Members Brian Gallagher, Lead Appraiser, SEI Lt Col Melanie Benhoff, Team Member, USAF C.R. (Bob) Weiser, Team Member, LM EPI Center M. Lynn Penn, Team Member, LM M&DS Dennis A. Ring, Team Member, LM M&DS William Pohle, Team Member, LM M&DS Richard Warren, Site POC/Librarian, LM M&DS Customer SCE Team Members Mike Orr, Paul Frommelt, Steve Wear, Deanne Weinberg 6
Scope of Appraisal CMMI Model: CMMI-SE/SW Version (Staged) Process Areas: All Process Areas Maturity Levels through 5 7
Schedule Week ( June 7 June 00): Monday Team Arrives, Orientation Tuesday Opening Briefing, Team Activities Wednesday Team Activities Thursday Interviews and Team Activities Friday Interviews and Team Activities Week : (7 June 00) Monday Interviews and Team Activities Tuesday Interviews and Team Activities Wednesday Team Activities Thursday Draft Findings Briefings Friday Final Findings Briefing 8
Presentation Agenda Scars of a SCAMPI Appraisal Introduction CMMI Artifact Development The On-Site Period Some Metrics Lessons Learned: Team Afterthoughts 9
Artifact Spreadsheet Description Organizational Policy Project x : Procedures Project x: Output SG Preparation for verification is conducted. SP Select the work products to be verified and the verification methods that will be used for each. PPS (6) PPS (9, 0, 7, 5) PPS. (4g & h, 7) PPS.5 (6, 7) Testing; PP Integration and Test; SDP Process Description, Figure.- Software Development Tasks and Products,.4. Software Formal Qualification Test Activities and Methodology ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Table of Contents, Section 4. General Information SP - Establish and maintain the environment needed to support verification. PPS (7) PPS. (5) PPS.7 (5) PPS.9 (0) PPS (6, 7, 8) PPS () Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Section Software Test Environment; ABCDE System Design Certification Test (SDCT) TRR, Software Under Test, SDCT Entrance Criteria, I5 Software Status SP - Establish and maintain verification procedures and criteria for the selected work products. PPS Section PPS (7) PPS.5 () PPS.6 (5) PPS.7 (4) PPS.9 (8, 9) PPS 0 () PPS () PPS () Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology,.4. Software Formal Qualification Test Exit Criteria ABCDE-00-05 Formal Qualification Test (FQT) Software Test Procedures for the Engagement Manager (EM) CSCI, Table EM FQT Testing (Surface); ABCDE System Design Certification Test (SDCT) TRR, Test Methodology and Sequence SG Peer reviews are performed on selected work products. SP Prepare for peer reviews of selected work products. PPS (6) MPE-090 (5.) SDP 4.7. (Inspections); JI 06 (ABCDE Inspection Support Data) Inspection Notice for Inspection EM0056 with completed SWAD Checklist SP Conduct peer reviews on selected work products and identify issues resulting from the peer review. MPE-090 (5. - 5.5) PPS PPS. (8) PPS.5 (5) PPS.6 (4h) PPS.7 (TBD) JI 06 (ABCDE Inspection Support Data) Inspection Notice/Report for Inspection EM0056, Defects 0
Artifact Spreadsheet Description Organizational Policy Project x : Procedures Project x: Output SG Preparation for verification is conducted. SP SP - Select the work products to be verified and the verification methods that will be used for each. Establish and maintain the environment needed to support verification. PPS (6) PPS (9, 0, 7, 5) PPS. (4g & h, 7) PPS.5 (6, 7) PPS (7) PPS. (5) PPS.7 (5) PPS.9 (0) PPS (6, 7, 8) PPS () Testing; PP Integration and Test; SDP Process Description, Figure.- Software Development Tasks and Products,.4. Software Formal Qualification Test Activities and Methodology CMMI REFERENCE: PROCESS AREA Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Table of Contents, Section 4. General Information ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Section Software Test Environment; ABCDE System Design Certification Test (SDCT) TRR, Software Under Test, SDCT Entrance Criteria, I5 Software Status SP - Establish and maintain verification procedures and criteria for the selected work products. PPS Section PPS (7) PPS.5 () PPS.6 (5) PPS.7 (4) PPS.9 (8, 9) PPS 0 () PPS () PPS () Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology,.4. Software Formal Qualification Test Exit Criteria SPECIFIC GOAL SPECIFIC PRACTICE ABCDE-00-05 Formal Qualification Test (FQT) Software Test Procedures for the Engagement Manager (EM) CSCI, Table EM FQT Testing (Surface); ABCDE System Design Certification Test (SDCT) TRR, Test Methodology and Sequence SG Peer reviews are performed on selected work products. SP Prepare for peer reviews of selected work products. PPS (6) MPE-090 (5.) SDP 4.7. (Inspections); JI 06 (ABCDE Inspection Support Data) Inspection Notice for Inspection EM0056 with completed SWAD Checklist SP Conduct peer reviews on selected work products and identify issues resulting from the peer review. MPE-090 (5. - 5.5) PPS PPS. (8) PPS.5 (5) PPS.6 (4h) PPS.7 (TBD) JI 06 (ABCDE Inspection Support Data) Inspection Notice/Report for Inspection EM0056, Defects
Artifact Spreadsheet Description Organizational Policy Project x : Procedures Project x: Output SG Preparation for verification is conducted. SP Select the work products to be verified and the verification methods that will be used for each. PPS (6) PPS (9, 0, 7, 5) PPS. (4g & h, 7) PPS.5 (6, 7) Testing; PP Integration and Test; SDP Process Description, Figure.- Software Development Tasks and Products,.4. Software Formal Qualification Test Activities and Methodology ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Table of Contents, Section 4. General Information SP - SP - Establish and maintain the environment needed to support verification. Establish and maintain verification procedures and criteria for the selected work products. PPS (7) PPS. (5) PPS.7 (5) PPS.9 (0) PPS (6, 7, 8) PPS () PPS Section PPS (7) PPS.5 () PPS.6 (5) PPS.7 (4) PPS.9 (8, 9) PPS 0 () PPS () PPS () Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology CMMI SPECIFIC GOAL This is the concern which will be rated Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology,.4. Software Formal Qualification Test Exit Criteria ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Section Software Test Environment; ABCDE System Design Certification Test (SDCT) TRR, Software Under Test, SDCT Entrance Criteria, I5 Software Status ABCDE-00-05 Formal Qualification Test (FQT) Software Test Procedures for the Engagement Manager (EM) CSCI, Table EM FQT Testing (Surface); ABCDE System Design Certification Test (SDCT) TRR, Test Methodology and Sequence SG Peer reviews are performed on selected work products. SP Prepare for peer reviews of selected work products. PPS (6) MPE-090 (5.) SDP 4.7. (Inspections); JI 06 (ABCDE Inspection Support Data) Inspection Notice for Inspection EM0056 with completed SWAD Checklist SP Conduct peer reviews on selected work products and identify issues resulting from the peer review. MPE-090 (5. - 5.5) PPS PPS. (8) PPS.5 (5) PPS.6 (4h) PPS.7 (TBD) JI 06 (ABCDE Inspection Support Data) Inspection Notice/Report for Inspection EM0056, Defects
Artifact Spreadsheet Description Organizational Policy Project x : Procedures Project x: Output SG Preparation for verification is conducted. SP Select the work products to be verified and the verification methods that will be used for each. PPS (6) PPS (9, 0, 7, 5) PPS. (4g & h, 7) PPS.5 (6, 7) Testing; PP Integration and Test; SDP Process Description, Figure.- Software Development Tasks and Products,.4. Software Formal Qualification Test Activities and Methodology ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Table of Contents, Section 4. General Information SG SP - SP - Establish and maintain the environment needed to support verification. Establish and maintain verification procedures and criteria for the selected work products. Peer reviews are performed on selected work products. PPS (7) PPS. (5) PPS.7 (5) PPS.9 (0) PPS (6, 7, 8) PPS () PPS Section PPS (7) PPS.5 () PPS.6 (5) PPS.7 (4) PPS.9 (8, 9) PPS 0 () PPS () PPS () Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology CMMI DESCRIPTION OF SPECIFIC PRACTICE Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology,.4. Software Formal Qualification Test Exit Criteria ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Section Software Test Environment; ABCDE System Design Certification Test (SDCT) TRR, Software Under Test, SDCT Entrance Criteria, I5 Software Status ABCDE-00-05 Formal Qualification Test (FQT) Software Test Procedures for the Engagement Manager (EM) CSCI, Table EM FQT Testing (Surface); ABCDE System Design Certification Test (SDCT) TRR, Test Methodology and Sequence This is the specific need (question) that must be answered) SP Prepare for peer reviews of selected work products. PPS (6) MPE-090 (5.) SDP 4.7. (Inspections); JI 06 (ABCDE Inspection Support Data) Inspection Notice for Inspection EM0056 with completed SWAD Checklist SP Conduct peer reviews on selected work products and identify issues resulting from the peer review. MPE-090 (5. - 5.5) PPS PPS. (8) PPS.5 (5) PPS.6 (4h) PPS.7 (TBD) JI 06 (ABCDE Inspection Support Data) Inspection Notice/Report for Inspection EM0056, Defects
Artifact Spreadsheet Description Organizational Policy Project x : Procedures Project x: Output SG Preparation for verification is conducted. SP Select the work products to be verified and the verification methods that will be used for each. PPS (6) PPS (9, 0, 7, 5) PPS. (4g & h, 7) PPS.5 (6, 7) Testing; PP Integration and Test; SDP Process Description, Figure.- Software Development Tasks and Products,.4. Software Formal Qualification Test Activities and Methodology ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Table of Contents, Section 4. General Information SG SP - SP - Establish and maintain the environment needed to support verification. Establish and maintain verification procedures and criteria for the selected work products. Peer reviews are performed on selected work products. PPS (7) PPS. (5) PPS.7 (5) PPS.9 (0) PPS (6, 7, 8) PPS () PPS Section PPS (7) PPS.5 () PPS.6 (5) PPS.7 (4) PPS.9 (8, 9) PPS 0 () PPS () PPS () Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology,.4. Software Formal Qualification Test Exit Criteria ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Section Software Test Environment; ABCDE System Design Certification Test (SDCT) TRR, Software Under Test, SDCT Entrance Criteria, I5 Software Status ORGANIZATIONAL POLICY WHICH REQUIRES (SUPPORTS) THIS SPECIFIC PRACTICE ABCDE-00-05 Formal Qualification Test (FQT) Software Test Procedures for the Engagement Manager (EM) CSCI, Table EM FQT Testing (Surface); ABCDE System Design Certification Test (SDCT) TRR, Test Methodology and Sequence M&DS Program Process Standard corresponding linkage to CMMI SP Prepare for peer reviews of selected work products. PPS (6) MPE-090 (5.) SDP 4.7. (Inspections); JI 06 (ABCDE Inspection Support Data) Inspection Notice for Inspection EM0056 with completed SWAD Checklist SP Conduct peer reviews on selected work products and identify issues resulting from the peer review. MPE-090 (5. - 5.5) PPS PPS. (8) PPS.5 (5) PPS.6 (4h) PPS.7 (TBD) JI 06 (ABCDE Inspection Support Data) Inspection Notice/Report for Inspection EM0056, Defects 4
Artifact Spreadsheet Description Organizational Policy Project x : Procedures Project x: Output SG Preparation for verification is conducted. SP Select the work products to be verified and the verification methods that will be used for each. PPS (6) PPS (9, 0, 7, 5) PPS. (4g & h, 7) PPS.5 (6, 7) Testing; PP Integration and Test; SDP Process Description, Figure.- Software Development Tasks and Products,.4. Software Formal Qualification Test Activities and Methodology ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Table of Contents, Section 4. General Information SP - Establish and maintain the environment needed to support verification. PPS (7) PPS. (5) PPS.7 (5) PPS.9 (0) PPS (6, 7, 8) PPS () Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Section Software Test Environment; ABCDE System Design Certification Test (SDCT) TRR, Software Under Test, SDCT Entrance Criteria, I5 Software Status SG SP - Establish and maintain verification procedures and criteria for the selected work products. Peer reviews are performed on selected work products. PPS Section PPS (7) PPS.5 () PPS.6 (5) PPS.7 (4) PPS.9 (8, 9) PPS 0 () PPS () PPS () Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology,.4. Software Formal Qualification Test Exit Criteria PROGRAM PROCEDURE (or) OTHER IMPLEMENTING ARTIFACT ABCDE-00-05 Formal Qualification Test (FQT) Software Test Procedures for the Engagement Manager (EM) CSCI, Table EM FQT Testing (Surface); ABCDE System Design Certification Test (SDCT) TRR, Test Methodology and Sequence SP Prepare for peer reviews of selected work products. PPS (6) MPE-090 (5.) SDP 4.7. (Inspections); JI 06 (ABCDE Inspection Support Data) Inspection Notice for Inspection EM0056 with completed SWAD Checklist SP Conduct peer reviews on selected work products and identify issues resulting from the peer review. MPE-090 (5. - 5.5) PPS PPS. (8) PPS.5 (5) PPS.6 (4h) PPS.7 (TBD) JI 06 (ABCDE Inspection Support Data) Inspection Notice/Report for Inspection EM0056, Defects 5
Artifact Spreadsheet Description Organizational Policy Project x : Procedures Project x: Output SG Preparation for verification is conducted. SP Select the work products to be verified and the verification methods that will be used for each. PPS (6) PPS (9, 0, 7, 5) PPS. (4g & h, 7) PPS.5 (6, 7) Testing; PP Integration and Test; SDP Process Description, Figure.- Software Development Tasks and Products,.4. Software Formal Qualification Test Activities and Methodology ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Table of Contents, Section 4. General Information SG SP - SP - Establish and maintain the environment needed to support verification. Establish and maintain verification procedures and criteria for the selected work products. Peer reviews are performed on selected work products. PPS (7) PPS. (5) PPS.7 (5) PPS.9 (0) PPS (6, 7, 8) PPS () PPS Section PPS (7) PPS.5 () PPS.6 (5) PPS.7 (4) PPS.9 (8, 9) PPS 0 () PPS () PPS () Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology DOES THIS ANSWER THE QUESTION? Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology,.4. Software Formal Qualification Test Exit Criteria PROGRAM PROCEDURE (or) OTHER IMPLEMENTING ARTIFACT (Direct Artifact) ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Section Software Test Environment; ABCDE System Design Certification Test (SDCT) TRR, Software Under Test, SDCT Entrance Criteria, I5 Software Status ABCDE-00-05 Formal Qualification Test (FQT) Software Test Procedures for the Engagement Manager (EM) CSCI, Table EM FQT Testing (Surface); ABCDE System Design Certification Test (SDCT) TRR, Test Methodology and Sequence SP Prepare for peer reviews of selected work products. PPS (6) MPE-090 (5.) SDP 4.7. (Inspections); JI 06 (ABCDE Inspection Support Data) Inspection Notice for Inspection EM0056 with completed SWAD Checklist SP Conduct peer reviews on selected work products and identify issues resulting from the peer review. MPE-090 (5. - 5.5) PPS PPS. (8) PPS.5 (5) PPS.6 (4h) PPS.7 (TBD) JI 06 (ABCDE Inspection Support Data) Inspection Notice/Report for Inspection EM0056, Defects 6
Artifact Spreadsheet Description Organizational Policy Project x : Procedures Project x: Output SG Preparation for verification is conducted. SP Select the work products to be verified and the verification methods that will be used for each. PPS (6) PPS (9, 0, 7, 5) PPS. (4g & h, 7) PPS.5 (6, 7) Testing; PP Integration and Test; SDP Process Description, Figure.- Software Development Tasks and Products,.4. Software Formal Qualification Test Activities and Methodology ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Table of Contents, Section 4. General Information SP - Establish and maintain the environment needed to support verification. PPS (7) PPS. (5) PPS.7 (5) PPS.9 (0) PPS (6, 7, 8) PPS () Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Section Software Test Environment; ABCDE System Design Certification Test (SDCT) TRR, Software Under Test, SDCT Entrance Criteria, I5 Software Status SG SP - SP Establish and maintain verification procedures and criteria for the selected work products. Peer reviews are performed on selected work products. Prepare for peer reviews of selected work products. PPS Section PPS (7) PPS.5 () PPS.6 (5) PPS.7 (4) PPS.9 (8, 9) PPS 0 () PPS () PPS () PPS (6) MPE-090 (5.) Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology,.4. Software Formal Qualification Test Exit Criteria SDP 4.7. (Inspections); JI 06 (ABCDE Inspection Support Data) ABCDE-00-05 Formal Qualification Test (FQT) Software Test Procedures for the Engagement Manager (EM) CSCI, Table EM FQT Testing (Surface); ABCDE System Design Certification Test (SDCT) TRR, Test Methodology and Sequence PROGRAM BY-PRODUCTS (OUTPUTS) THAT SHOW THE PRACTICE IS ROUTINELY ADDRESSED (Indirect Artifacts) Inspection Notice for Inspection EM0056 with completed SWAD Checklist SP Conduct peer reviews on selected work products and identify issues resulting from the peer review. MPE-090 (5. - 5.5) PPS PPS. (8) PPS.5 (5) PPS.6 (4h) PPS.7 (TBD) JI 06 (ABCDE Inspection Support Data) Inspection Notice/Report for Inspection EM0056, Defects 7
Artifact Spreadsheet Description Organizational Policy Project x : Procedures Project x: Output SG Preparation for verification is conducted. SP SP - Select the work products to be verified and the verification methods that will be used for each. Establish and maintain verification procedures and criteria for the selected work products. PPS (6) PPS (9, 0, 7, 5) PPS. (4g & h, 7) PPS.5 (6, 7) ONE SUCCINCT ANSWER IS SP - Establish and maintain the PPS (7) Testing; PP Integration and BETTER environment needed to PPS. (5) support verification. THAN PPS.7 A (5) WHOLETest; SDP.4. Software PPS.9 (0) Formal Qualification Test Activities LITTANY / LAUNDRY LIST PPS (6, 7, 8) and Methodology PPS () (Hit the target!) PPS Section PPS (7) PPS.5 () PPS.6 (5) PPS.7 (4) PPS.9 (8, 9) PPS 0 () PPS () PPS () Testing; PP Integration and Test; SDP Process Description, Figure.- Software Development Tasks and Products,.4. Software Formal Qualification Test Activities and Methodology Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology,.4. Software Formal Qualification Test Exit Criteria ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Table of Contents, Section 4. General Information ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Section Software Test Environment; ABCDE System Design Certification Test (SDCT) TRR, Software Under Test, SDCT Entrance Criteria, I5 Software Status ABCDE-00-05 Formal Qualification Test (FQT) Software Test Procedures for the Engagement Manager (EM) CSCI, Table EM FQT Testing (Surface); ABCDE System Design Certification Test (SDCT) TRR, Test Methodology and Sequence SG Peer reviews are performed on selected work products. SP Prepare for peer reviews of selected work products. PPS (6) MPE-090 (5.) SDP 4.7. (Inspections); JI 06 (ABCDE Inspection Support Data) Inspection Notice for Inspection EM0056 with completed SWAD Checklist SP Conduct peer reviews on selected work products and identify issues resulting from the peer review. MPE-090 (5. - 5.5) PPS PPS. (8) PPS.5 (5) PPS.6 (4h) PPS.7 (TBD) JI 06 (ABCDE Inspection Support Data) Inspection Notice/Report for Inspection EM0056, Defects 8
Artifact Spreadsheet Description Organizational Policy Project x : Procedures Project x: Output SG Preparation for verification is conducted. SP Select the work products to be verified and the verification methods that will be used for each. PPS (6) PPS (9, 0, 7, 5) PPS. (4g & h, 7) PPS.5 (6, 7) Testing; PP Integration and Test; SDP Process Description, Figure.- Software Development Tasks and Products,.4. Software Formal Qualification Test Activities and Methodology ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Table of Contents, Section 4. General Information SP - Establish and maintain the environment needed to support verification. PPS (7) PPS. (5) PPS.7 (5) PPS.9 (0) PPS (6, 7, 8) PPS () Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology ABCDE-00-055 Formal Qualification Test (FQT) Software Test Plan for the Common Services (CS) CSCI, Section Software Test Environment; ABCDE System Design Certification Test (SDCT) TRR, Software Under Test, SDCT Entrance Criteria, I5 Software Status SP - Establish and maintain verification procedures and criteria for the selected work products. PPS Section PPS (7) PPS.5 () PPS.6 (5) PPS.7 (4) PPS.9 (8, 9) PPS 0 () PPS () PPS () Testing; PP Integration and Test; SDP.4. Software Formal Qualification Test Activities and Methodology,.4. Software Formal Qualification Test Exit Criteria ABCDE-00-05 Formal Qualification Test (FQT) Software Test Procedures for the Engagement Manager (EM) CSCI, Table EM FQT Testing (Surface); ABCDE System Design Certification Test (SDCT) TRR, Test Methodology and Sequence SG Peer reviews are performed on selected work products. SP Prepare for peer reviews of selected work products. PPS (6) MPE-090 (5.) SDP 4.7. (Inspections); JI 06 (ABCDE Inspection Support Data) Inspection Notice for Inspection EM0056 with completed SWAD Checklist SP Conduct peer reviews on selected work products and identify issues resulting from the peer review. MPE-090 (5. - 5.5) PPS PPS. (8) PPS.5 (5) PPS.6 (4h) PPS.7 (TBD) JI 06 (ABCDE Inspection Support Data) Inspection Notice/Report for Inspection EM0056, Defects 9
Process Area Binders A Tabbed Binder for Each Process Area Requirements Management Project Planning Organizationa Training Requirements Management Process Area (Binder Names) Identified in Column A of CMMI Matrix Specific Practice Tabs SP- Specific Practice Number identified in Column C of CMMI Matrix 0
For Each Specific Practice in a Binder Specific Practice Tabbed Separator SP- + Organization Artifacts + Project x Separator Sheet How is the + project + going to do it Proof Project is doing it
For Each Specific Practice Specific Practice Tabbed Separator SP- + Column E of CMMI Matrix (What must be done) Organization artifacts: policy, procedure, guidebook, etc. + Project x Separator Sheet Use different color for each program + Column F of CMMI Matrix (How is the project going to do it) Project x s plans, procedures, directives, etc. + Column G of CMMI Matrix (Artifacts that the project is doing it) Project xs status reports, minutes, metrics, agendas expenditures, etc.
BREAK TIME
Presentation Agenda Scars of a SCAMPI Appraisal Introduction CMMI Artifact Development The On-Site Period Some Metrics Lessons Learned: Team Afterthoughts 4
Appraisal On-Site Time 5
Some Metrics Number of people interviewed: 67 Data originally supplied: over 4000 items Additional documentation requests: 78 Number of on-site clock hours: 8 6
7 Week Day CAR OID.4 QPM.5 OPP.6.5 DAR RSKM 4. 4. 4..5 IPM OT.5 OPD.4 OPF VAL.4. PI.4 TS.5.4. RD CM PPQA.4 M&A.4 SAM.7.6.5 PMC..7.6.5.4 PP.5 REQM 4 0 9 8 7 6 5 4 4 5 4 PA SP VE DI AB CO GENERIC PRACTICES SPECIFIC PRACTICES PAs
8 Week Day 4 CAR OID.4 QPM.5 OPP.6.5 DAR RSKM 4. 4. 4..5 IPM OT.5 OPD.4 OPF VAL.4. PI.4 TS.5.4. RD CM PPQA.4 M&A.4 SAM.7.6.5 PMC..7.6.5.4 PP.5 REQM 4 0 9 8 7 6 5 4 4 5 4 PA SP VE DI AB CO
9 Week Day 6 CAR OID.4 QPM.5 OPP.6.5 DAR RSKM 4. 4. 4..5 IPM OT.5 OPD.4 OPF VAL.4. PI.4 TS.5.4. RD CM PPQA.4 M&A.4 SAM.7.6.5 PMC..7.6.5.4 PP.5 REQM 4 0 9 8 7 6 5 4 4 5 4 PA SP VE DI AB CO
0 Week Day 7 CAR OID.4 QPM.5 OPP.6.5 DAR RSKM 4. 4. 4..5 IPM OT.5 OPD.4 OPF VAL.4. PI.4 TS.5.4. RD CM PPQA.4 M&A.4 SAM.7.6.5 PMC..7.6.5.4 PP.5 REQM 4 0 9 8 7 6 5 4 4 5 4 PA SP VE DI AB CO
Week Day 8 CAR OID.4 QPM.5 OPP.6.5 DAR RSKM 4. 4. 4..5 IPM OT.5 OPD.4 OPF VAL.4. PI.4 TS.5.4. RD CM PPQA.4 M&A.4 SAM.7.6.5 PMC..7.6.5.4 PP.5 REQM 4 0 9 8 7 6 5 4 4 5 4 PA SP VE DI AB CO
Week Day 9 CAR OID.4 QPM.5 OPP.6.5 DAR RSKM 4. 4. 4..5 IPM OT.5 OPD.4 OPF VAL.4. PI.4 TS.5.4. RD CM PPQA.4 M&A.4 SAM.7.6.5 PMC..7.6.5.4 PP.5 REQM 4 0 9 8 7 6 5 4 4 5 4 PA SP VE DI AB CO
Presentation Agenda Scars of a SCAMPI Appraisal Introduction CMMI Artifact Development The On-Site Period Some Metrics Lessons Learned: Team Afterthoughts
Process Improvement Journey Began in early 990s (GE EPI) to establish foundation within Engineering 995 - Integrated Process activities at enterprise level (PCBs/ EPSC) 995 - Integrated Program Process Standard (PPS) at enterprise level 000 - SE CMM Level 5 ( PAs) SI LOB 000 - SW CMM Maturity Level 5 4
Process Excellence - 00 Region SE CMM SW CMM ISO Delaware Valley Washington MA Southwest Western Rocky Mt. 5
Transition to CMMI 8 months of additional work to get CMMI Compliance Functional/ LOB Support Mature Programs Dedicated/ experienced staff 6
Team Composition Experienced & trained All six had 0+ years experience All completed Intermediate CMMI course SCAMPI Lead Appraisers + Lead Evaluator Mini-teams help parallel development Cross mini-team work More team-wide interviews 7
Team Training How to develop common understanding when mini-teams used Common criteria for evaluation of PA s Not consistent on OE filtering What constitutes acceptable entries Team building & trust Extra day of ATT? 8
Data Access to data on-line Problem - try project CD-ROM Network configuration w/ documentation IT support for team Plan for backup method (if data lost) 9
Process Validation = BAD for draft findings presentation When non-attribution is used for projects Just write down: Don t give us more rocks Use POC as filter for supplied data No Synergy between SCAMPI & SCE Different intents, team focus 40
Generics GP leveling (common ground) No clear (early) definition of how to be handled GP recommended distribution: GP.5? OT, 0? OPF/OPD GP.6? CM, GP.8? MA GP.9? PPQA GP,,.4,.7,.8,? stay w/ PA 4
Tool Issues Tool helped capture /manage data Networking necessary for team Make sure tool is designed for SCAMPI License or seats for all team members Training & tailoring before on-site use Focused tool training 4
Thoughts Site coordinator / project POC s Lunch / facilities Ability to go unstructured into projects Hours Downtime (interval week off): refresh Didn t take advantage of downtime (recover losses) 4
SCAMPI WORKS! QUESTIONS?