Quality Excellence fr Suppliers f Telecmmunicatins Frum (QuEST Frum) TL 9000 Quality Management System Measurements Handbk Cpyright QuEST Frum
Sftware Fix Quality (SFQ) Examples 8.1 8.1.1 SFQ Example The fllwing example illustrates calculatin f the Sftware Fix Quality measurement. Sftware fixes are cunted regardless f the methd used t package/deliver the fix. Fr this example, release numbers are used t identify the packages f released fixes. An rganizatin has three active releases, 1.0, 1.1, and 1.2. Sftware fixes have been made available as shwn in the left side f Table 8.1.1-1. The right side f the table shws the number f sftware fixes reprted fr the Sftware Fix Quality measurement in the reprting mnths shwn. Sftware Fixes Released Reprting Mnth Release Release Date Fix Cnt Jan Feb Mar Apr May 1.0 Dec-15 n/a 1.0 1.0.1 Jan-16 6 1.0.2 Apr-16 1 1 1 1 1.0.3 Jun-16 5 5 5 5 5 5 Jun 1.1 Jun-16 15 15 15 15 15 15 1.1 1.1.1 Dec-16 4 4 4 4 4 4 4 1.1.2 Jun-17 2 2 1.2 Jan-17 10 10 10 10 10 10 10 1.2 1.2.1 Mar-17 5 5 5 5 5 1.2.2 Jun-17 1 1 Sftware Fixes Cunted in the Reprting Mnth 35 35 40 39 39 22 Table 8.1.1-1 Example SFQ Release/Fix Data In the January thrugh June timeframe, the fllwing prblems with the delivered fixes are identified: 1. In January, during the installatin f Release 1.1.1 it becmes evident that the release cannt be installed. All 4 fixes delivered in the release are cunted as defective fixes fr January. 2. In February, a custmer reprts a fix in Release 1.1.1 did nt fix the intended prblem. The fix is nt cunted as a defective fix since it was already reprted as a defective fix in January. Cpyright QuEST Frum Versin 2.0 8.1-1
Sftware Fix Quality (SFQ) Examples 3. Als in February a Minr prblem is reprted because ne f the fixes in Release 1.1 did nt fix the intended prblem. The fix is cunted as a defective fix in February since the fix did nt crrect the intended prblem. 4. Als in February, a Majr prblem determined t be a side effect f anther (different) fix prvided in Release 1.1 is reprted. The fix is nt cunted as a defective fix in February because the date the fix is fund defective is utside the 6-mnth windw f the fix release date fr Release 1.1. 5. In March a Majr prblem is reprted because ne f the fixes in Release 1.2 did nt fix the intended prblem. The fix is cunted as a defective fix in March. 6. In April, additinal internal testing by the rganizatin fr the fixes in Release 1.2.1 determines that ne f the fixes desn t cmpletely crrect the intended prblem. The fix is cunted as a defective fix in April. 7. In May, a Critical prblem determined t be caused by a fix in Release 1.1.2 is reprted. The fix is cunted as a defective fix in May. 8. Als in May, a Minr prblem determined t be a side effect f the secnd fix in Release 1.1.2 is reprted. The fix is nt cunted as a defective fix in May since the prblem fr the side effect is pened as a Minr prblem. 9. In June, a new custmer cming up n Release 1.0.3 reprts a Critical prblem and cannt install the release. Since the installatin date is nt reprted within 12 mnths frm the release date f the fixes, the 3 fixes prvided in the release are nt included in the June defective fix cunt. The number f defective sftware fixes reprted each mnth fr the Sftware Fix Quality measurement wuld be determined as shwn in Table 8.1.1-2. Table 8.1.1-2 Example SFQ Mnthly Defective Fixes Jan Feb Mar Apr May Jun Number f Defective Sftware Fixes Release 1.0 0 0 0 0 0 0 Release 1.1 4 1 0 0 1 0 Release 1.2 0 0 1 1 0 0 Ttal 4 1 0 1 1 0 Cpyright QuEST Frum Versin 2.0 8.1-2
Sftware Fix Quality (SFQ) Examples The resulting mnthly surce data and measurement calculatins (runded t the nearest integer) are shwn in Table 8.1.1-3. Table 8.1.1-3 Example SFQ Mnthly Surce Data and Measurement Calculatin Jan Feb Mar Apr May Jun Afactr (Fa) 12 12 12 12 12 12 Sftware Fixes (Fc) 35 35 40 39 39 22 Defective Fixes (DFc) 4 1 0 1 1 1 %Defective Per Yr 137% 34% 0% 31% 31% 55% Nte that, as shwn in the January SFQ measurement calculatin, the mnthly calculatin may exceed 100% due t the annualizatin factr being applied t the measurement. Fr the mnth f June, the TL 9000 SFQ data reprted is shwn in Table 8.1.1-4. Table 8.1.1-4 SFQ Data Table Reprt Identifier Value MeasurementID SFQ Fa 12 DFc 1 Fc 22 8.1.2 Frequently Asked Questins 8.1.2.1 Hw d I cunt sftware fixes? Organizatins have ne r mre means by which sftware fixes are delivered r made available t the custmer fr implementatin. These different types f delivery mechanisms include (but are nt limited t) patches, files, maintenance releases, updates, dt releases, fix releases, etc. Althugh the actual implementatin methd differs, each f these means wuld include sme type f ntificatin f the availability f the sftware change and infrmatin n what fixes are included, such as a release letter r prduct bulletin. Organizatins can use the custmer ntificatin f the sftware change t btain the number f fixes t be included in the SFQ measurement. Cpyright QuEST Frum Versin 2.0 8.1-3
Sftware Fix Quality (SFQ) Examples The infrmatin must be descriptive enugh t ensure nly fixes t prblems requiring changes in the prduct sftware are cunted. Fixes assciated with paper dcumentatin r enhancement requests wuld nt be cunted. The custmer ntificatin wuld need t include fixes t prblems in the delivered sftware fund by the rganizatin as well as thse fund by custmers; therwise, these fixes wuld need t be identified and cunted separately and the internal and custmer fix cunts added tgether t btain the reprted SFQ cunts. An alternate means f identifying fixes t be included in the SFQ measurement is by utilizing the rganizatin s prblem tracking tl t identify prblems fixed. As with the previus methd, the tl must be able t identify what release(s) the prblem is being fixed in, distinguish between defects and enhancements as well as distinguishing between fixes requiring prduct sftware changes and thse that d nt (fr example, paper dcumentatin changes, third-party sftware changes). If separate tls are used t track custmer prblems and internally fund prblems, cunts frm the tw systems wuld need t be added tgether t btain the reprted SFQ cunts. 8.1.2.2 Hw can I use the SFQ Measurement? The Sftware Fix Quality measurement is the percentage f sftware fixes determined t be defective. The higher the percentage, the greater is the risk that the installatin f a sftware fix t crrect a prblem will either fail t crrect it r else intrduce additinal prblems int the netwrk. When using the SFQ measurement, especially t set gals and drive cntinuus imprvement, it is imprtant t cnsider the TL 9000 Perfrmance Data Reprts smthing rules and use the smthed averages. Mnthly snapshts may demnstrate t much variability t prvide an accurate representatin f the Sftware Fix Quality trend due t the delay in the discvery f defective fixes. This variability will be mre bvius in prducts that nly release fixes a few times a year. The SFQ measurement shuld trend dwnward, eventually reaching 0. Where this is nt happening, rganizatins shuld cnsider perfrming defect analysis n the defective fixes t identify pssible prcess imprvements. The analysis shuld be fcused n why the fix prvided did nt wrk, nt n what caused the riginal prblem. A detailed defect analysis shuld identify the rt cause f the prblem, and measures that culd have detected the defective fix befre it was made available and prevented it frm being intrduced int the sftware. If there is a spike in the percentage f defective fixes, the rganizatin culd perfrm a high-level defect analysis t see if the prblems are Cpyright QuEST Frum Versin 2.0 8.1-4
Sftware Fix Quality (SFQ) Examples related t a particular release, custmer, prduct platfrm, etc. This culd help identify pssible areas fr imprvement at a lwer cst than a mre detailed analysis f individual defects. Cpyright QuEST Frum Versin 2.0 8.1-5