This copy of the Validation Protocol / Report for. # ZTC-10, for version 2 of ZTCP's. "MDD-PLUS C=0 SAMPLING PLANS.xls"

Transcription

1 John N. Zorich, Jr. Zorich Technical Consulting & Publishing Silicon Valley CA and Reno NV (cell) This copy of the Validation Protocol / Report for # ZTC-10, for version 2 of ZTCP's "MDD-PLUS C=0 SAMPLING PLANS.xls" was prepared and performed for use only** by and at by????? MDD-PLUS C=0 SAMPLING PLANS, you will be sent, free of charge, signed copies of the most current revision of the validation protocol and of the completely executed validation report. John N. Zorich, Jr. President/Owner of ZTC ** Use by any other parties or locations violates the license agreement under which ZTC sells this software. 12/18/ by ZTC Page 1 of 1

2 VALIDATION PROTOCOL For "MDD-PLUS C=0 SAMPLING PLANS.xls" 1.0 OBJECTIVE For the name and signature of the author of this Protocol, see the CoverPage of this combined Protocol + Report PDF file. This document outlines the validation protocol for version 2 of the computer file named "MDD-PLUS C=0 SAMPLING PLANS", referred to below only as "MDP". 2.0 SCOPE This document applies to the MS Excel file named in the "Objective" section above, but which might be named differently by a purchaser of this program. Even if the file name is changed, the "Instructions" worksheet inside the file will always have the original file name. This protocol applies to all versions 2.XX of this program, starting with version The XX refers to minor variations in the When program that you are designed purchase to change the a appearance copy of worksheets or to improve the clarity of error messages. The formulas/functions and "" sampling plan table cells remain unchanged in all 2.XX MDD-PLUS versions, and therefore re-validation C=0 SAMPLING is not required when changes are implemented in the 2.XX series. If a change is required to a formula/function cell or to a sampling plan table cell, PLANS, the new revision you will will be given be version sent, number free 3.01 and of formally revalidated. The file is password protected on all formula/function and "" sampling plan table cells (password retained by ZTCP). current Those cells cannot revision be changed without of the password, validation and ZTCP does not provide the password to purchasers of this program. 3.0 POLICY charge, signed copies of the most protocol and of the completely executed validation report. Custom statistical software must be formally validated for its intended use. The purpose of MDP is to calculate sample sizes that, for a given Lot (2 thru 10,000) exhibiting a given % defective (2%, 3%, 4%, 5%, 6%, 7%, or 10%), will result in no more than a probability that the Lot will be approved, based on a C=0 sampling plan (that is, if any defectives ("nonconformities") are found in the, the Lot is rejected). By using the word "validation" in this protocol, it is not meant to imply that MDP's program "code" is being validated. MDP is a relatively simple "application" based upon a commercial program, Excel, which has been sold by Microsoft Corporation for over 20 years. By saying that MDP is to be "validated" is to say that its functions and features will be critically and systematicly checked, to see if they meet their pre-determined design requirements and intended end-user purposes. MDP contains no Validation Protocol for MDD-PLUS C=0 SAMPLING PLANS.xls, version 2.XX, Page 1 of 5

3 "macros" or other code-based add-ons (such as VBA scripts); MDP does not even use any ActiveX controls. MDP contains only standard Excel functions and charts, and thus no additional code to validate. 4.0 REFERENCES Excel 2002 Formulas, by J. Walkenbach, 2002 by M&T Books. Juran's Quality Control Handbook, by Juran, J. M., and Gryna, F. M., editors, 4th edition, 1988 by McGraw-Hill Inc. Excel 2002 Formulas, by J. Walkenbach, 2002 by M&T Books. 5.0 DEFINITIONS MDP --- See "Objective" section above. All other terms are equivalent to the common definitions in standard statistical textbooks. 6.0 NOTES ON PROGRAM DESIGN MDP is a simple "lookup" application. Based upon user input on the " " worksheet, the program outputs the sample size and probability that it finds on a hidden " ". The " vs. Lot" curve and the "OC Curve" are created based upon that same input. The probabilities in the have all been calculated using the Hypergeometric Distribution (the Binomial Distribution was not used because it is an approximation to the exact Hypergeometric; the binomial approximation MDD-PLUS is a good approximation, C=0 except in SAMPLING the commonly encountered situation where sample size is a significant portion of the lot size). PLANS, you will be sent, free of charge, signed copies of the most To calculate a hypergeometric probability, 4 values are needed: Lot,, # of nonconformities in the Lot, and # of nonconformities in the. In order to calculate the probabilities, all lot sizes current from 2 thru revision were entered of into the a column validation in a %Defective-specific Spreadsheet (i.e., a separate Spreadsheet was created for 3%, and so on). The "# of nonconformities in the Lot" was calculated by protocol taking the corresponding and of %Defective the completely (e.g., 3%), multiplying it by the Lot, and then rounding down to the nearest whole number; the resulting values were then placed into a column next to the Lot executed column, in the corresponding validation Spreadsheet report. (where the result equaled zero, the value was manually changed to unity, i.e., " 1 ", so that subsequent calculations were performed as if every Lot had at least one nonconformity --- without such an assumption, the probability of acceptance would always be 100%, because there are no nonconformities to find in the sample!). Justification for "rounding down": The nature of the hypergeometric calculation is such that if any calculated fractional "# of nonconformities in the Lot" is "rounded up", the resulting probability is lower than if we had "rounded down". If we were to thus artificially inflate the # of nonconformities in the Lot, we would artificially lower the probability calculated; the problem with such a plan is that when the calculated probabilities are in the neighborhood of 0.05, this Validation Protocol for MDD-PLUS C=0 SAMPLING PLANS.xls, version 2.XX, Page 2 of 5

4 could lead to situations where the probability is less than but the true probability is greater than The # of nonconformities in the is set to zero for all situations (that is, C = 0 ). The 4th and final value needed for a hypergeometric calculation is the sample size. A semi-automated method was developed (using Excel) for use in each Spreadsheet, to identify the smallest sample size that would result in a hypergeometric probability of no larger than , for a given %Defective. The resulting column of 9,999 value pairs (sample size and corresponding probability) were then copy/pasted into the appropriate %Defective location in the, next to a column of corresponding Lot s. Unfortunately, the MS Excel Hypergeometric Distribution function ("=HYPGEOMDIST...") is unable to calculate probability for large lot sizes when the theoretical percent defective is 2%, because the intermediary factorial calculations (as performed by Excel) require astronomically large numbers (Microsoft Excel cannot calculate the factorial for any number greater than 170, and cannot handle numbers that have more than 308 significant digits). The bottom line is this: the Hypergeometric function in Excel, as used in MDP, cannot calculate probability for any lot size greater than 6815 when the theoretical percent defective is 2%. Therefore, for a 2% defective lot, the probabilities associated with lot sizes of 6816 and higher in MDP are determined "manually" by using Excel to calculate the ratios of factorials, which produces the exact same result as the Excel HYPGEOMDIST function (for the relevant formulas, see either Juran's text mentioned in the Reference section above, or the MS Excel "Help" file on HYPGEOMDIST). Such "manual" calculations are tedious to perform 3815 times ( = 3185), and no automated method was known to the designer of this program. In an effort to get past such a mountain of "manual" work, the designer of this program noted that sample sizes in the 2%Defective section of the for lot sizes from When 4751 through you 6815 purchase are exclusively either a copy 147 or 148. of Those two sample sizes appear in large sequences, for example starting at lot size 5650 there are 25 lot sizes where the MDD-PLUS C=0 SAMPLING sample size has been determined to be 147; the next 25 lot sizes have sample sizes of 148, and the 25 lot sizes following that have PLANS, sample sizes of 147. you For will be = sent, 148, the "manual" free of probability was found, by sampling, to slowly increase from (at lot size = 6815) to a maximum of (at lot size = 9999) and then fall charge, to at lot signed size 10,000. copies In addition, it of was the determined, most by sampling, that Lot would have be approximately in order for a sample size of 148 to result in a probability that would be greater than current Therefore, revision it was decided of to the use a validation sample size of 148 for all lot sizes from 6816 thru 10000, realizing that there might be a few lot sizes in that range for whom the smaller sample size of 147 might have protocol corresponded and to acceptable of the probability completely (that is, one that does not exceed ). 7.0 INSTALLATION VALIDATION executed validation report. MDP is a standard Microsoft Excel File, without added macros, ActiveX controls, or VBA scripts. To function correctly, the file needs to be used on a computer running Microsoft Windows (v 95, NT4, or later), that is running MSExcel version 8 or later (that is, "Excel 97" or later). (NOTE: Excel 97 thru Excel XP all use the same file format, per Walkenbach, p. 5, as referenced above; according to press releases from Microsoft, the version of Excel being sold for use with the newest version of Windows ("Vista"), has the same file format as does the Excel XP version.). Per Microsoft's website, Excel 2007 Validation Protocol for MDD-PLUS C=0 SAMPLING PLANS.xls, version 2.XX, Page 3 of 5

5 has a new file format ("XLSX") that has the capability of automatically configuring itself to be fully compatible with "XLS" files such as MDP. Installation validation is performed by verifying that the file is installed on such a computer. Since the file cannot run on any other type of installation, this validation is self-evident and will not be documented as part of implementing this protocol. 8.0 DESIGN VALIDATION 1. Design validation is first achieved by ensuring that no calculation cells nor any data / value containing cells are user modifiable on any worksheet. The only such cell that should be usermodified is the one for input of Lot, on the " " worksheet. Validation Criteria #1: The only user-modifiable cell on the " " worksheet is the one for input of Lot ; the only other usermodifiable cells in the program are on the "Doc#etc" worksheet. 2. The spreadsheet uses Excel "format controls" to output numbers that identify which %Defective situation has been chosen for analysis (i.e., 2%, 3%...or 10%). Such controls output integers (1, 2, 3...) which are then used by the VLOOKUP functions to search the appropriate column in the (e.g., if the user chooses "4%Defective" the format control output is "3", and the VLOOKUP function looks for a value in the 3rd column to the right of the Lot column in the ). Design validation of the "format controls" will not be performed, since if they were set up incorrectly, the Performance Validation (below) would fail. 3. There are no custom equations in MDP, only Excel "functions". Therefore, design validation is next achieved by checking that all the Excel "functions" have been entered correctly. The only functions that have been When used are the you VLOOKUP, purchase HLOOKUP, a and copy BINOMDIST of functions; if they were entered incorrectly, they would output an error message. Design validation will be accomplished by inspection, to MDD-PLUS determine that they do not C=0 output SAMPLING an error message; no other design validation of Excel functions will be performed because, if they have been otherwise constructed incorrectly, the PLANS, you will be sent, free of Performance Validation (below) would fail. charge, signed copies of the most current revision of the validation protocol and of the completely executed validation report. 9.0 PERFORMANCE VALIDATION Validation Criteria #2: None of the cells containing VLOOKUP, HLOOKUP, or BINOMDIST functions contain an error message. 1. This part of validation will make sure that the hypergeometric probability listed in the for a given Lot / combination is correct. This is to be done by re-calculating all of the approximately 75,000 probabilities directly from the values in the, using the HYPGEOMDIST function. For the 2% Defective part of the, larger than Lot of 6815, a sampling of at least 20 lot sizes will be "manually" calculated (see "Notes on Program Design, above). Validation Protocol for MDD-PLUS C=0 SAMPLING PLANS.xls, version 2.XX, Page 4 of 5

6 Validation Criteria #3: All of the probabilities listed in the must match exactly the probabilities calculated using the HYPGEOMDIST function, except for the part of the table that deals with 2%Defective lot sizes greater than 6815 (see Validation Criteria #4). Validation Criteria #4: Using the textbook formula (i.e., ratios of factorials) for the Hypergeometric probability, determine the largest probability in the range of Lot s 6816 to 10,000, for a 2%Defective lot, and ensure that the, in the part of the table that deals with 2%Defective lots, lists that probability exactly for every Lot in that range. MDD-PLUS C=0 SAMPLING PLANS, you will be sent, free of charge, signed copies of the most current revision of the validation protocol and of the completely executed validation report. Validation Criteria #5: Using the Excel "MAX" function, determine that none of the probabilities in the exceed NOTE: Validation will not check that the absolute smallest sample size possible has been chosen, but rather only that a large enough sample is chosen such that the resulting probability is no more than exactly. 2. Next, we assure that the output on the " " worksheet (which is performed using a VLOOKUP function) corresponds to the correct value on the. Validation Criteria #6: For a sampling of at least 20 different lot sizes thruout the range allowed by the program, ensure that the and Probability shown on the " " worksheet matches exactly the values in the corresponding row of the, as determined by visual inspection. This is to be done for each "%Defective". 3. Validation must also ensure that the vs. Lot chart is accurate, for the %Defective chosen on the " " worksheet. Validation Criteria #7: The vs. Lot chart created automaticly by MDP must reference the correct data in the, for each of the %Defective levels. 4. Finally, Validation must ensure that the OC Curve is accurate, for the %Defective chosen on the " " worksheet. Validation Criteria #8: The OC Curve created automaticly by MDP must reference the correct data in the, for each of the %Defective levels. Validation Protocol for MDD-PLUS C=0 SAMPLING PLANS.xls, version 2.XX, Page 5 of 5

7 VALIDATION REPORT For "MDD-PLUS C=0 SAMPLING PLANS.xls" For the name and signature of the author of this Report, see the CoverPage of this combined Protocol + Report PDF file. 1.0 INSTALLATION VALIDATION Self-evident -- see Validation Protocol for discussion. 2.0 DESIGN AND PERFORMANCE VALIDATION See Protocol for expanded explanations of Validation Criteria. MDD-PLUS C=0 SAMPLING PLANS, you will be sent, free of charge, signed copies of Valida- Validation Criteria Results of the Validation Criteria tion # met? 1 The only user-modifiable (By the means most of standard current MS Excel menu revision commands, and using of a copy of MDP cell on the " [not the File]...) "Protection" was removed from the " " worksheet is the " validation worksheet. The "format" protocol of the "Lot " and input of cell was then YES the one for input of Lot changed from "unlocked" to "locked". The entire worksheet was then selected. The only other usermodifiable cells in the examined --- the dark check mark on a white background in the "locked" (using the CTRL-A) completely and then the "protection" executed tab on the Format/Cells menu was program are on the checkbox indicates validation that all cells are report. now "locked". This means that the only cell "Doc#etc" worksheet. in the entire worksheet that was "unlocked" (prior to this validation test) was the "Lot " input cell. The same basic process was used to confirm that no cells or charts are user-modifiable on any of the other worksheets, except on the "Doc#etc worksheet." Validation Report for MDD-Plus C=0 Sampling Plans.xls, version 2.XX, Page 1 of 10

8 Valida- Validation Criteria Results of the Validation Criteria tion # met? 2 None of the cells The MS Excel function "SUM" will return an error-message if it is asked to sum containing VLOOKUP, a series of numbers, one or more of which contains an MS Excel error-message. HLOOKUP, or When all the cells containing VLOOKUP functions were SUMmed, the function YES BINOMDIST functions did not return an error-message. The same result was obtained when the contain an error message. HLOOKUP and BINOMDIST MDD-PLUS cells were SUMmed. C=0 SAMPLING 3 All of the probabilities New separate spreadsheets were set up for each %Defective. Each of these listed in the spreadsheets was confirmed** PLANS, to contain the exact you same will values be of Lot sent, and free YES must match exactly the as in the corresponding columns in in MDP. The probabilities calculated HYPGEOMDIST function was of then charge, used in those new signed spreadsheets copies to calculate of using the HYPGEOMDIST the probability of finding zero defects in the sample (the number of defectives in function, except for the the population, for each calculation, the was most determined current by the following revision formula: of part of the table that deals ROUNDDOWN((%Defective x Lot),0) with 2%Defective lot sizes as further explained in the Validation the validation Protocol). Then the protocol resultant probabilities and of greater than 6815 (see were compared** to the corresponding probabilities in the in MDP. the completely executed Validation Criteria #4). In all cases, the comparisons showed that the values being compared were identical. validation report. **NOTE: the "comparisons" were "confirmed" using MS Excel's IF function ---- columns of such functions were set up so that they outputted a value of zero (0.000) if the comparisons showed identical data, and a value of otherwise. The sum of the output of all such thousands of IF statements yielded a value of 0.000, thus indicating that all the comparisons showed that the spreadsheet values were identical to those in MDP. Validation Report for MDD-Plus C=0 Sampling Plans.xls, version 2.XX, Page 2 of 10

9 Validation # Validation Criteria Results of the Validation Criteria met? 4 Using the textbook formula (i.e., ratios of factorials) for the Hypergeometric Textbook formula calculation of Hypergeometric probabilities for a sampling of Lot s from 6816 to 10000, for a 2% Defective YES probability, determine the largest lot, when the is a constant 148, showed a slowly probability in the range of Lot s 6816 increasing When probability you up purchase to a lot size of 9999, and a copy then a slight of to 10,000, for a 2%Defective lot, and drop in probability at (see Appendix A for how the ensure that the, in the part of "textbook" MDD-PLUS calculation was performed C=0 using SAMPLING MS Excel, and the table that deals with 2%Defective lots, Appendix B for the output of the evaluation between lot sizes of lists that probability exactly for every Lot 6616 PLANS, and 10000). The you highest will probability be found sent, was for a free Lot in that range. 9999; that probability was: Using the MAX and MIN of functions charge, in Excel, it was signed confirmed that copies the only value of in the 2%Defective probability column in the MDP, the most current revision of for Lot s greater than 6815 is Using the Excel "MAX" function, Using the validation MS Excel "MAX" function, protocol it was determined and that of the determine that none of the probabilities in largest probability in the is YES the exceed (that is, the exactly equal completely to 1 / 20 ). executed 6 For a sampling of at least 20 different lot The lot sizes that were tested, and the results that were obtained, sizes thruout the range allowed by the are shown in Appendix validation C. In all cases, report. the and YES program, ensure that the and Probability shown on the " " worksheet matches exactly the Probability shown on the " " worksheet match exactly the values in the corresponding row of the. values in the corresponding row of the, as determined by visual inspection. This is to be done for each "%Defective". 7 The vs. Lot chart created automaticly by MDP must reference the correct data in the, for each of the %Defective levels. By visual inspection of the "source data" reference that is accessed by "right clicking" on a chart and choosing "source data", it was confirmed that when, e.g., 3%Defective has been chosen on the " " worksheet, the _vs._lot chart references the data corresponding to 3%, on the. The correct references were also found when checked vs. 2, 4, 5, 6, 7, and 10% Defective. Validation Report for MDD-Plus C=0 Sampling Plans.xls, version 2.XX, Page 3 of 10 YES

10 Valida- Validation Criteria Results of the Validation Criteria tion # met? 8 The OC Curve created automaticly by MDP By visual inspection of the "source data" reference that is must reference the correct data in the accessed by "right clicking" on a chart and choosing "source YES, for each of the %Defective data", MDD-PLUS it was confirmed that when, C=0 e.g., 3%Defective SAMPLING has been levels. chosen on the " " worksheet, the OC Curve PLANS, chart references you the Binomial will data be corresponding sent, to free 3%, on the. The correct references were also found when checked vs. 2, 4, 5, 6, 7, and 10% Defective. 3.0 EXCEPTIONS TO THE VALIDATION PROTOCOL: There were no exceptions. 4.0 CONCLUSION The validation criteria have all been met (with the "Exceptions" noted above), and therefore MDD-PLUS C=0 SAMPLING PLANS.xls is considered validated for its intended uses. 5.0 APPENDICES of charge, signed copies of the most current revision of the validation protocol and of the completely executed validation report. APPENDIX A (an example of how to perform a "manual" hypergeometric calculation) APPENDIX B (the results of manual hypergeometric calculations for lot sizes from 6816 to 10000) APPENDIX C (comparison of "" output vs. visual inspection of the ) Validation Report for MDD-Plus C=0 Sampling Plans.xls, version 2.XX, Page 4 of 10

11 APPENDIX A (an example of how to perform a "manual" hypergeometric calculation) A Lot 9,999 B % Defective in the Lot 2.00% C # Defective in Lot (rounding down), before start of sampling 199 Individual probability is (assuming that samples are taken one a time (in " Sequence #" order) and not returned to the Lot) the # Good Parts in Lot prior to taking the current sample, divided by the remaining size of the Lot prior to taking the current sample. That is... (A C (Sequence# 1)) / ( A (Sequence# 1)) MDD-PLUS C=0 SAMPLING PLANS, you will be sent, free of charge, signed copies of the most current revision of the validation protocol and of the completely executed validation report. Cumulative probability is the result of multiplying together all of the Individual Probabilities that have been calculated for all Sequence #s so far, including that from the current Sequence#. Sequence # Individual Cumulative and so on... and so on... and so on Validation Report for MDD-Plus C=0 Sampling Plans.xls, version 2.XX, Page 5 of 10

12 APPENDIX B (the results of manual hypergeometric calculations for lot sizes from 6816 to 10000) Lot Hypergeometric probability of zero nonconformities found in a of 148, if Lot is 2% defective MDD-PLUS C=0 SAMPLING PLANS, you will be sent, free of charge, signed copies of the most current revision of the validation protocol and of the completely executed validation report. Validation Report for MDD-Plus C=0 Sampling Plans.xls, version 2.XX, Page 6 of 10

13 APPENDIX C (comparison of "" output vs. visual inspection of the ) pg 1 of 4 2% Defective 3% Defective SAMPLE SIZE PROBABILITY SAMPLE SIZE PROBABILITY Same? Lot Lot Same? yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes MDD-PLUS C=0 SAMPLING yes yes yes PLANS, you will yesbe sent, 993 free yes yes yes of charge, signed yes copies 2974 of yes yes yes the most current yes revision 3970 of yes yes yes yes yes the validation protocol and of yes yes yes yes the completely yes executed yes yes yes validation yes report yes Validation Report for MDD-Plus C=0 Sampling Plans.xls, version 2.XX, Page 7 of 10

14 APPENDIX C (comparison of "" output vs. visual inspection of the ) pg 2 of 4 4% Defective SAMPLE SIZE PROBABILITY Same? Lot 5% Defective SAMPLE SIZE PROBABILITY Lot Same? yes yes When yes you 21 purchase a copy of yes yes yes yes MDD-PLUS 99 52C=052SAMPLING yes yes yes yes yes PLANS, you will be sent, free yes yes yes yes yes of charge, signed 55 copies of yes yes yes yes the most 1005 current revision of yes yes yes the yes validation protocol and of yes yes yes yes yes yes the 4099 completely 59 59executed yes yes yes yes yes validation report yes yes yes yes yes yes yes yes yes yes yes yes Validation Report for MDD-Plus C=0 Sampling Plans.xls, version 2.XX, Page 8 of 10

15 APPENDIX C (comparison of "" output vs. visual inspection of the ) pg 3 of 4 6% Defective 7% Defective SAMPLE SIZE PROBABILITY SAMPLE SIZE PROBABILITY Same? Lot Lot Same? yes yes When yes you 24 purchase a copy of yes yes yes yes MDD-PLUS C=035SAMPLING yes yes yes yes yes PLANS, yes 215 you 38 will 38 be sent, free yes yes yes yes of charge, signed 40 copies of yes yes yes yes the most 1014 current revision of yes yes yes yes yes the validation protocol and of yes yes yes yes yes the 4092 completely 42 42executed yes yes yes yes yes validation report yes yes yes yes yes yes yes Validation Report for MDD-Plus C=0 Sampling Plans.xls, version 2.XX, Page 9 of 10

16 APPENDIX C (comparison of "" output vs. visual inspection of the ) pg 4 of 4 Lot 10% Defective SAMPLE SIZE PROBABILITY Same? yes yes yes yes MDD-PLUS C=0 SAMPLING PLANS, you will be sent, free of charge, signed copies of the most current revision of the validation protocol and of the completely executed validation report yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes Validation Report for MDD-Plus C=0 Sampling Plans.xls, version 2.XX, Page 10 of 10