Dataworks Development, Inc. P.O. Box 174 Mountlake Terrace, WA (425) fax (425)

Transcription

1 Dataworks Development, Inc. P.O. Box 174 Mountlake Terrace, WA (425) fax (425) The Freezerworks Validation Verification Package Dataworks Development, Inc. has over 20 years of experience devoted to designing and revamping its Software Development Lifecycle, which includes program testing. When requested, we provide documentation of this validation to our customers as a means for them to measure our internal dedication to quality product design. This Validation Verification Package contains the following items for your examination: Software Development LifeCycle (SDLC) Validation Process Freezerworks Modules Validated in the Test Scripts Validation Completion by Round, Spreadsheet Samples of Completed Test Script Pages Certificate of Validation Final documentation of the Test Scripts, above, encompasses approximately 6000 printed pages. For ease of review, selected representative samples of pages have been provided. Please note that the Test Scripts, referenced above, were written to satisfy requirements of validation by a software development company. While Dataworks strives to include the most extensive variety of test cases and data ranges in the Test Scripts, they are in no way intended to replace the user validation required by the FDA. They are not a substitution for the laboratories responsibility to assess their own unique needs (functional requirements) and risk levels. FDA philosophy dictates that users draft and perform their own test plans, based on these requirements and risk assessments. Only when all rounds of validation test scripts have been completed satisfactorily, all testing procedures performed, and all documentation reviewed, will the project manager certify a version of the software for public release. This Certificate of Validation is included at the conclusion of this document. Every staff member at Dataworks is devoted to providing our customers with the highest quality software possible. We hope that this Validation Verification Package will illustrate some of the steps we take to do just that. Shannon Murray Quality Assurance and Project Manager Vice President - Dataworks Development, Inc. (425) shannon@dwdev.com

2 SDLC Validation Process The Dataworks Software Development LifeCycle (SDLC) was written in accordance with the FDA s Guidelines for Software Development and other current information technology standards. This extensive SDLC system governs the creation of our software from start to finish, including: User Requirements and Systems Analysis Functional Specifications Design Specifications Coding Phase and Programming Conventions Validation and Testing Distribution Tracking Post-distribution Modification Requests or Error Reports Users' Manual File and Document Storage The following excerpt from the SDLC lists the Dataworks policies and procedures covered in the Validation and Testing section. Acceptable performance is documented at each step of the testing procedures and all documentation is reviewed, signed, and dated by the Project Manager. A. Programmer Validation Each programmer is required to perform initial validation of his own work by running the program to verify compliance with the Functional Specifications. The programmer must provide documentation in the EMP showing the validation steps taken (e.g., a checklist or test plan) and any resulting documentation (e.g., screen cast, output, etc.) B. Unit Testing During the Coding Phase of the SDLC, the Development Team Leader will create compiled copies of the program just for unit testing to be performed by an inhouse Quality Assurance Technician. Each version created will be assigned a unique subletter. The features comprising each unit to be tested will be determined by the Development Team Leader, assuring limited domino effects and avoiding crashes with unfinished modules. 1. Test Plan Outlines Developers will create Test Plan Outlines for the QA Technician to use in Unit Testing. These documents will be named according to the module, Error Report or Modification Report to which they refer and should be stored under G:\Dataworks Products\Product Name & Version\Test Plan Outlines\Module Name. The author s name must be at the end of each Test Plan. These outlines will be the foundation for later creating formal Test Plans. The Test Plan Outlines should include basic test case situations for the QA Technician to use in coordination with his own judgment, extrapolation and intuition when testing each module. Last Updated 04/24/2012 Validation Verification SDLC summary.doc Page 2 of 7

3 2. Documenting Unit Testing Unit Testing documents will be maintained in G:\Dataworks Products\Product Name & Version\Validation\Unit Testing. a. The Development Team Leader and Project Manager will maintain an Excel spreadsheet called Unit Testing with at least the following fields: Module Name Developer Date Code Completed Version Number Assigned Unit Testing Completed b. The software tester will use a copy of the Test Plan Outline in this folder as a testing document. Version Number and Date of Testing will be added to the top of each document. c. When requested by the Development Team Leader or the Project Manager, the software tester will make a WebEx recording of each testing session so that steps can be easily recreated. Recording requirement will be determined based on risk assessment and/or complexity of the module. d. Errors found during Unit Testing will be logged in the E s and M s Database. A summary, verbal or written, of each Unit Testing session will be reported to the Development Team Leader. The Development Team Leader will assign coding correction to a programmer. e. Modifications may be requested by the software tester at this time and logged into the E s and M s Database. The Development Team Leader and Project Manager will determine if the modification is worthy of incorporation into the current project, must wait for a subsequent version, or is denied. C. Source Code Review Initiation of source code review will be at the discretion of the Development Team Leader based on risk analysis of the programs altered. Source code will be reviewed by the Development Team Leader or a qualified programmer designated by the Project Manager. The reviewer should verify that the source code is in compliance with Programming Conventions and the program's design document. The reviewer must electronically sign the appropriate section of the Errors and Modifications database record, if applicable. D. Write Test Cases Last Updated 04/24/2012 Validation Verification SDLC summary.doc Page 3 of 7

4 The test plan outline will be initially developed by the programmer prior to coding, in the design phase, for each module (functional unit of code) modification. The test plans will be stored in Word and will be named according to the module, Error Report, or Modification Report to which they refer. The author's name must be at the end of each Test Plan. The program name, version #, authorized signature and date will be on the first page of each section of the test plan. The test plan may be modified during coding to reflect changes in the program. The objective of the test cases is to determine if the program meets the program specifications and to find errors via high yield test cases. (High yield test cases have a good probability of finding errors and reduce the number of test cases needed.) Know the specifications Know the expected results (exact output from specific inputs) Use test cases for valid and invalid input Watch for side effects; with every change, the probability of errors increases; for every fix after a successful test, start over with first test case Document test cases to avoid repeating the same tests Documented test cases include expected results Complete test set must be validated when any new set is added 1. Test cases developed parallel to the software from program requirements and specifications (black box testing). a. Equivalence Partitioning Analysis Only considers the input domain Test cases for valid input and invalid input One test case for each element of each class or partition (1) Range of Values - 3 test cases One valid test case with a value within domain Two invalid test cases with values without domain - One less than the lowest domain value - One greater than the highest domain value e.g., 1<=X<=9 as the domain where x is a value valid: 1<=X<=9 invalid: X < 1 & X > 9 (2) Number of Values - 3 test cases One valid test case with number of value within domain Last Updated 04/24/2012 Validation Verification SDLC summary.doc Page 4 of 7

5 Two invalid test cases with number of values without the domain e.g., 1<=N<=6 as the domain where N is the number of values valid: 1<=N<=6 invalid: N < 1 & N > 6 (3) Enumerations of values (input values have individual identities, or specific values) n + 1 test cases (where n is the number of enumerations) N valid test cases with values within the domain One invalid test case with value without the domain e.g., the domain is red, blue, or white three valid test cases (red, blue, white) and one invalid test case (green) (4) Must values - two test cases (Single Value Domains) One valid value that equals the must value One invalid value that does not equal the must value (5) Special condition values One test case for each special condition b. Boundary Value Analysis Considers both input and output Test cases from edges of equivalence classes or partitions (1) Range Test - 4 test cases for each domain e.g., domain is 1<=X<=9 two valid test cases: 1 & 9 two invalid test cases: 0 & 10 (2) Number of values - 4 test cases two valid test cases: minimum number and maximum number of values two invalid test cases: min - 1 and max + 1 (3) Consider outputs - test case for each possibility e.g., a positive and a negative (input and output) for size, quantities, hours worked, payment, etc. Last Updated 04/24/2012 Validation Verification SDLC summary.doc Page 5 of 7

6 c. Error Guessing From specifications and intuition, what are the problem areas and where might the errors be 2. Test cases developed from program code review for each module (white-box testing) Does not test for logic errors Does not test for correct usage of boolean operators a. Statement Coverage Test cases to ensure every statement is executed at least once with the expected results Test every path b. Decision Coverage Test case for each true and false path for each decision box Does not look inside decision box c. Condition Coverage Tests inside decision boxes A true and a false test case for each condition within the decision box (tests the >, <, =, <>, <=, >= operators) d. Decision / Condition Coverage Test case for each condition for each true and each false path within the decision box e. Multiple Decision / Condition Coverage Test case each path of each decision Tests each condition within each decision box following each possible path 3. Test cases for testing modules and for integrated testing of the modules (testing the interface between the modules). a. Desk Checking: step by step examination of the code b. Inspection: inspection of the requirements, the design and the code to find faults c. Walk Through: visual and verbal step by step evaluation of the module to critique style, errors, development standards, areas of possible errors, etc. Last Updated 04/24/2012 Validation Verification SDLC summary.doc Page 6 of 7

7 d. Static Analysis: paper review to find logic errors, questionable coding practices, and other weaknesses of the source code 4. Test Cases for Systems Testing a. Stress Tests: test real time process for ability to handle and react to program interrupts b. Volume Tests: test system using the maximum amount of data over a period of time c. Usability Tests: machine - user interface d. Performance Testing: response times, throughput, number of transactions e. Storage Testing: compares the amount of storage actually used against the amount expected 5. The Test Plan will be reviewed and approved by the Development Team Leader and the Project Manager prior to its use for In-House Formal Validation. E. In-House Formal Validation Testing must be performed in-house by one or two people (depending on the size and scope of the program or module) according to the test plan. At least one of those performing the testing must not have participated in the programming of the project being tested. Testing Attitude Assume the program is incorrect For every error that is found, the probability of other existing errors increases A successful test will find an error Every modification will cause errors There will always be a combination of circumstances in the "real world" not accounted for in testing Testing does not prove correctness of a program, only that the program is correct for specific conditions (or "reliable across an expected domain producing expected results") F. Beta-site Testing Beta site testing on a user system will be performed after the in-house testing is complete and any errors corrected. Modifications requested or errors reported by the Beta site will be fully documented in the Errors and Modifications Database. Last Updated 04/24/2012 Validation Verification SDLC summary.doc Page 7 of 7

8 Dataworks must confirm with Beta sites that the tested programs comply with Functional Specifications. Confirmation must be documented and approved by the Project Manager. G. Reporting Errors or Requesting Modifications During the testing phase any errors or omissions found will be entered in the Errors and Modifications Database. These errors will be corrected and the validation process will begin again. H. Documentation Acceptable performance must be documented at each step of the testing procedures. All documentation of testing must be reviewed, signed, and dated by the Project Manager. 1. The test plan documentation includes the test cases, the expected results for each test case, and the actual results from each test case. Results will be verified through screen captures, reference reports, etc., which demonstrate that the logic flow is correct. 2. The test plan documentation will include: the source code tested the domain tested (black box testing) the critical pathways (white box testing) 3. After approval by the Development Team Leader and the Project Manager, the completed documentation will be kept in the program documentation notebooks. Last Updated 04/24/2012 Validation Verification SDLC summary.doc Page 8 of 7

9 Freezerworks Modules Validated in Test Scripts The following is an overview of modules covered in the Freezerworks Validation Test Scripts. Some program menu options are tested under multiple modules, and some modules contain validation of multiple menu options. See the Validation Completion spreadsheet for a list of Test Script documents. Initialization Data Files Serial Number Login System Properties Windows Option Multi-User System Configuration and Setup Client Licenses Menu Access (limited login) System Administration Users Groups Permissions Security Sample Types User Defined Fields Configure Samples Input Form Configure Samples List View Configure Aliquots List View Configure Transactions List View User Defined Reports. Audit Trail Samples Add, Modify, Delete Samples View Move Samples Check for Duplicates Create Duplicate Aliquots Add, Modify, Delete Aliquots Transactions View Sub-aliquoting Lineage Shipping Last Updated: 11/24/2010 Validation Verif FW Modules Validated.doc Page 1 of 2

10 Search Simple, Advanced, Quick Search, Scheduling, By Scanned Field, By Location, By ASCII File Add, View Transactions Batch Update Configure Batch Entry Enter and Update: Samples, Aliquots, New Positions, Transactions Freezers Configuration Add, Modify, Delete Security Explore Freezers (by alias, by tree, and by visual ) QC Check Freezer Import Data Export Data Printing Configure and Print Reports Configure and Print Labels Label Specifications Adjust Printer Settings Scanner alternate keyboard input Web Links Installation Windows Standalone Windows Server Mac Standalone Mac Server Windows and/or Mac Client Upgrading Older Versions From Freezerworks Basic to Freezerworks Unlimited From Older Freezerworks Unlimited to Current Freezerworks Unlimited Last Updated: 11/24/2010 Validation Verif FW Modules Validated.doc Page 2 of 2

11 Printed: 09/27/2013 Validation Verification FUL 6.0 Rounds.xls Page 1

12 Printed: 09/27/2013 Validation Verification FUL 6.0 Rounds.xls Page 2

13 Printed: 09/27/2013 Validation Verification FUL 6.0 Rounds.xls Page 3

14 Printed: 09/27/2013 Validation Verification FUL 6.0 Rounds.xls Page 4

15 Printed: 09/27/2013 Validation Verification FUL 6.0 Rounds.xls Page 5

16 Sample Pages of Validation and Test Plan Documentation A complete set of printed documentation is available for inspection at Dataworks. As referenced earlier, these samples should give a comprehensive overview of the kinds of documents required by our validation process. Sample 1: The Test Plan Outline As required by the SDLC, a Test Plan Outline must be created by the programmers to be used by the Validators in Unit Testing and in drafting the formal Test Plans. This particular change to the software affects the Shipping Module. The Test Plan Outline lists all items to be checked that are linked to that change.

17 Sample 2: Completed Test Script With No Problems The following is the first pages of a five page Test Plan chapter testing transaction activities. The Validator has performed and verified each step. The Project Manager has reviewed the page, as noted on the page footer. All printed copies are stored with the Project Manager.

18 Sample 3: Program Error Reported During Testing The following is a single page of a Test Script chapter within the Freezers Entry module. The Validator has performed each step, but noted one step in which the program did not function according to the script. The Validator reports the error by making an entry in the Dataworks internal EMP database, as noted by the initials next to Entered in DB: on the footer of this page. The database assigns the unique tracking number E30128 and the Validator makes a note of it. The Project Manager follows the detailed progress of the reported error through the database s Error Report, Disposition & Release, and Status History tabs.

19 (Sample 3 continued)

20 (Sample 3 continued)

21 Sample 4: Test Plan Error Discovered During Testing The following is a single page of a Test Plan chapter within the Aliquots module. The Validator noticed that the value for the row should be 1 not 7, as required by the specifications and displayed in the software. After consulting with a supervisor, a mistake in the Test Plan (noted with TP on the page) is determined. The Project Manager confirms, and corrects the Test Plan as noted by the initials next to Doc Corrected:

22 (Sample 4 continued) The Test Plan is corrected at step #55. The MS Word function track changes is used any time a Test Plan is modified. The heading of the document is also updated. Below is the resulting corrected document, ready for the next round of validation.

23 Sample 5: Confirmation Chart for Configurable Fields Due to the configurable nature of Freezerworks, the Test Scripts often contain charts of User Defined Fields for use when confirming the same item for a multitude of fields. The chart displayed below illustrates one set of data used in this round of validation. Each field name in the chart represents a User Defined Field set up to test a unique combination of limits and boundaries for each field type.

24 Sample 6: Testing Label Printing Printing labels and other reports is an important feature of Freezerworks. Below is an example of verifying the Barcode Label feature. When other printed reports are tested, a similar task is performed: printed materials generated are attached directly to the Test Plans before they are submitted to the Project Manager for review.

25 (Sample 6 continued)

26 (Sample 6 continued) Labels are attached to the submitted Test Plan.

27 (Sample 6 continued)

28 Sample 7: Confirmation Chart for Performance Requirements The following chart is an example of one of many pages of moves tested with the Move Samples option. Due to the configurable nature of Freezerworks, a multitude of freezers must be configured to test all of the various ranges, limits, and boundaries. The sample below shows that not only was each move performed successfully, but was also checked afterwards for accuracy in the Explore Freezers and Audit Trail options.

29