System Test Plan. Team: Kingpin. Project: Pin Deck Camera System. Team Members: Shawn Dobbins Jason Grey Eric Nelson Bhuwan Shrestha

Transcription

1 Department of Computer Science and Engineering The University of Texas at Arlington Last Updated: October 1 st :04am System Test Plan Team: Kingpin Project: Pin Deck Camera System Team Members: Shawn Dobbins Jason Grey Eric Nelson Bhuwan Shrestha

2 Table of Contents Table of Contents... 2 List of Figures... 6 List of Tables Introduction Document Overview and Purpose Scope Acronyms References Overview System Requirement Specification Key Requirements System Inputs and Outputs Architecture Design Specification Architecture Design Diagram Data Flow Definition Producer-Consumer Relationships Requirement Mapping Layer Data Descriptions Detailed Design Specification Detailed Design Diagram Data Flows Producer/Consumer Matrix Test Items Overview Relational Diagram Hardware Tests Unit Tests Interface Layer Control Layer October 1 st, 2013 P a g e 2 Team Kingpin

3 3.4.3 Image Processing Layer File I/O Layer Image Acquisition Layer Component Tests Interface Layer Control Layer Image Processing Layer File I/O Layer Image Acquisition Layer Integration Tests Interface Layer Control Layer Image Processing Layer File I/O Layer Image Acquisition Layer System Verification Tests Risks Overview Risk Table Testable Features Overview Customer Requirements Packaging Requirements Performance Requirements Non-Testable Features Overview Safety Requirements Camera Mounting Lighting Mounting Cabling / wiring User Safety Maintenance and Support Requirements October 1 st, 2013 P a g e 3 Team Kingpin

4 6.3.1 Source Code Troubleshooting Instructions System Maintenance Training Windows OS Support Code Modularity Team Galaxy Interface Integration Testing Approach/Strategy Overview Overall Test Strategy Methodology Testing Metrics Testing Requirements Item Pass/Fail Criteria Overview Hardware Tests Unit Tests Component Tests Integration Tests System Verification Tests Test Deliverables Overview Deliverables Test Schedule Overview Schedule Approvals Overview Approvals October 1 st, 2013 P a g e 4 Team Kingpin

5 Document Revision History Revision Number Revision Date Description Rationale /01/2013 Outline Initial creation of document /11/2013 Contribution Merge Inclusion of all individual contributions /17/2013 Document Review Final check and correction of document /17/2013 Baseline Submission Initial Submission of document October 1 st, 2013 P a g e 5 Team Kingpin

6 List of Figures Figure # Title Page # 2-1 Architecture Diagram Detailed Architecture Diagram Relational Diagram 20 October 1 st, 2013 P a g e 6 Team Kingpin

7 List of Tables Table # Title Page # 2-1 Key Customer Requirements System Inputs and Outputs Data Flow Definition Producer-Consumer Relationships Requirement Mapping Data Flows Producer-Consumer Matrix Hardware Tests Interface Layer Unit Tests Control Layer Unit Tests Image Processing Layer Unit Tests File I/O Unit Tests Image Acquisition Unit Tests Interface Layer Component Tests Control Layer Component Tests Image Processing Layer Component Tests File I/O Layer Component Tests Image Acquisition Layer Component Tests Interface Layer Integration Tests Control Layer Integration Tests Image Processing Layer Integration Tests File I/O Layer Integration Tests Image Acquisition Layer Integration Tests System Verification Tests Risk Table System Test Plan Schedule Approval Signatures 48 October 1 st, 2013 P a g e 7 Team Kingpin

8 1. Introduction 1.1 Document Overview and Purpose The System Test Plan Document provides a complete testing plan for the. The primary sources of reference for the development of the test plan include the SRS, ADS and DDS documents which were produced during the requirements specification and design phases of the project development process. The test items section illustrates the design decomposition and relates specific modules to the test plan through test cases. It also defines limitations of the product under test such as restrictions, assumptions, and caveats as well as other product level restraints on testing. In the risks section, specific risks are identified that may affect testing outcomes and provides an impact assessment and management plan as it relates to these risks. Features to be tested/not tested sections provide lists of the features that are to be tested or not accompanied with reasoning for not testing. The testing approach strategy outlines the methodology in which the system will be tested. The pass/fail criteria section defines the criteria in which the module, subsystem, or layer will pass or fail for all tests. The test deliverables section defines all artifacts that will be provided on completion of the project, while the testing schedule provides the timetable in which the various components of the system will be tested. 1.2 Scope The scope of testing for the PDTS is confined to the testing facility of the USBC. Any usage of the PDTS outside of the USBC testing facility will be considered outside the scope of this product, and any and all results will be considered invalid. 1.3 Acronyms USBC United States Bowling Congress PDTS SRS System Requirements Specification ADS Architecture Design Specification DDS Detailed Design Specification October 1 st, 2013 P a g e 8 Team Kingpin

9 2. References 2.1 Overview This section provides the relevant information from the references used to develop the test plan. These references are the SRS, ADS, and DDS documents produced prior to the implementation phase of the project. Each reference document provides specific information required to complete the test plan. The SRS subsection contains customer requirements and system inputs and outputs that will be used to determine tests for system integration and verification. The ADS subsection contains layers and data flows between layers which will be used to determine layer integration testing. The DDS subsection contains subsystem, modules and data flows, which will be used for unit and component testing development. 2.2 System Requirement Specification Key Requirements The key requirements table provides a listing of all the customer requirements considered for testing purposes. These requirements determine the testable and non-testable items. Number Name Description 3.1 Entry Board The system shall provide the board number the ball is on just before the ball strikes the first pin. 3.2 Exit Board The system shall provide the board number the ball is on just before leaving the pin deck. 3.3 Entry Angle The system shall provide the entry angle of the ball just before the ball strikes the first pin. 3.4 Exit Angle The system shall provide the exit angle of the ball just before leaving the pin deck. 3.5 Ball Speed The system shall output the speed of the bowling ball at a given position on the pin deck. 3.6 Ball Path The system shall output the position of the ball for every frame the ball is located on the pin deck. 3.8 Camera Calibration The system shall have the ability to calibrate the camera. 3.9 Data Output The system shall provide data output in a comma separated value file. 5.1 Calibration Performance The system s calibration module shall complete in an acceptable time frame 5.2 Data Analysis Performance The system s data analysis module shall process images in an acceptable time frame. 8.3 GUI A Graphical User Interface will be provided. Due to the time constraints of the project, GUI integration with Team Galaxy s B.O.L.T.S system is designated as a future item. Table 2-1: Key Customer Requirements October 1 st, 2013 P a g e 9 Team Kingpin

10 2.2.2 System Inputs and Outputs The system inputs and outputs table provides a listing of all inputs the system can expect as well as the outputs generated from any given input. This information is used to develop system integration testing as well as verification testing. Input: Name Description Use Start Capture Starts the camera and begins capturing video. The user will use this input when they are ready to being capturing video. Calibrate Calibrates the camera in order to provide the best possible estimates. The user will use this input when they wish to calibrate the camera, such as when the camera is moved. Filename Sets the name of the output file. The user will use this input when they decide which directory to output the data from the run. Export Video File A Boolean value describing whether the video for the run should be output to a file. The user will set this input if they desire to output the video for the run to a file. Export CSV File A Boolean value describing whether the data for the run should be output to a file. The user will set this input if they desire to output the data for the run to a file. Video from Camera Slow motion video of the ball hitting the pins will be input and then analyzed. Output: Entry Angle The entry angle will be the angle at which the ball hits the pins measured positive to the right of the center board and negative to the left. Exit Angle The exit angle will be the angle at which the ball leaves the back of the pin deck. Entry The entry board will be the board that the Board ball is on when it first touches a pin. Exit Board The exit board will be the board that the ball is on when it leaves the pin deck. Ball Speed The ball speed across the pin deck will be measured. Ball Path The path of the ball from the point of entry to the point of exit will be tracked. Calibration Notification Progress Notification Saved Video After the system has finished calibrating, the system will output a notification of the calibrations success or failure. During a capture or a saving event the current status will be output to the user. Video of the ball as it moves through the pins. Table 2-2: System Inputs and Outputs The video of the ball will be input to be analyzed so the data of the run can be printed or written to a file. The entry angle will be displayed to the user or written to a file. The exit angle will be displayed to the user or written to a file. The entry board will be displayed to the user or written to a file. The exit board will be displayed to the user or written to a file. The ball speed will be displayed to the user or written to a file. The ball path will be displayed to the user or written to a file. The user will be notified of the calibrations success or failure. This will display the progress of the current operation If the saved video option is selected, the video will be saved to the disk and stored in a file. October 1 st, 2013 P a g e 10 Team Kingpin

11 2.3 Architecture Design Specification Architecture Design Diagram The Architecture Design Diagram provides a high-level overview of the system layers, the layer subsystems and the data flows between the layers and subsystems. This information will be used to develop the component testing of the subsystems as well as the integration testing of each layer. User Interface Layer I1 GUI I5 I2 User Interface Controller Control Layer C6 I3 I4 C7 Event Manager Calibration Manager Image Processing Layer C1 C5 C2 P2 File Output Layer P1 Locate Ball Locate Pin Deck Grid File Output Controller C3 C4 F1 F2 P3 Detect Motion Detect Pin Movement CSV File Output Video File I/ O Calibration I/O File P4 Image Acquisition Layer P5 A3 F3 Camera Interface A2 Image Preparation A1 Camera Figure 2-1: Architecture Diagram October 1 st, 2013 P a g e 11 Team Kingpin

12 2.3.2 Data Flow Definition The Data Flow Definition Table provides information on the data flows between the layers and subsystems. As with the Architecture Design Diagram this information is used to develop the component testing of the subsystems as well as the integration testing of each layer. Data Flow Description I1 I2 I3 I4 I5 C1 C2 C3 C4 C5 C6 C7 P1 P2 P3 P4 P5 F1 F2 F3 A1 A2 A3 Information about the user s interaction with the GUI, such as clicks. Information about what GUI component was manipulated. The types of files the user would like to output. Notification that the system should be calibrated and the result of the calibration operation. Any information that will be displayed to the user in the GUI. The size of the ball and context information about the ball location. System context information. The size of the ball and system state. System context information and system state. The types of files the user would like to output and the data that should be output for those files. Information such as the data for the run that should be displayed to the user in the GUI. Result of the calibration operation. Pixel location of the ball within the image. Pixel locations of the four corners of the pin deck within the image. A Boolean value representing whether or not motion was detected. A Boolean value representing whether or not pin motion was detected. A request for the next image in the video stream. Data to be output to a CSV file. Video data to be written to a file. Correctly formatted data written to a file. Raw video from the camera. A single frame from the camera that is to be analyzed. A single frame in the correct format that is ready for the imageprocessing layer. Table 2-3: Data Flow Definition Table October 1 st, 2013 P a g e 12 Team Kingpin

13 Producers GUI User Interface Controller Event Manager Calibration Manager Locate Ball Locate Pin Deck Detect Motion Detect Pin Movement File Output Controller CSV File Output Video File Output Camera Interface Image Preparation System Test Plan Producer-Consumer Relationships The Producer-Consumer Relationships Table provides relationships between the output and input of subsystems. This information will be useful in development of the component testing of the subsystems in addition to the integration testing of the layers. KEY Consumers Interface Layer Control Layer Image Processing Layer File Output Layer Image Acquisition Layer GUI User Interface Controller I2 I5 I3 I4 Event Manager C6 C1 C3 C4 C5 Calibration Manager C7 C2 Locate Ball P1 P5 Locate Pin Deck P2 P5 Detect Motion Detect Pin Movement File Output Controller CSV File Output Video File Output Camera Interface Image Preparation P3 P4 A3 A3 A3 A3 Table 2-4: Producer-Consumer Relationships F1 F2 P5 P5 A2 October 1 st, 2013 P a g e 13 Team Kingpin

14 2.3.4 Requirement Mapping # Name Interface Layer Control Layer Image Processing Layer File Output Layer Image Acquisition Layer 3.1 Entry Board X X X X 3.2 Exit Board X X X X 3.3 Entry Angle X X X X 3.4 Exit Angle X X X X 3.5 Ball Speed X X X 3.6 Ball Path X X 3.7 Pin Movement 3.8 Camera Calibration X X 3.9 Data Output X 5.1 Calibration Performance X X X 5.2 Data Analysis Performance X X 8.3 Team Galaxy Interface Integration X 8.7 Leftover Pins X X X Table 2-5: Requirement Mapping October 1 st, 2013 P a g e 14 Team Kingpin

15 2.3.5 Layer Data Descriptions Interface Layer The interface layer will receive data from the end user through the user s interaction with the GUI. The GUI will have clickable buttons, check boxes, and radio buttons allowing user to manipulate the system. The user interface controller will then take actions in response to manipulations of the GUI by the user. Depending upon the input type, the user interface controller will send the information to either the event manager or the calibration manager of the control layer. The user interface controller also receives responses back from control layer, which will be displayed to the user via the GUI Control Layer The control layer will receive information from the interface layer about which type of output the user has selected. The control layer then sends this information to the file output layer. The control layer will be receiving information from the image-processing layer. The calibration manager will receive the four corners of the pin deck from the image-processing layer. The event manager will receive information about ball motion, pin motion, or the location of the ball depending on the state of the system Image Processing Layer The image-processing layer will receive an image from the image acquisition layer upon request File I/O Layer The file output layer will receive information from the event manger subsystem of the control layer. The control layer will send the file type information and other data such as the ball position, speed, angle, entry board, and exit board to file output controller subsystem of the file output layer. Depending upon user s choice of output file types, the file output controller will send the data to either CSV file output module or video file output module. Then, the file output module will save the data to the disk Image Acquisition Layer The image acquisition layer will receive raw images from camera and prepares images to feed into image processing layer. First camera interface module will receive raw images from camera and those captured images will be sent to image preparation module. Image preparation module will determine if raw images need to be preprocessed or formatted. After images are formatted and preprocessed as required, the image preparation subsystem will send images to image processing layer. October 1 st, 2013 P a g e 15 Team Kingpin

16 2.4 Detailed Design Specification Detailed Design Diagram The Detailed Design Diagram provides an overall view of the system modules. User Display I1 Interface Layer GUI User Interface Controller I6 Button Options GUI Output Input Controller Output Controller I2 I3 I4 Control Layer I5 Event Manager C19 C15 State Controller C24 C1 C2 C3 C4 C12 FOV Before Pin Deck Before Pin Contact Off Pin Deck C5 Real World Conversion C6 C26 C7 C8 C27 C9 C10 Image Processing Controller C11 C21 C22 C23 Calibration Manager C20 C16 Calibration (TIMER) C13 Pin Deck Locator C17 C25 C18 C14 Image Processing Layer File I/O Layer Locate Ball Subsystem Locate Pin Deck Subsystem P8 File I/O Controller Erode and Dilate P11 Checkerboard Input Output Image Crop Circle Crop P12 Expansion P13 Center P5 Pin Deck Corners P9 F3 F2 F1 F4 F5 CSV File Output Video File I/O Calibration I/O Detect Motion Subsystem Detect Pin Movement Subsystem P4 P7 P3 Image Difference P6 Pin Movement Controller File Format F6 Video In Video Out XML Parser XML Encoder File P10 P14 P1 P2 CSV Write Background Subtraction Pin Crop F7 F11 F8 F10 F9 Image Acquisition Layer Camera Interface Subsystem Image Preparation Subsystem Interface Contoller A8 A1 A2 A3 A7 Video Conversion On/Off Options Capture A4 A5 A6 Camera Figure 2-2: Detailed Architecture Diagram October 1 st, 2013 P a g e 16 Team Kingpin

17 2.4.2 Data Flows The module data flows chart provides a description for every data flow between each module within the system. This information is used to develop all system testing. Data Flow Description I1 User clicks/interactions with the GUI. I2 Unsigned integer ID of button clicked. I3 Unsigned integer ID of checkbox clicked. I4 Request for calibration I5 Array of Booleans for each option in the GUI. I6 String to update GUI. C1 Next frame in the video stream (Mat object). C2 Next frame in the video stream (Mat object). C3 Next frame in the video stream (Mat object). C4 Next frame in the video stream (Mat object). C5 Location (Real World) structure. C6 Mat objects (2) to be processed. C7 Mat object to be processed. C8 Mat object to be processed. C9 Mat object to be processed. C10 PixelLocation (Pixel) structure. C11 Intrinsic and Extrinsic value matrices and an array of PixelLocation[4] representing the four corners of the pin deck. C12 Ball object to be output to a file and array of Booleans for the type of files to output. C13 Location[4] representing the four corners of the pin deck in the real world. C14 Intrinsic and Extrinsic value matrices and an array of PixelLocation[4] representing the four corners of the pin deck. C15 Boolean of whether to start capturing video or stop capturing video. C16 Boolean of whether to start capturing video or stop capturing video. C17 Mat object to be processed. C18 Mat object to be processed C19 Ball object to be output to the GUI. C20 Result of calibration unsigned integer. C21 Mat object to be processed. C22 Mat objects (2) to be processed C23 Mat objects (2) to be processed C24 The next frame in the video to be processed. October 1 st, 2013 P a g e 17 Team Kingpin

18 C25 C26 C27 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 A1 A2 A3 A4 A5 A6 A7 A8 The next frame in the video to be processed. Boolean value of whether or not motion was detected. Boolean value of whether or not pin motion was detected. Mat object to be cropped. Cropped Mat object. Cropped Mat object. A Boolean value whether or not motion was detected. PixelLocation of the center of the ball and the radius. A Boolean value whether or not motion was detected. Boolean value representing whether or not pin motion was detected. Intrinsic and Extrinsic value matrices. The four corners of the pin deck in PixelLocation[4] The two frames to detect motion (Mat objects). Mat object to be cropped. Cropped Mat object. Mat object to be processed. Boolean whether or not motion was detected. Mat object (next frame in video stream). Intrinsic and Extrinsic matrices and PixelLocation[4] for the four corners of the pin deck. Ball object to be written to a file. AVI encoded video structure. Intrinsic and Extrinsic matrices and PixelLocation[4] for the four corners of the pin deck. CSV formatted data to be written to disk. CSV file. AVI file. XML file. XML file. AVI file. Boolean whether the camera should turn on or off. Integer frame rate, other options. Boolean whether to start or stop a capture. Boolean whether the camera should turn on or off. Integer frame rate, other options. Boolean whether to start or stop a capture. Cine video structure. AVI video structure. Table 2-6: Data Flows October 1 st, 2013 P a g e 18 Team Kingpin

19 Img Prep Camera Interface Subsystem Calibration I/O Video File I/O CSV File Output File I/O Controller Detect Pin Movement Subsystem Detect Motion Subsystem Producers Locate Pin Deck Subsystem Locate Ball Subsystem Calibration Manager Event Manager UI Controller GUI Button Options GUI Output Input Controller Output Controller State Controller FOV Before Pin Deck Before Pin Contact Off Pin Deck module Real World Conversion Image Processing Controller Calibration Pin Deck Locator Erode and Dilate Image Crop Circle Center Crop Expansion Checkerboard Pin Deck Corners Image Difference Background Subtraction Pin Movement Controller Pin Crop Input Output File Format CSV Write Video In Video Out XML Parser XML Encoder Interface Controller On/Off Options Capture Video Conversion System Test Plan Producer/Consumer Matrix KEY Consumers Interface Layer Control Layer Image Processing Layer File Output Layer Image Acquisition Layer GUI UI Controller Event Manager Calibration Manager Locate Ball Subsystem Locate Pin Deck Subsystem Detect Motion Subsystem Detect Pin Movement Subsystem File I/O Controller CSV File Output Video File I/O Calibration I/O Camera Interface Subsystem Img Prep Button I2 Options GUI Output I3 Input Controller I5 I4 Output Controller I6 State Controller C19 C1 C2 C3 C4 C12 C15 FOV Before Pin Deck Before Pin Contact Off Pin Deck Real World Conversion Image Processing Controller C5 C5 C5 C6 C7 C8 C9 C26 C27 C10 C21 C22 C23 Calibration C20 C17 C14 C16 Pin Deck Locator C13 C18 Erode and Dilate P5 P11 P13 P15 Image Crop Circle Center Crop Expansion Checkerboard Pin Deck Corners Image Difference Background Subtraction Pin Movement Controller Pin Crop P12 P14 P16 P8 P9 P6 P10 P4 P17 P7 P3 P1 P2 Input Output F3 F4 F5 File Format F6 CSV Write Video In Video Out XML Parser XML Encoder F1 F2 Interface Controller A1 A2 A3 On/Off Options Capture A7 Video Conversion Table 2-7: Producer-Consumer Matrix A8 October 1 st, 2013 P a g e 19 Team Kingpin

20 3. Test Items 3.1 Overview This section will provide detail covering the five phase testing plan needed to ensure the creation of a successful project. The following relational Diagram (Figure 3.1) represents the five testing phases in a flowchart layout. The five phases are as follows; hardware testing, unit testing, component testing, integration testing, and system verification. The layout of this test plan allows us to show that once everything in a phase is complete, the data received from the modules in that phase can be reasonably assumed correct. This allows us to focus almost completely on the tests on our current phase. 3.2 Relational Diagram Figure 3-1 Relational Diagram October 1 st, 2013 P a g e 20 Team Kingpin

21 3.3 Hardware Tests ID Hardware Component Predecessors Input Expected Output Priority HW1 Desktop Computer User input through the keyboard and mouse Correct feedback to the user via the GUI and program HW2 Phantom Miro speed camera API calls Video file that can be saved to disk. Table 3-1: Hardware Tests 3.4 Unit Tests Interface Layer ID Predecessors Input Expected Output Verification Priority UI1 Button Mouse click Return buttonid if button is clicked. The buttonid belongs to logged button Medium UI2 Options Radio Buttons Selected Array of Booleans corresponding to chosen options A simple output comparison to correct output Medium UI3 GUI Output Ball object Matching ball data and GUI data. Visual Inspection Medium UI4 Input Controller Event ID & Options array Event ID determines specific function calls The correct function calls are made and data is passed without corruption. Medium UI5 Output Controller Ball object or Calibration results Information from data structures is displayed on GUI Visual Inspection Medium Table 3-2: Interface Layer Unit Tests October 1 st, 2013 P a g e 21 Team Kingpin

22 3.4.2 Control Layer ID Predecessors Input Expected Output Verification Priority UC1 State Controller Image with trigger or Ball Object Same Image or Ball Object in the correct state i.e. the next in succession State conditions must be logged and checked after a run UC2 FOV Mat Object Boolean value of motion detected or not detected. Assert with expected results UC3 Before Pin Deck Mat Object & Ball Location Ball Location as ball enters Pin deck. Assert with expected results UC4 Before Pin Contact Mat Object & Ball Location Bool flag for contact detected & collection of Mat Objects Manual verification UC5 Off Pin Deck Mat Object & Ball Location Bool flag for Off Pin Deck detected Assert with expected results UC6 Real World Conversion Pixel Location & Calibration Matrices Real World Location Assert with expected results UC7 Image Processing Mat Objects & Pixel Locations Mat Objects & Pixel Locations and correct function calls Assert with expected results Medium UC8 Calibration Calibration Matrices, Request for calibration Calibration matrices and pixel locations of deck corners Real world values would be measured against the results UC9 Pin Deck Locator Mat Object, Pixel Locations Four coordinate points representing the corners of the pin deck Manual comparison between precomputed and current run. Table 3-3: Control Layer Unit Tests October 1 st, 2013 P a g e 22 Team Kingpin

23 3.4.3 Image Processing Layer ID Predecessors Input Expected Output Verification Priority UC1 Erode & Dilate Mat Object UC2 Image Crop Mat Object Processed Mat Object Processed Mat Object Manual inspection to ensure proper filters applied Manual Inspection to ensure proper crop applied UC3 Circle center Mat Object Pixel Location Manual Inspection UC4 Crop Expansion Mat Object Expands area around location of the previous Images ball location Manual Inspection that ball is within the cropped region UC5 Checkerboard Mat Object Calibrated matrices values to be written to disk Manual verification based on previous good inputs UC6 Pin Deck Corners Mat Object Vector of 4 pixel locations at which the corners are located Manual verification. Verify against known readings UC7 Image Difference Mat Objects[2] Bool value indicating movement Assert with expected results UC8 Background Subtraction Mat Objects[2] Bool value indicating movement Assert with exposed result UC9 Pin Movement Controller Mat Objects[2], motion flag Cropped Mat Object Manual verification through visual inspection Low UC10 Pin Crop Mat Objects[2] Cropped Mat Object Manual Verification through visual inspection Low Table 3-4: Image Processing Layer Unit Tests October 1 st, 2013 P a g e 23 Team Kingpin

24 3.4.4 File I/O Layer ID Predecessors Input Expected Output Verification Priority The intrinsic and UF1 extrinsic matrices Ensure values from Input Request for calibration for the camera and files match values data. the PixelLocations returned of the four corners Medium of the pin deck. UF2 UF3 UF4 UF5 UF6 UF7 UF8 UF9 UF10 Input Output Output Output File Format CSV Write Video In Video Out XML Parser Request for next frame in video. Ball object containing results and array of Booleans. The intrinsic and extrinsic matrices for the camera and the PixelLocations of the four corners of the pin deck. AVI video structure. Ball object to be written. CSV formatted data. Request for next frame. AVI video structure. Request for calibration data. Mat object of next frame in the video. Ball object to File Format module The intrinsic and extrinsic matrices for the camera and the PixelLocations of the four corners of the pin deck to the XML Encoder. AVI video structure to the Video Output module. Correct formatted CSV data. A file with the formatted data written. Mat object of next frame in the video. AVI file with video for run. The intrinsic and extrinsic matrices for the camera and the PixelLocations of the four corners Ensure next frame is returned (manual). Ensure correct data is output in file (manual). Ensure correct data is saved to XML file (manual). Ensure correct video saved to disk (manual). Parse formatted data and compare with original. Manual inspection. Ensure next frame is returned (manual). Manual inspection. Comparison of data in file with data returned. October 1 st, 2013 P a g e 24 Team Kingpin Medium Medium

25 UF11 XML Encoder The intrinsic and extrinsic matrices for the camera and the PixelLocations of the four corners of the pin deck. of the pin deck. XML endcoded data file. Table 3-5: File I/O Layer Unit Tests Manual inspection. Medium Image Acquisition Layer ID UA1 UA2 UA3 UA4 UA5 UA6 Interface Controller On/Off Options Capture Capture Video Conversion Predecessor s Input Expected Output Verification Priority Boolean representing to start capturing video or to stop. Boolean whether to turn the camera on/off. Request to initialize camera options. Requests for the capture to start. CINE video structure CINE video structure The camera should turn on and begin capturing video if the Boolean is true or turn off and stop if it is false. The camera should turn on if the Boolean is ture or off otherwise. Hard coded values sent to camera. The camera should begin to capture video. Full CINE encoded video. Full AVI encoded video Table 3-6: Image Acquisition Layer Unit Tests Ensure a video is saved when a true Boolean is passed first and later is passed a false. Ensure the camera is on when a true Boolean is passed and off otherwise. After a video is saved ensure that the video has the correct FPS and other options. First request the camera to start capturing and later to stop. Then verify the video began at the correct time. Ensure video conversion module receives full video by manual inspection. Play back saved AVI to ensure correct encoding of video. Low Medium October 1 st, 2013 P a g e 25 Team Kingpin

26 3.5 Component Tests Interface Layer ID Subsystem Predecessors Input Expected Output Verification Priority CI1 GUI UI1-UI3 Options and clicks from the user, Ball Object Boolean option arrays and buttonclickid. Program window fills in result data Unit testing using simulated click and option patterns Medium CI2 User Interface Controller UI4-UI5 Boolean array of options and buttonclickid, Ball Object Depending on the buttonclickid passed, separate function calls are made. Packages data to be set to the GUI. Compare function calls with correct function calls, assert with expected results Medium Table 3-7: Interface Layer Component Tests Control Layer ID Subsystem Predecessors Input Expected Output Verification Priority CC1 Event Manager UC1-UC7 Request for a system run Set proper system state and call appropriate functions Simulated inputs, examination of state changes with accompanying video CC2 Calibration Manager UC8-UC9 Request for camera calibration Intrinsic & extrinsic matrices, calibration measurements Manual Inspection Image Processing Layer Table 3-8: Control Layer Component Tests ID Subsystem Predecessor Input Expected Output Verification Priority CP1 Locate Ball UC1-UC4 Mat Object Pixel location of ball Assert with known results October 1 st, 2013 P a g e 26 Team Kingpin

27 CP2 Locate Pin Deck UC5-UC6 Mat Object Calibration Data Manual Inspection CP3 Detect Motion UC7-UC8 Mat Objects[2] Bool value for motion detected Assert with known results CP4 Detect Pin Movement UC9-UC10 Mat Objects[2] Bool value for motion detected, cropped Mat objects Manual Inspection, ball must not be visible in cropped image Low Table 3-9: Image Processing Layer Component Tests File I/O Layer ID Subsystem Predecessors Input Expected Output Verification Priority Ensure correct Requests for input The correct module File I/O methods of the CF1 UF1-UF5 or output and any being called to get Medium Controller correct classes are necessary data. input or put output. called via a log. CF2 CSV File Output UF6-UF7 CF3 Video File I/O UF8-UF9 CF4 Calibration I/O UF10-UF11 Ball object. Requests for next frame or video data to be written. Request for or the intrinsic and extrinsic matrices for the camera and the PixelLocations of the four corners of the pin deck. CSV formatted file written to disk. AVI encoded video data or next frame in video. The intrinsic and extrinsic matrices for the camera and the PixelLocations of the four corners of the pin deck or that data written correctly to XML file. Table 3-10: File I/O Layer Component Tests Verify printed values with values in file. Ensure correct video data is output in file (manual). Also, verify next frame is returned to correct module. Ensure correct data is saved to XML file (manual). Also, verify calibration data is returned to correct module. Medium October 1 st, 2013 P a g e 27 Team Kingpin

28 3.5.5 Image Acquisition Layer ID Predecessors Input Expected Output Verification Priority Boolean Camera Manual playback of representing to Full encoded CINE CA1 Interface UA1-UA5 video after being start capturing video. Subsystem saved. video or to stop. CA2 CA3 Camera Interface Subsystem Image Preparation Subsystem UA1-UA5 UA6 3.6 Integration Tests Interface Layer CINE video structure CINE video structure Full encoded CINE video structure. Full AVI encoded video structure. Table 3-11: Image Acquisition Layer Component Tests Manual playback of video after being saved. Ensure the video output module gets video data by verifying the correct video is written to a file. ID Input Predecessor Expected Output Verification Priority II1 User Operations Log the user interactions and compare with interactions with the GUI. CI1 chosen by user to be executed. action system took. II2 Results from calibration or system run CI2 Results to be displayed to user. Ensure correct data appears on GUI by manual inspection. Table 3-12: Interface Layer Integration Tests Control Layer ID Input Predecessor Expected Output Verification Priority IC1 Array of Files with data Log the user interactions and compare with Booleans of from run action system that was taken. CC1 which files to output. IC2 Results from calibration or system run CC2 Results to be displayed to user. Table 3-13: Control Layer Integration Tests Ensure correct data appears on GUI by manual inspection. October 1 st, 2013 P a g e 28 Team Kingpin

29 3.6.3 Image Processing Layer ID Input Predecessor Expected Output Verification Priority IP1 Mat object to process CP1-CP4 PixelLocation of center of ball. Manually verify the center of the ball in frame matches PixelLocation returned. IP2 Two Mat objects to process CP1-CP File I/O Layer Boolean value of whether motion was detected. Table 3-14: Image Processing Layer Integration Tests Manually verify motion detected Boolean matches whether there is motion in the two frames. ID Input Predecessors Expected Output Verification Priority Requested data to Ensure correct data from file is returned be constructed if Requests for file from this layer to the correct module by IF1 CF1-CF4 necessary and input. calling methods and comparing results returned to the returned. correct module. IF2 Requests for file output and any necessary data. CF1-CF Image Acquisition Layer Requested data formatted correctly and output to a file on the disk. Table 3-15: File I/O Layer Integration Tests Ensure correct data is written to file by printing or comparing the known values with what is written in the file. ID Input Predecessors Expected Output Verification Priority IA1 Boolean Ensure the video output module gets video representing to Full AVI encoded CA1-CA2 data by verifying the correct video is written start capturing video structure. to a file. video or to stop. IA2 CINE video structure from camera. CA3 Full AVI encoded video structure. Table 3-16: Image Acquisition Layer Integration Tests Ensure the video output module gets video data by verifying the correct video is written to a file. October 1 st, 2013 P a g e 29 Team Kingpin

30 3.7 System Verification Tests ID Requirement Predecessor Input Expected Output Verification Priority The board at which System will be run the ball first contacts Manually verify for one video All integration the pin is correctly entry board SV1 Entry Board capture of the ball Critical tests output to the file and matches calculated across the pin displayed to the entry board. deck. user. SV2 SV3 SV4 SV4 SV5 SV7 Exit Board Entry Angle Exit Angle Ball Path Ball Speed Calibration All integration tests All integration tests All integration tests All integration tests All integration tests All integration tests System will be run for one video capture of the ball across the pin deck. System will be run for one video capture of the ball across the pin deck. System will be run for one video capture of the ball across the pin deck. System will be run for one video capture of the ball across the pin deck. System will be run for one video capture of the ball across the pin deck. System will take a series of images of an empty pin deck. The board at which the ball first leaves the pin is correctly output to the file and displayed to the user. The angle at which the ball travels when it first contacts the pins is correctly output to the file and The angle the user. at which the ball travels when it first leaves the pin deck is correctly output to the file and the user. The correct path of the ball across the pin deck displayed to user. The average speed of the ball across the pin deck is output to the user. The intrinsic and extrinsic matrices for the camera and the PixelLocations Manually verify exit board matches calculated exit board. Manually verify entry angle correlates with calculated entry angle. Manually verify exit angle correlates with calculated exit angle. Manually verify position for each point in the ball path correlates with calculated position. Manually calculate the average speed and verify it correlates with the systems calculated value. Manually verify the matrices with a known camera position October 1 st, 2013 P a g e 30 Team Kingpin Critical Critical Critical Medium Medium Critical

31 SV8 SV9 System Performance Data Output All integration tests All integration tests System will be run for one video capture of the ball across the pin deck. System will be run for one video capture of the ball across the pin deck. of the four corners of the pin deck or that data written correctly to XML file. The data for a run within a reasonable amount of time. The correct data for a run of the system output to a file. Table 3-17: System Verification Tests Time the run time of the system. Manually verify the file is output as a comma separated value file. Medium Critical October 1 st, 2013 P a g e 31 Team Kingpin

32 4. Risks 4.1 Overview The risk section shall identify any risks associated with the System Test Plan for the Pin Deck Tracking System. All risks found in test phase will be given Risk ID and the impact of the risk shall be assessed in the risk table. Once risk is found its severity will be measure and affected component will be addressed. Pin Deck Tracking system shall develop a risk management plan for each risk. 4.2 Risk Table ID Risk Impact Severity Affected Components R1 Not able to find ball in image R2 Image loss during capture and transfer R3 Noise in images R4 Network connection loss R5 Computer Malfunctions and stops working R6 Light setting of the calculation can be done without location of ball. Not enough image to process which will lower the accuracy Difficult to find object in image The network goes down while images are being transferred from camera to disk Computer freezes while processing image Image too bright or too dark to process. Medium Medium Medium Image Processing Layer Image Acquisition Layer Image Acquisition Layer Image Acquisition Layer Management Plan Use different algorithm s to find ball in image Run multiple test at different Frame rates to capture images Use different light setting on pin deck and apply filters to reduce noise in image Have a reliable network connection and use Ethernet cable with enough data transfer rate low All layer Reboot the computer Medium Table 4-1: Risk Table Image processing layer Place light in different position to minimize the shadow or glare October 1 st, 2013 P a g e 32 Team Kingpin

33 5. Testable Features 5.1 Overview This section includes the list of customer requirement in SRS. These features can be tested and verified by the users as well as by team King Pin. Features listed in this section are to be thoroughly tested to ensure all the requirements are satisfied. Following are the levels of risks associated with each test : Medium: Low: Feature may be difficult to test Has been tested and may not work as expected Will be implemented and work properly 5.2 Customer Requirements Entry Board Risk Medium Description This test will verify whether the board number the ball is on just before the ball strikes the first pin is accurate to half the width of a single board Test Approach This test shall manually verify if entry board number produced by Pin Deck tracking system matches the calculated Exit Board Risk Medium October 1 st, 2013 P a g e 33 Team Kingpin

34 Description This test will verify whether the board number the ball is on just before the ball leaves the pin deck is accurate to half the width of a single board Test Approach This test shall manually verify if exit board number produced by Pin Deck tracking system matches the calculated Entry Angle Risk Description This test shall verify the entry angle of the ball just before the ball strikes the first pin with accuracy of angle within degree Test Approach This test shall manually verify entry angle with calculated entry angle Exit Angle Risk Description This test shall verify the exit angle of the ball just before the ball just before the ball falls off the pin deck with accuracy of angle within degree Test Approach This test shall manually verify exit angle with calculated exit angle Ball Speed Risk October 1 st, 2013 P a g e 34 Team Kingpin

35 Description This test shall verify whether the speed of the bowling ball at a given position on the pin deck is within accuracy of 0.01 mph Test Approach This test will manually calculate the average speed and verify it correlates with the systems calculated value Ball Path Risk Description This test shall verify whether the position of the ball for every frame located on the pin deck is within accuracy of ¼ of an inch Test Approach This test shall manually verify position for each point in the ball path correlates with calculated position Pin Movement Risk Description This test shall verify the position of each pin while the ball is still on the pin deck represented by a pair of (x,y,z) coordinates Test Approach This test shall manually verify the position of each pin with calculated position of each pin while the ball is still on the pin deck Camera Calibration Risk October 1 st, 2013 P a g e 35 Team Kingpin

36 Description This test shall verify whether the camera is properly calibrated or not Test Approach This test shall be manually verified the matrices with a known camera position Data Output Risk Low Description This test shall verify whether the output data from system is accurately written in a comma separated value file and saved is specified location in disk Test Approach Output file shall be located in specified folder in disk and data in comma separated value shall be testified against the expected result. 5.3 Packaging Requirements System Hardware Assembly Risk Low Description This test shall verify whether all the hardware components like camera, Ethernet cable, system computer are assembled as required by the pin deck tracking system Test Approach The System computer shall be powered and we shall run the camera calibration test program to make sure camera captures specified number of frame and saves to specified location of system computer Software Delivery Risk Low October 1 st, 2013 P a g e 36 Team Kingpin

37 Description This test shall verify whether the system software is properly transferred to a memory stick with required library files, source file, and APIs Test Approach Memory stick with Pin deck tracking system software shall be inserted in three other computers with different configurations and we shall check manually whether all required source file, library files and APIs has been copied or not Software Installation Risk Low Description This test shall verify whether the system software is properly installs in the system computer or not Test Approach Memory stick with Pin deck tracking system software shall be inserted in system computer and install the software. Team leader shall verify whether software is properly installed or not by running the dummy test programs. Then software is uninstalled. This process is repeated 3 times to make sure software installs without any errors User Manual Risk Low Description This test shall verify whether the user manual is copied in memory stick or not Test Approach Memory stick with Pin deck tracking system software shall be inserted in system computer Team leader shall verify manually whether User Manual is copied or not. October 1 st, 2013 P a g e 37 Team Kingpin

38 5.4 Performance Requirements Calibration Performance Risk Low Description This test shall verify whether the system s calibration module shall complete in an acceptable time frame Test Approach The software shall be run in system computer. Calibration button and stop watch shall be pressed together and time taken to perform calibration shall be recorded. Above process shall be run five times and average time is calculated. Then we will check if average time for calibration falls under acceptable time frame Data Analysis Performance Risk Low Description This test shall verify whether the systems data analysis module shall process images in an acceptable time frame Test Approach The software shall be run in the system computer. Start Capture button and stop watch shall be pressed together and time taken to perform Data Analysis shall be recorded. Above process shall be run five times and average time is calculated. Then we will check if average time for Data Analysis falls under acceptable time frame October 1 st, 2013 P a g e 38 Team Kingpin

39 6. Non-Testable Features 6.1 Overview The following features listed are not to be tested since they are verified by system design. These features describe the system properties of the product and do not much functionality. Some features are tested and verified by USBC. 6.2 Safety Requirements Camera Mounting The camera shall be mounted away from moving machine parts and in such a way that no harm or damage is caused to the camera by surrounding machinery, pin movement or ball movement Lighting Mounting Lighting shall be mounted so that it will not obstruct the functionality of any other device or machinery in the pin deck area creating opportunity for hardware damage Cabling / wiring Cables and wires must be laid in a way to prevent the obstruction of the bowling lane, pin deck and any area trafficked by the user User Safety The system shall be built in a way to prevent the need of the user to interact with the system during the capture and analysis process to prevent harm to the user from moving machinery. 6.3 Maintenance and Support Requirements Source Code Team Kingpin s source code shall be well documented with comments and details to allow future teams to modify functionality, troubleshooting procedures, maintenance procedures, or upgrade the system. October 1 st, 2013 P a g e 39 Team Kingpin