Team: Overdrive. Department of Computer Science and Engineering The University of Texas at Arlington. System Test Plan

Transcription

1 Department of Computer Science and Engineering The University of Texas at Arlington Team: Overdrive System Test Plan Project: Auto Performance Analyzer Team Members: Gary Johns Raul Hurtado Brendon Harris Zang Pham Last Updated: Friday, October 18, :12:06 PM

2 Table of Contents Table of Contents... ii Document Revision History... v List of Figures... vi List of Tables... vii 1. Introduction Product Overview Purpose and Use Project Scope References Overview System Requirements Specification Architecture Design Specification Detailed Design Specification Test Items Overview Relational Diagram Hardware Tests Unit Tests Subsystem Tests Layer Integration Tests System Validation Tests Risks Overview Risk Table /04/2013 page ii Overdrive

3 5. Features To Be Tested Packaging Requirements Performance Requirements Maintenance and Support Requirements Other Requirements Features Not To Be Tested Customer Requirements Packaging Requirements Safety Requirements Maintenance and Support Requirements Other requirements Overall Test Strategy Overview Configurations... Error! Bookmark not defined Strategy Acceptance Criteria Overview Hardware Testing Unit Testing Component Testing Integration Testing System Verification Testing Suspension Criteria and Resumption Requirements Overview Suspension Criteria Resumption Criteria /04/2013 page iii Overdrive

4 10. Test Deliverables Overview Deliverables Test Schedule Overview Test Schedule Approvals /04/2013 page iv Overdrive

5 Document Revision History Revision Number Revision Date Description /11/2013 Rough Draft /18/2013 Baseline /18/2013 Added System Verification Pass/Fail Criteria cases Rationale 10/04/2013 page v Overdrive

6 List of Figures Figure 2-1 Architectural Diagram of System... 7 Figure 2-2 Detailed Design diagram of system...11 Figure 3-1 System Relational Diagram /04/2013 page vi Overdrive

7 List of Tables Table 2-1 Customer Requirements... 3 Table 2-2 Packaging Requirements... 4 Table 2-3 Performance Requirements... 4 Table 2-4 Safety Requirements... 5 Table 2-5 Maintenance and Support Requirements... 6 Table 2-6 Other Requirements... 6 Table 2-7 Subsystem Data Flow Definitions... 9 Table 2-8 Requirements Mapping...10 Table 2-9 Module Component Data Flow Definitions...12 Table 2-10 Module Requirements Traceability matrix...13 Table 3-1 Hardware Components Tests...16 Table 3-2 Unit Tests for GUI layer...17 Table 3-3 Unit Tests for Event Manager layer...17 Table 3-4 Unit Tests for Processing layer...18 Table 3-5 Unit Tests for DBMS layer...18 Table 3-6 Unit Tests for Bluetooth layer...18 Table 3-7 Component/Subsystem Tests for GUI layer...19 Table 3-8 Component/Subsystem Tests for Event Manager layer...19 Table 3-9 Component/Subsystem Tests for Processing layer...19 Table 3-10 Component/Subsystem Tests for DBMS layer...20 Table 3-11 Component/Subsystem Tests for Bluetooth layer...20 Table 3-13 Layer Integration Tests for GUI layer...20 Table 3-14 Layer Integration Tests for Event Manager layer...21 Table 3-15 Layer Integration Tests for Processing layer /04/2013 page vii Overdrive

8 Table 3-16 Layer Integration Tests for DBMS layer...21 Table 3-17 Layer Integration Tests for Bluetooth layer...22 Table 3-18 System Validation Tests...22 Table 4-1 Risks...23 Table 5-1 Features to be Tested...29 Table 6-1 Features Not to be Tested Table 8-1 Pass/Fail Criteria for Hardware Components...34 Table 8 2 Hardware Testing Pass/Fail Criteria...35 Table 8-3 Unit Testing Pass/Fail Criteria...35 Table 8-4 Component Testing Pass/Fail Criteria...36 Table 8-5 Integration Testing Pass/Fail Criteria...37 Table 8-6 System Verification Pass/Fail Criteria...37 Table 9-1 Suspension Criteria...38 Table 9-2 Resumption Criteria...38 Table 11-1 Test Schedule...40 Table 12-1 Approvals /04/2013 page viii Overdrive

9 1. Introduction 1.1 Product Overview The Auto Performance Analyzer shall consist of one mobile application, running on a smartphone or tablet, for a pre-existing COTS (Commercial Off The Shelf) device that will plug into any car s (model 1996 or newer) On Board Diagnostics generation II connector ( OBD-II). The pre-existing module shall connect and communicate with the mobile application, only returning data that was requested, via Bluetooth. The COTS device is commercially designed to be small and can handle temperature extremes of -20 C to +75 C. The mobile application shall query the on-board computer through the COTS device, which will obtain the data from the car and will provide such data to the mobile application. This application will give the user the option to choose the parameters of interest from a list and will display the data in real time. The mobile application shall provide the user with the option to retrieve and clear trouble codes that are produced by the car when the check-engine light is on. The mobile application will retrieve the trouble codes data from the car and pull the information from a trouble code database stored in the device when the application is installed. The trouble code retrieval, interpretation and clear feature will be displayed in a separate screen independent of the parameter s display screen. 1.2 Purpose and Use The purpose of this System Test Plan (STP) document is to provide detailed testing procedures to ensure that the Auto Performance Analyzer (APA) meets all necessary requirements and will function the way it is expected to when delivered to our customers. The STP document will describe the testing procedures that must be performed on the modules, the components, the layers and the hardware. After these standalone pieces have been test, then integration testing will be performed and finally system verification testing. 1.3 Project Scope The scope of the Auto Performance Analyzer project is to create a diagnostics module that interfaces with the OBD-II connector system in automobiles to retrieve information from that automobile. The Auto Performance Analyzer will provide a mobile user interface with enhanced tools for data logging, vehicle diagnosis, and real time statistics. The interface will be available for Android devices, such as 10/04/2013 page 1 Overdrive

10 smart phones and tablets. The application will be able to display the cars parameters data as digital numeric readouts and graphs to the user. In order to populate the user interface with data, the module retrieves this data from the existing OBD-II connector, parses and reformats it. The module will then transmit it over the Bluetooth connection, to the mobile device, in real time. 10/04/2013 page 2 Overdrive

11 2. References 2.1 Overview The STP document will utilize elements from previously created documents: the System Requirements Specification, Architecture Design Specification, and Detailed Design Specification. Testing will be conducted with these elements to ensure that all parts of the system and all features are working correct. 2.2 System Requirements Specification Customer Requirements SRS Requirement Description Priority No. 3.1 Real-Time Data Acquisition Data must be retrieved and displayed in 1 Critical real time. 3.2 Performance Mode Select The mobile application will display 1 Critical Readings performance readings. 3.3 Performance Mode - Monitor The mobile application will have a feature 3 Moderate that will start and stop recording/monitoring the selected readings. 3.4 Troubleshoot Mode Vehicle The mobile application will require the 1 Critical Information user to input the vehicle's information such as the make, model, and year. 3.5 Troubleshoot Mode - Comparison The mobile application will display any 1 Critical trouble codes detected on screen. 3.6 Store Readings Data read will be stored in the internal 2 - High History database. 3.7 Database (Trouble Code) The mobile application will have a 1 Critical Trouble Code database that is stored on the phone. 3.8 Database (History) The mobile application will have a History 2 - High database that contains all past readings. 3.9 Graphs/Charts The mobile application will be able to display graphs/charts of past runs. 2 - High 3.10 User Friendly The mobile application must be user 2 - High friendly, in that it must be easy to navigate through, and have an easy to use GUI GPS Tracking Capability The application will be able to track the 4 Low phone and the vehicle's location Transmission of Data Report The Mobile application will have the option to have a report generated and send it to a user entered account. Table 2-1 Customer Requirements 3 Moderate 10/04/2013 page 3 Overdrive

12 2.2.2 Packaging Requirements SRS Requirement Description Priority 4.1 User Manual The system shall be packaged with a user manual cd. 4.2 Bluetooth OBDII Adapter The system will include an OBD-2 reader with Bluetooth. 3 Moderate 4.3 Android Application Store Submission APA will be available in the Android Market as a free download. 2- High 4.4 Bluetooth OBDII Adapter Installation The user manual will include instructions to install the adapter. Table 2-2 Packaging Requirements Performance Requirements SRS Requirement Description Priority 5.1 Real-Time output The system shall be able to read the data from the OBD-II connector and transmit such data to be displayed and saved on the mobile device. 5.2 Reliable Data transfer The system shall be successful in the transfer of the data packets over Bluetooth to be displayed and saved on the mobile device. 5.3 Android Devices compatibility The system shall be compatible with most Android mobile devices Table 2-3 Performance Requirements 10/04/2013 page 4 Overdrive

13 2.2.4 Safety Requirements SRS Requirement Description Priority 6.1 Exposed Circuitry The Bluetooth OBD-II Adapter should not have any exposed circuitry that may damage/injure the vehicle nor the user through electrical shock. 6.2 Sharp Edges The Bluetooth OBD-II Adapter should not have any sharp edges that may damage/injure the vehicle/user. 6.3 Overheat The Bluetooth OBD-II Adapter should not overheat to the point of being able to burn the user, even after long hours of usage. 3 Moderate 6.4 Installation During installation of the COTS unit, with the OBD-II interface connector inside your vehicle, the vehicles engine should be completely off. 6.5 Do Not Operate While Driving Using the phone while driving may lead to serious injuries and even death. 6.6 System Modification The application should not allow the modifications of critical system parameters Table 2-4 Safety Requirements 3 Moderate 1 Critical 1 Critical Maintenance and Support Requirements SRS Requirement Description Priority 7.1 Support Future Mobile Operating Systems The mobile application will be supported on future releases of other mobile 3 - Moderate operating systems. 7.2 Code Documentation Code shall be well documented with comments detailing information regarding intended use and functionality. 2 - High 7.3 Testing and Troubleshooting Documentation Testing and troubleshooting documentation shall include detailed unit and module test covering every aspect of the hardware and software units. 7.4 Code Enhancement Team Overdrive will continue to improve its program code to further increase speed, performance, and overall efficiency. 7.5 Maintenance Cutoff Date Project shall be maintained through 12/ Critical 2 - High 1 Critical 10/04/2013 page 5 Overdrive

14 Table 2-5 Maintenance and Support Requirements Other Requirements SRS Requirement Description Priority 8.1 Compatibility with most car The system shall be able to read the data 2 - High manufacturers from the OBD-II connector from most of the vehicle manufacturer s model 1996 or newer. 8.2 Statistics Database The system will store the data gathered 2 - High from the OBD-II in a log file for later use. 8.3 Trouble Codes Database The mobile application will contain a database with the Diagnostic Trouble Code 3 - Moderate Information to provide the user with the description of the Trouble Code 8.4 User Accounts The mobile application will provide the 4 - Low user with the capability of creating different vehicle profiles with the purpose to store data for several types of vehicles 8.5 Accurate Data Display Any parameter displayed in the mobile 2 - High application will be as accurate as the ones displayed on the vehicle dashboard if available. 8.6 On-line connection security The mobile application will comply with the user security settings when transferring the data file through Low Table 2-6 Other Requirements 2.3 Architecture Design Specification Layer Definition This section describes Auto Performance Analyzer s application (APA) system architecture, the overall structure of how the system will be built. The architecture design is shown in Fig 3-1. APA s system architecture consists of five major layers: 10/04/2013 page 6 Overdrive

15 Figure 2-1 Architectural Diagram of System GUI Layer Definition The user will select the desired parameters on the mobile device using the provided GUI checklist in the application. Upon completing the parameter desired list, this layer will pass the list to the Event Manager Layer through the I/O controller subsystem. The Event Management layer through the processing handler; will pass the list to the processing layer, to convert such parameters to the corresponding OBD-II commands. The Processing layer will pass the converted command list to the 10/04/2013 page 7 Overdrive

16 Bluetooth OBD-II adapter connected to the car s OBD-II port. When the car replies with the actual parameter s information, the Bluetooth layer will send the cars data back to the Processing layer to be processed. This layer will receive the information from the Event Management layer adapter via the I/O controller subsystem, which then it will passed to the presentation subsystem that will display the info on the mobile device s screen Processing Layer Definition Upon receiving the parameters list from the Event Manger layer through the processing handler, the Processing layer will convert this list into a stream of commands that the car s On-board computer can understand, and will pass the converted list to the Bluetooth layer. Then it will get from the same Bluetooth layer the actual car s parameter data; which the Processing layer will convert and process to decimal format that it will be required either to be displayed on the mobile device s screen through the GUI event handler or stored on the database file trough the Data Base controller in the DBMS Layer Event Manager Layer Definition This layer will route the users parameter selection list from the GUI Layer, actual parameters information data from the Processing layer and route the data necessary back to the GUI layer to be displayed in the mobile devices screen or to be passed to the Storage Management layer to be saved to the spreadsheet file Database Management System Layer Definition After receiving the data from the Event Manager layer, this layer will be responsible for the formatting of the data (if necessary) in a table form (spreadsheet) that will contain the time, parameters reading, and the mobile device sensors reading (such as GPS, Accelerometers, etc. if they are available on the mobile device). This layer will also be responsible for managing and retrieval of the Trouble Code Description database information. The information from this layer will be sent back to the Event Manager Layer, through the DBMS Event Handler, to be passed subsequently to the GUI layer if needed to display it Bluetooth Layer Definition The user s parameter selection list will be received already converted to the right format from the Processing Layer. This list of commands will be transmitted via Bluetooth to the STN1170 OBD-II Adapter. When the car s On-board computer responds with the car s actual parameter data, the OBD-II Adapter will send the data to back to the application, which will be received by the Bluetooth Layer and passed back to the Processing Layer for processing Data Flow Definitions The table below provides a description of the data that flows between each subsystem shown in figure 2-1 (located on next page). Data originating from the GUI Layer will begin with a G, Event Manager Layer will begin with E, Processing Layer will begin with P, DBMS Layer will begin with D, and Bluetooth Layer will begin with B. 10/04/2013 page 8 Overdrive

17 Layer Data Element Description GUI Layer Event Management Layer Processing Layer DBMS Layer Bluetooth Layer G1 G2 G3 G4 G5 E1 E2 E3 E4 E5 P1 D1 D2 B1 B2 B3 B4 B5 User request sent to GUI by phone/tablet. Create Java Class Object and pass to I/O Controller Flag Variable > 0, pass to Event Manager Update Presentation event sent from I/O Controller to Presentation. Send updated GUI Presentation to Android Device. Receive user input from GUI Event Controller. Pass Java Object for Store/Fetch data Pass Hex code parameters to Bluetooth for fetch Returned updated Java Class Object with data retrieved from DB Send/Retrieve Data. Object with converted DTC code or parameters Query Historical Data Query Error Codes Data Send Bluetooth data. Pass Bluetooth values. Synchronize Transfers Bluetooth data. Return Bluetooth Data Table 2-7 Subsystem Data Flow Definitions Requirement Mapping This section describes the relationships between the requirements outlined in the SRS document and the architecture defined in this document. It will help our group to ensure that the requirements are covered by the architecture. Table 10-1 shows the relationships between each layer and each requirement. An X means that the requirement is covered by the layer with an X. 10/04/2013 page 9 Overdrive

18 # Name GUI Layer Event Manager Layer 3.1 Real-Time Data Acquisition X X X 3.2 Performance Mode - Select Readings X X 3.3 Performance Mode - Monitor X X X 3.4 Troubleshoot Mode - Vehicle Information X X X 3.5 Troubleshoot Mode - Comparison X X X 3.6 Store Readings X X 3.7 Database (Trouble Codes) X 3.8 Database (History) X 5.1 Real-Time Output X X X 5.2 Reliable Data Transfer X X X X Table 2-8 Requirements Mapping Processing Layer DBMS Layer Bluetooth Layer 2.4 Detailed Design Specification Prologue The purpose of this document, Detailed Design Specification or DDS, is to extend the Architecture Design Specification Document (ADS) and presented in detail how the Auto Performance Analyzer will be implemented. The DDS takes the layers and subsystems from the ADS and breaks them down even more into individual modules inside the subsystems. Included is a graphical display of the systems architecture along with a layer by layer detailed description. Each of these modules will handle specific tasks and will describe how modules handle those tasks and interface with other modules. The DDS will also describe internal data, dependencies, and Pseudo Code that goes along with these modules. The DDS document provides the Auto Performance Analyzers operating system dependencies with the Android operating system along with the many testing considerations that must be implemented. The DDS will lead to software implementation. 10/04/2013 page 10 Overdrive

19 Figure 2-2 Detailed Design diagram of system Data Flow Definitions Table 12-1 (Shown below) provides a description of the data that flows between each module. Data originating from the GUI Layer will begin with a G, Event Manager Layer will begin with E, Processing Layer will begin with P, DBMS Layer will begin with D, and Bluetooth Layer will begin with B. 10/04/2013 page 11 Overdrive

20 Layer Data Element Description G1 Button Pressed Event Management Layer GUI Layer G2 Send Button event G3 Pass object to I/O Controller G4 User interface and run object G5 Send updated GUI Presentation to Android Device. G6 Object E1 Object with parameter list (hex code) E2 Object with converted hex DTC (P301) & flag E3 Object with converted hex DTC (P301) with fetch instruction E4 Object with flag E5 Object with fetch/store instructions E6 Packaged object E7 Return packed object I/O Controller Processing Layer P1 Object with converted DTC code or parameters DBMS Layer Bluetooth Layer D1 D2 D3 D4 B1 B2 B3 B4 B5 Return fetched object with past runs Return fetched Object with DTC (code + description) Read, compare, fetch past runs or store info Read, compare, fetch DTC (code + description) Hex code to be fetched Connect to bluetooth adapter, send hex code, synchronize Returns Hex code, maintains connection. Returns Hex Code. Pass hex code to be converted. Table 2-9 Module Component Data Flow Definitions 10/04/2013 page 12 Overdrive

21 2.4.3 Module Requirements Traceability 3.1 Real-Time Data Acquisition 3.2 Performance Mode - Select Readings 3.3 Performance Mode - Monitor 3.4 Troubleshoot Mode - Vehicle Information Button Listener X X Package X X Route X X X X Display X X X Route X X X X X X X Package X X X X X X Get X X X X X X Runs/Store/ Get DTC list X X X Query History X X X X X Query Error Code X X X X Conversion X X X X Run/Fetch X X X Write/Return X X X Cancel X X 3.5 Troubleshoot Mode - Comparison 3.6 Store Readings 3.7 Database (Error Code) Table 2-10 Module Requirements Traceability matrix 3.8 Database (History) 5.1 Real-Time Output 5.2 Reliable Data Transfer 10/04/2013 page 13 Overdrive

22 3. Test Items 3.1 Overview This section provides details on the different phases of testing necessary to generate a successful and complete product. The purpose of the relational diagram, depicted below (Figure 3-1), is to give a visual representation of each items that will be tested in the form of a flowchart. The System Test Plan (STP) is divided into 5 testing phases: hardware, unit, component, integration, and system verification. For clarification, units represent the modules, components represent the subsystems, and integration represents the layers. Each path of the Relational Diagram can be tested concurrently, but must be traversed in order from left to right. Each phase on the path must be successfully completed before moving on to the next phase. The specific tests are described in the following subsections. They describe the inputs, expected outputs/actions, and risk level. 10/04/2013 page 14 Overdrive

23 3.2 Relational Diagram Figure 3-1 System Relational Diagram 10/04/2013 page 15 Overdrive

24 3.3 Hardware Tests Test ID H1 Hardware Component Bluetooth Adapter Input Expected Output/Action Test Risk User Connects Bluetooth Adapter to car s OBD-II port. Ignition key is off LEDs on the Bluetooth adapter will not lit Visual Inspection of the Bluetooth Adapter LEDs H2 H3 Bluetooth Adapter Bluetooth Adapter, Mobile Device User turns the ignition key to ON position. Bluetooth adapter plugged in User pairs the Bluetooth adapter to the mobile device LEDs on the on the Bluetooth adapter will blink and then the power LED will stay lit while the Bluetooth LED will remain blinking Power LED and Bluetooth LED will stay lit. Visual Inspection of the Bluetooth Adapter LEDs Visual Inspection of the Bluetooth Adapter LEDs H4 H5 H6 Bluetooth Adapter, Mobile Device Bluetooth Adapter, Mobile Device Bluetooth Adapter, Mobile Device User Starts the Auto Performance Application Mobile device requests data parameters User Exits the Application Bluetooth LED will not be interrupted and the application loads the initial screen POWER and Bluetooth LEDs will not be interrupted, OBD and HOST LEDs will blink alternatively The Application closes the Bluetooth connection before closing, Power LED remains lit, Bluetooth LED will start blinking, OBD and HOST LEDs will remain OFF Visual Inspection of the Bluetooth Adapter LEDs and the mobile device s screen Visual Inspection of the Bluetooth Adapter LEDs and the mobile device s screen Visual Inspection of the Bluetooth Adapter LEDs and the mobile device s screen Table 3-1 Hardware Components Tests 10/04/2013 page 16 Overdrive

25 3.4 Unit Tests GUI Layer Unit Test Test ID Unit Input Expected Output Test Risk UG1 Button User touches button Appropriate screen Manually activate each button and visually 1-Critical Listener on screen of phone is displayed inspect UG2 Package Selected text values Object created with Write function to pass test values to 1-Critical correct values module and print contents of object Route Flag with 0 or 1 If 0 pass to display, Write function to pass test values to 1-Critical UG3 if 1 pass to Event Handler system module and make sure it handles as expected Display Class with visual 1-Critical representations UG4 Screen on phone showing correct information displayed Table 3-2 Unit Tests for GUI layer Manually enter test data into appropriate fields and test each class to see if correctly displayed Event Management Layer Unit Test Test ID UEM1 UEM2 UEM3 UEM4 Unit Input Expected Output/Actions Test Risk Route Java Class Object Java Class Object with parameter list Java Class Object with converted hex code (decimal) Updated Java Class Object with retrieved data from database Package Java Class Object Updated Java Class Object with requested data with requested data from from database database. Get Runs/ Store/ Get DTC List Java Class Object with History Data request Java Class Object with DTC data request Separates Java Class Object with Fetch/Store instructions Separates Java Class Object with converted hex code and fetch instructions Table 3-3 Unit Tests for Event Manager layer Pass in a Java Object, with various flags, into the function (switch/case statement). Use Print statements to determine where Object is sent to. Pass in a Java Object with fake data to see if the package function will assign the data into the corresponding variables via Print Statements(i.e. Toyota, Camry, 2007, should be stored into the variables Make, Model, Year, respectively). Pass in a Java Object with fetch/store instructions and determine if the Object has been successfully separated via Print Statements. Pass in a Java Object with fetch instructions and determine if the Object has been successfully separated via Print Statements. 10/04/2013 page 17 Overdrive

26 3.4.3 Processing Layer Unit Test Test ID UP1 Unit Input Output Test Risk Data Conversion Hex Number Integer Pass Hex numbers that correspond to the parameters to be monitored to the Data Conversion Unit. This Hex codes will be converted to Integer numbers Table 3-4 Unit Tests for Processing layer DBMS Layer Unit Test Test ID UDB1 UDB2 Query History Unit Input Expected Output/Actions Test Risk Query Error Code Formatted SQL Statement Formatted SQL Statement Java Class Object with retrieved data from database Java Class Object with requested data from database. Table 3-5 Unit Tests for DBMS layer Simulate multiple SQL query statements and use them to communicate with the database. Then check the data retrieved against assumed results. Simulate multiple SQL query statements and use them to communicate with the database. Then check the data retrieved against assumed results Bluetooth Layer Unit Test Test ID UB1 UB2 UB3 UB4 Unit Input Output Sync Hex Code Parameter Hex Data, Bluetooth Connection not interrupted Cancel Run/Fetch Write/Return Android Bluetooth Stop Instruction Parameter Hex Code Parameter Hex Data Bluetooth Connection closed Parameter Hex Data Parameter Hex Data Table 3-6 Unit Tests for Bluetooth layer Test Hex codes will be transmitted via Bluetooth to the Bluetooth Adapter, the Bluetooth connection should not be interrupted The mobile application will be closed and the Bluetooth connection will be released Hex codes will be transmitted via Bluetooth to the Bluetooth Adapter. The vehicle should respond to the actual parameter reading in Hex numbers The Write/Return Unit will pass the actual parameter reading to the Processing Layer Risk 3 - Medium 10/04/2013 page 18 Overdrive

27 3.5 Subsystem Tests GUI Layer Subsystem Tests Test ID SG1 SG2 Subsystem Input Expected Output Test Risk Pretested IDs User 1-Critical UG1, Input UG2 I/O Controller User touches button on screen of phone Selected text values SG3 Presentation Class with visual representations Appropriate screen displays and object created with correct values Object created with correct values Screen on phone showing correct information displayed Manual activate each button, then visually inspect and print contents of object Write function to pass test values to module and print contents of object Manual enter test data into appropriate fields and test each class to see if correctly displayed Table 3-7 Component/Subsystem Tests for GUI layer 1-Critical 1-Critical UG3 UG Event Manager Layer Subsystem Tests Test ID SEM1 SEM2 Subsystem Input Expected Output/Actions Test Risk Pretested IDs GUI Event Java Class Send Java Class Object to Pass in a Java Object, with UEM1, Handler Object other modules depending on various flags, into the UEM2, flag variable function (switch/case UEM3 statement). Use Print statements to determine where Object is sent to, print the flag variable, and what is being passed. DBMS Java Class Separates class object to Pass in a Java Class, and use UEM4, Event Object store data or send requests parsing function to separate UEM5, Handler to database data that can be used to fetch UEM6 (compare)/store data into database. Use Print statements and observe data manipulation. Table 3-8 Component/Subsystem Tests for Event Manager layer Processing Layers Subsystem Tests Test ID Subsystem Input Output Test Risk Pretested IDs SP1 Data Conversion Hex code Integer value Hard-code in a random Hex values into conversion function and Print out results of the conversion in Integer values. Verify result with calculator (Windows or online) UP1 Table 3-9 Component/Subsystem Tests for Processing layer 10/04/2013 page 19 Overdrive

28 3.5.4 DBMS Subsystem Tests Test ID SDB1 Subsystem Input Expected Output/Actions Test Risk Pretested IDs DB Formatted SQL UDB1, Controller Statement with UDB2 database flag. Correctly query data from the correct database and that it can insert into the history but not the error code Simulate SQL query statements with a random DB flag and observe the changes to the database and data returned Table 3-10 Component/Subsystem Tests for DBMS layer Bluetooth Layers Subsystem Tests Test ID Subsystem Input Output Test Risk Pretested IDs SB1 Data Acquisition Data parameter Hex code SB2 Sync/Pair Data parameter Hex code Data parameter Hex code sent to Sync/Pair subsystem Data parameter Hex code Built-in function of Bluetooth. Check Write/Return module to verify if correct data is fetched Built-in function of Bluetooth. Check Write/Return module to verify if correct data is fetched Table 3-11 Component/Subsystem Tests for Bluetooth layer UB1, UB3, UB4 UB2 3.6 Layer Integration Tests GUI Layer Integration Tests Test ID IG1 IG2 Subsystem Input Expected Output Test Risk Pretested ID s User 1-Critical SG1, SG2 Input and I/O Controller User Input, I/O Controller, Presentation User touches button on screen of phone User touches button on screen of phone Appropriate screen pops up and object created with correct values: that either displays on screen or goes to Event Management Layer Appropriate screen pops up with correct values and object goes to Event Management Layer Manual activate each button, then visually inspect screen, add print statement to Route module to print what flag was pass and contents of object Write function add test values to created object and visually inspect screen output and print contents of object passed to Event Management Layer Table 3-12 Layer Integration Tests for GUI layer 1-Critical SG1,SG2, SG3 10/04/2013 page 20 Overdrive

29 3.6.2 Event Manager Layer Integration Tests Test ID IEM1 IEM2 IEM3 Subsystem GUI Event Handler, DBMS Event Handler GUI Event Handler, DBMS Event Handler GUI Event Handler, DBMS Event Handler Input Java Class Object (with parameters) Java Class Object (with converted Hex Code) Java Class Object (with requested data from database) Expected Output/Actions Pass Object to Bluetooth layer for data acquisition. Pass Object to DBMS layer to be stored into database. Pass Object to GUI layer for output Test Pass in an Object and check flag variable. Determine where the Object is sent to by using Print statements just before and after sending the object. Observe Print statements Pass in an Object and check flag variable. Determine where the Object is sent to by using Print statements just before and after sending the object. Observe Print statements Pass in an Object and check flag variable. Determine where the Object is sent to by using Print statements just before and after sending the object. Observe Print statements Table 3-13 Layer Integration Tests for Event Manager layer Risk 1-Critical 1-Critical 1-Critical Pretested IDs SEM1, SEM2 SEM1, SEM2 SEM1, SEM Processing Layer Integration Tests Test ID IP1 Subsystem GUI Event Handler, Bluetooth Data Acquisition Input Hex data from Bluetooth Write/Return Expected Output/Actions Converted Hex data to Integer that will be passed to the GUI Event Handler Test Verify data received from Bluetooth adapter using Print statements Table 3-14 Layer Integration Tests for Processing layer Risk Pretested IDs 1 Critical SP DBMS Layer Integration Tests Test ID IDB1 Subsystem DB Controller Input Formatted SQL Statement with database flag. Expected Output/Actions Correctly query data from the correct database and that it can insert into the history but not the error code Test Receive SQL statements from the event management layer and return correctly queried data to be packaged and routed in the event management layer Table 3-15 Layer Integration Tests for DBMS layer Risk Pretested IDs 1 Critical SDB1 10/04/2013 page 21 Overdrive

30 3.6.5 Bluetooth Layer Integration Tests Test ID IB1 IB2 Subsystem Input Expected Output/Actions GUI Event Parameter Hex Hex code passed to Handler, data from GUI OBDLink MX Bluetooth Data Event Handler adapter Acquisition, layer Sync/Pair GUI Event Handler, Data Acquisition, Sync/Pair, Data Conversion Hex code from Bluetooth adapter Retrieved Hex code from OBDLink MX Bluetooth adapter that is passed to the Processing layer for conversion. Table 3-16 Layer Integration Tests for Bluetooth layer Test Risk Pretested IDs Pass in a Hex code, and use 1 Critical SB1, SB2 Print statement to verify if the Hex code has been received. Built-in Bluetooth function will communicate with adapter Built-in Bluetooth function. 1 Critical SB1, SB2 Verify data received from Bluetooth adapter using Print statements 3.7 System Validation Tests Test ID Input Expected Output/Action SV1 User Selection of Display Vehicle Vehicle Year, Year, Make, Model Make, Model SV2 SV3 SV4 User Selects desired parameters User selects record button User selects file to view Display Parameters Information Information saved to a file Information from file displayed in the screen Test Use the mobile device settings to verify that a new file has been created Use the Scantool software to verify the information matches what is displayed on the screen Open the file with MS Excel Check with MS Excel to verify that the graphic displayed corresponds to the data in the History file Risk 3 - Medium SV5 User selects DTC button Error codes and description displayed on screen Use the ECU simulator. Error codes should be the same SV6 SV7 SV8 User selects clear Error codes button User selects button User selects Exit/Close application Error codes cleared. Check Engine Light off File received in destination inbox Bluetooth connection unpaired and application closed Visually verify the Check Engine Light is OFF User selects and enters the destination . Verify the file has been received Close APA app and visually inspect the Android device s Bluetooth Settings and check whether the connection to the OBDLink MX is unpaired/disconnected. Table 3-17 System Validation Tests 10/04/2013 page 22 Overdrive

31 4. Risks 4.1 Overview The following section will specify potential risks that may develop as a result of testing the Supply Fetch system. Each risk can have a specific impact on the system at a level of high, medium or low, depending on the severity. The strategy for each risk will describe a possible approach for eliminating or reducing the risk. 4.2 Risk Table ID Risk Impact Severity Strategy 1 Fixing bugs creates new bugs Creates delays for end product High Find root cause for original bug if possible 2 Damaging hardware Prongs on the ODB II Module components could potentially be broken Medium Use with caution and precision 3 Logic errors may be missed System may not function Identify all possible scenarios users (condition paths, unusual correctly or handle the error Low may be exposed to during run time scenarios) appropriately 4 5 Unpredictable behavior on software or hardware units Issues from user testing system prototype Creates delays for end product May question critical features or usability if not intuitive to the user Table 4-1 Risks Low Medium Test at the lowest level as much as possible prior to integration and progressing to next test level Provide a prototype to customer before final deadline 10/04/2013 page 23 Overdrive

32 5. Features To Be Tested 5.1 Overview The following features to be tested are derived from the requirements laid out in the Software Requirements Specification Document. These requirements will be tested to ensure that the AutoPerformance Mobile application satisfies functional and nonfunctional specifications. The risk included define the priority the requirement has in the SRS Document 5.2 Customer Requirements Real-Time Data Acquisition Description: The Bluetooth OBD-II Adapter will read the data from the vehicle's OBD-II connector, which it will then send to an mobile device through Bluetooth in quick successions (real-time if possible). Risk: Very High Test Approach: Communication between the Bluetooth Adapter and the Mobile Application will be measured to ensure the maximum data transfer rate Performance Mode Select Readings Description: The mobile application will display performance readings such as speed, engine RPM, mileage, etc... as well as sensor readings such as oil, tire pressure, temperature, etc... (if applicable). The mobile application will have a list of check-boxes that that contains these readings that the user can select to have monitored. Risk: Very High Test Approach: Tests will be designed to ensure the GUI Layer will pass the correct user selected parameters to monitor Performance Mode - Monitor Description: The mobile application will have a START and STOP feature that will start recording/monitoring the selected readings and stop recording/monitoring the selected readings. Risk: Medium Test Approach: There will be test cases for each of the user buttons on the button listener subsystem Troubleshoot Mode Vehicle Information 10/04/2013 page 24 Overdrive

33 Description: The mobile application will require the user to input the vehicle's information such as the make, model, and year. This is to determine which set of Diagnostic Trouble Codes to be used and compared with (Some trouble codes are manufacturer specific). Risk: Very High Test Approach: Test cases that will reflect the user s input Troubleshoot Mode - Comparison Description: The mobile application will display any trouble codes detected, and display them on screen. The Application will then compare the trouble code read to the internal Diagnostic Trouble Code (DTC) database. Once a match has been found, a description will then be displayed along with its DTC. If no comparison could be found, a Trouble Code not found message will be displayed. Risk: Very High Test Approach: Test cases that will take the Diagnostic Trouble Code and will query the Database to retrieve the description for that particular Trouble Code Store Readings Description: Data read will be stored in the internal History database. This will occur automatically when Performance Mode is running, as well as when a report (or data has been saved/sent) has been generated in the Troubleshoot Mode. Risk: High Test Approach: Test Cases made within SQL Lite to store and read data from the History database, this data will be compared to the information read and displayed on the mobile application to make sure it will store and read the right data Database (Trouble Code) Description: The mobile application will have a Trouble Code database that is stored on the phone (comes with the application). This database will contain all generic Trouble Codes (for the time being) that will be used for comparison in Troubleshoot Mode. Risk: Very High Test Approach: Test Cases performed with the SQL Lite to retrieve information from the Trouble Codes database to make sure that the mobile application will retrieve the right information Database (History) Description: The mobile application will have a History database that is stored on the phone. This database will contain all past readings on a particular vehicle. Risk: High Test Approach: Test Cases executed with SQL Lite testing utility that will read the History database and will make sure the mobile application will retrieve such information correctly. 10/04/2013 page 25 Overdrive

34 5.2.9 Graphs/Charts Description: The mobile application will be able to display graphs/charts of past readings. Risk: High Test Approach: There will be a test case using MS Excel that will ensure that the graphs displayed will show the information correctly User Friendly Description: The mobile application must be user friendly, in that it must be easy to navigate through, and have an easy to use GUI. Risk: Moderate Test Approach: By survey, with the help of various people (including TA and the Sponsor) and taking their opinion on the user friendliness of the mobile application Transmission of Data Report Description: The Mobile application will have the option to have a report generated from the current or past readings and send it to a user entered account. Risk: Moderate Test Approach: The report will be ed to different addresses to verify that the report reaches the target destination. 5.3 Packaging Requirements User Manual Description: The system shall be packaged with a user manual explaining how to use AutoPerformance Analyzer. Risk: Low Test Approach: Have various user to proof read the manual Bluetooth OBDII Adapter Description: The system will include the Bluetooth Adapter. Risk: Very High Test Approach: This Bluetooth Adapter can be tested with a set of Hardware Tests. 10/04/2013 page 26 Overdrive

35 5.4 Performance Requirements Real-Time output Description: The system shall be able to read the data from the OBD-II connector and transmit such data to be displayed and saved on the mobile device. Risk: High Test Approach: Communication between the Bluetooth Adapter and the Mobile Application will be measured to ensure the maximum data transfer rate Reliable Data transfer Description: The system shall be successful in the transfer of the data packets over Bluetooth to be displayed and saved on the mobile device. Risk: High Test Approach: Test cases that will test different transfer rates to measure the Communication between the Bluetooth Adapter and the Mobile Application will be measured to ensure the minimum data loss Android Devices compatibility Description: The system shall be compatible with most Android mobile devices. Risk: Moderate Test Approach: The Mobile application will be installed on different Android devices (phones and tablets) to ensure that it won t be compatibility issues. 5.6 Maintenance and Support Requirements Testing and Troubleshooting Documentation Description: Testing and troubleshooting documentation shall include detailed unit and module test covering every aspect of the hardware and software units. Risk: High Test Approach: By consensus with different users the team will verify the completeness of the documentation. 5.7 Other Requirements Compatibility with most car manufacturers 10/04/2013 page 27 Overdrive

36 Description: The system shall be able to read the data from the OBD-II connector from most of the vehicle manufacturers. Risk: High Test Approach: The mobile application will be tested with several vehicles of different manufacturers Statistics Database Description: The system will store the data gathered from the OBD-II in a log file for later use. The user will be able to transmit this file by and use a desktop computer to analyze and gather statistics from the data. Risk: High Test Approach: The report will be ed to different addresses to verify that the report reaches the target destination Trouble Codes Database Description: The mobile application will contain a database with the Diagnostic Trouble Code Information to provide the user with the description of the Trouble Code. Risk: High Test Approach: Test Cases performed with the SQL Lite to retrieve information from the Trouble Codes database to make sure that the mobile application will retrieve the right information User Accounts Description: The mobile application will provide the user with the capability of creating different vehicle profiles with the purpose to store data for several types of vehicles. Risk: Low Test Approach: Verification by file creation inspection Accurate Data Display Description: Any parameter displayed in the mobile application will be as accurate as the ones displayed on the vehicle dashboard if available. Risk: High Test Approach: Comparison between the reading displayed in the mobile device s screen and the vehicle s gauges, both readings should match and there will not be more that 5% difference. 10/04/2013 page 28 Overdrive

37 SRS ID Requirement Type Priority 3.1 Real-Time Data Acquisition Customer Critical 3.2 Performance Mode Select Readings Customer Critical 3.3 Performance Mode Monitor Customer Moderate 3.4 Troubleshoot Mode Vehicle Information Customer Critical 3.5 Troubleshoot Mode Comparison Customer Critical 3.6 Store Readings Customer High 3.7 Database (Trouble Code) Customer Critical 3.8 Database (History) Customer High 3.9 Graphs/Charts Customer High 3.10 User Friendly Customer Low Transmission of Data Report Customer Moderate 4.1 User Manual Packaging Critical 4.2 Bluetooth OBDII Adapter Packaging Critical 5.1 Real-Time Output Performance High 5.2 Reliable Data Transfer Performance High 5.3 Android Devices Compatibility Performance Critical 7.3 Testing and Troubleshooting Documentation Maintenance Very High and Support 8.1 Compatibility with most car manufacturers Other High 8.2 Statistics Database Other High 8.3 Trouble Codes Database Other Medium 8.4 User Accounts Other Low 8.5 Accurate Data Display Other High Table 5-1 Features to be Tested 10/04/2013 page 29 Overdrive

38 6. Features Not to Be Tested 6.1 Overview The following features will not be tested since some are verified by design. In essence, they describe properties of the product and hold minimum or no functionality at all. The GPS tracking capability due to the limited time might not be included in the design and will only be tested if there s time to be included into the design 6.2 Customer Requirements GPS Tracking Capability Description: The system will make use of the phone s GPS Tracking Capability (if available). The application will be able to track the mobile device and the vehicle's location. Risk: Moderate Test Approach: This feature was not originally a critical requirement and will be implemented and tested only if there s enough time. 6.3 Packaging Requirements Android Application Store Submission Description: APA will be available in the Android Market as a free download. Risk: Low Test Approach: Since the AutoPerformance Analyzer mobile application will become UTA property, the team is not sure if the application can be posted in the Android Market (Google Play). This requirement probably will not be tested. 6.4 Safety Requirements Exposed Circuitry Description: The Bluetooth OBD-II Adapter should not have any exposed circuitry that may damage/injure the vehicle/user through electrical shock. The unit should be covered/sealed to avoid this. Risk: High Test Approach: This requirement is already met by the manufacturer and there will not be additional tests. 10/04/2013 page 30 Overdrive

39 6.4.2 Sharp Edges Description: The STN1170 Bluetooth OBD-II Adapter should not have any sharp edges that may damage/injure the vehicle/user. Risk: High Test Approach: This requirement is already met by the manufacturer and there will not be additional tests Overheat Description: The STN1170 Bluetooth OBD-II Adapter should not overheat to the point of being able to burn the user, even after long hours of usage. Risk: High Test Approach: This requirement is already met by the manufacturer and there will not be additional tests. 6.5 Maintenance and Support Requirements Support Future Mobile Operating Systems Description: The mobile application will be supported on future releases of other mobile operating systems. Team Overdrive will release new versions of the application for such operating systems. Risk: High Test Approach: Non testable item because there will not be version for other mobile operation system Code Documentation Description: Code shall be well documented with comments detailing information regarding intended use and functionality. A Flowchart shall be included that explains the operation and interaction of all the software units. Risk: High Test Approach: Non testable item Code Enhancement Description: Team Overdrive will continue to improve its program code to further increase speed, performance, and overall efficiency. These changes shall be released as a new revision of the application. Risk: High Test Approach: Non testable item Help Desk Support 10/04/2013 page 31 Overdrive