Kansas State University. RestGate - Test Plan. Prepared for: Dr. Daniel Andreson Prepared by: Carlos Kirkconnell. March 15, 2010 Reference Number: 001

Transcription

1 Kansas State University RestGate - Test Plan Prepared for: Dr. Daniel Andreson Prepared by: Carlos Kirkconnell March 15, 2010 Reference Number: 001

2 Table of Contents Introduction 4 Test Plan Identifier 4 Objectives 4 Background 4 Scope 4 Test Items 5 Items 5 Features 5 User Interface 5 Functions 5 Database Requirements Functions 6 Recovery 6 Performance 6 Reliability 6 Availability 6 Maintainability 6 Hardware Interfaces 6 Test Plan 1

3 Software Interfaces 6 Reporting 6 Features not to be Tested 7 Security 7 System Portability 7 Approach 8 Interface Testing 8 Security Testing 8 Recovery Testing 8 Performance Testing 8 Regression Testing 8 Criteria 9 Pass/Fail 9 Suspension and Resumption 9 Test Tasks 10 Environmental Needs 11 Hardware 11 High End Server 11 Low End Server 11 Software 11 Operating System 11 Responsibilities 12 Test Plan 2

4 Graduate Student 12 Project Advisor 12 Test Plan 3

5 Introduction Test Plan Identifier RestGate Version 0.1 Objectives The following objectives should be supported by the RestGate test plan: I. Define the activities needed to perform testing II. III. IV. Communicate the test plan approach to the responsible groups. Define the testing tools and environment needed to conduct testing Identify the risks involved in testing Background Ruby on Rails is a revolutionary web application development framework that s being widely adopted by thousands of developers. Unfortunately Ruby on Rails hasn t had a great degree of success on the enterprise environment. One of the reasons of this is the limited support for system integration. Ruby on Rails answer to system integration is REST. Although REST is great for transmitting information between two applications, the tools that currently exists require developers to rewrite much of the messaging logic for every application. Scope Our goal is to provide a tool that will help develop and manage distributed systems using XML in a REST style. This will allow Ruby on Rails application to to avoid rewriting part of the redundant logic required for handling messages between applications. Test Plan 4

6 Test Items We need to be sure we are doing the testing throughout the life cycle of the project. So the team members are responsible to test and review each artifact that has been produced after every iteration. In the Inception and Elaboration phases, the requirements and design models should be reviewed and in Construction and Transition phases, the source code and other artifacts need to be tested. Items The following items must be tested thoroughly to ensure the quality of the product: I. User Interfaces II. III. IV. Functions Database Requirement Functions Recovery V. Performance VI. VII. VIII. IX. Reliability Availability Maintainability Hardware Interfaces X. Software Interfaces XI. Reporting Features User Interface There are two ways through which users interface with a RestGate. The first one are the gate configuration files. The second one are the instrumentation reports. Functions The software must be tested to ensure that messages are sent according to the configuration file describing the channel subjected to the test suite. Test Plan 5

7 Database Requirements Functions All integrity constraints required by the vast and complex database of RestGate is required to be tested. Recovery A live server crash will be simulated to verify that the recovery procedure works as intended. Performance Extensive testing will be done on the performance area. RestGate is a service that communicates system, the performance numbers of the application are of extreme importance to the success of the project. A controlled environment will be provided to perform the tests in the best possible manner. Reliability The reliability of the application will be verified by checking the logs of messages sent and received to the server. Availability RestGate should be able to be configured in a redundant matter in such a way that it s 100% reliable and dependable. Maintainability The quality of the source code should be verified to ensure code maintainability. This will be done by doing code reviews and using complexity detection tools on the source code. Hardware Interfaces The system will be tested on both high end servers (8 core processors, 8 gigabytes of RAM) as well is in a low end environment (1 core processor, 512 megabytes of RAM). Software Interfaces The interfacing of the server with the systems it depends on should be tested. These systems are: I. Database Server II. III. IV. MemCache Server Sender Applications Receiver Applications V. Web Server Reporting The validity of the reporting results in comparison to the database values and the log files will be verified. Test Plan 6

8 Features not to be Tested The following features will not be tested on this version of RestGate: Security Due to time constraints, security testing will be omitted on the first version of the application. The built in security features of the development framework as well as good programming practices will be used to reduce risks. System Portability The system is targeted to run on *nix based systems. No windows version is planned at the moment. Test Plan 7

9 Approach Interface Testing RSpec will be used to test that the correct database values are displayed on the online reports. RSpec will also be used to verify that the configuration file commands generate the correct configuration based on the user s specifications. Security Testing This will be omitted. Recovery Testing Recovery testing will be performed to simulate a hardware failure. The system should be able to restart correctly with no data loss on guaranteed channels. Performance Testing Performance testing of extreme high traffic peaks will be simulated on a controlled testing environments. The results of such testing will give us an idea of how well the application will suite the demands of the market. Regression Testing RSpec will be run with the help of AutoSpec. This combination will provide continuos regression testing of all the specifications, resulting in the immediate detection of failures. Test Plan 8

10 Criteria Pass/Fail RSpec flags features as either passed or failed. Features tested under RSpec will use the run results as pass/ fail criteria. Performance tests will be measured based on the reporting capabilities of RestGate. Messages accepted for delivery by the server in times equal or less than 150 ms will be considered as passed. Times higher than that will be consider fail. The delivery time will be measured with a bigger range of success. Delivery times of up to 5 seconds will be considered passed. Delivery times longer than 5 seconds are fail. Recovery testing will have it s pass/fail criteria based on weather the service is able to get up running again without message loses for guaranteed channels. Suspension and Resumption Software that s not available for the testing schedule will not be tested. Software that s found faulty at the time of testing will have it s testing suspended and resumed until the software works correctly. Test Plan 9

11 Test Tasks See the test workflow in the work break down structure of RestGate. Test Plan 10

12 Environmental Needs Hardware For testing, RestGate will need two type of servers. High End Server This server needs 8 cores and 8 gigabytes of RAM. Multiple worker processes will be delivering messages in parallel Low End Server This server needs 1 core and 512 gigabytes of RAM. A single worker process will be used to deliver messages. Software For database storage, SQLite will be used to store messages. Operating System Any linux distribution capable of running ruby should be a good suite for testing. Once the linux version is picked. We will stick to it for performance testing Test Plan 11

13 Responsibilities Graduate Student Responsible of planning, coordinating and executing the planning of the tests. Configuration of the testing environment and the verification that all the required conditions are met, is also his responsibility. Resolving issues and system debugging are also responsibility of the graduate student. Project Advisor Providing the resources requested by the graduate student to ease the testing of the application. Review the results and provide guidance on the actions required to be performed. Test Plan 12