UNIT-2 Levels of Testing

Transcription

1 Levels of Testing Unit testing Integration testing System testing Acceptance testing - alpha testing, beta testing Static and dynamic testing Manual and automation testing 1 PARUL UNIVERSITY [MCA] Garima Pathak

2 :Introduction to Levels of Testing: There are different FOUR levels during the process of testing. 1) Unit Testing: is a level of the software testing process where individual units/components of a software/system are tested. The purpose is to validate that each unit of the software performs as designed. 2) Integration Testing: is a level of the software testing process where individual units are combined and tested as a group. The purpose of this level of testing is to expose faults in the interaction between integrated units. 3) System Testing: is a level of the software testing process where a complete, integrated system/software is tested. The purpose of this test is to evaluate the system s compliance with the specified requirements. 4) Acceptance Testing: is a level of the software testing process where a system is tested for acceptability. The purpose of this test is to evaluate the system s compliance with the business requirements and assess whether it is acceptable for delivery. There are two sub levels of this level are-alpha Testing and Beta Testing: [Figure of Testing Levels] 2 PARUL UNIVERSITY [MCA] Garima Pathak

3 :Unit Testing: Unit Testing is a level of software testing where individual units/ components of a software are tested. The purpose is to validate that each unit of the software performs as designed. A unit is the smallest testable part of an application like functions, classes, procedures, interfaces. Unit testing is a method by which individual units of source code are tested to determine the individual parts are correct. The unit test focuses on the internal processing logic and data structures within the boundaries of a component. This type of testing can be conducted in parallel for multiple components. METHOD: o Unit Testing is performed by using the White Box Testing method. When is it performed? o Unit Testing is the first level of testing and is performed prior to Integration Testing. Who performs it? o Unit Testing is normally performed by software developers themselves or their peers. In rare cases it may also be performed by independent software testers. o Unit testing frameworks, drivers, stubs, and mock/ fake objects are used to assist in unit testing. Need of Unit Testing: There are a number of reasons to do unit testing rather than testing the entire product: o The size of a single module is small enough that we can locate an error fairly easily. o The module is small enough that we can attempt to test it in some demonstrably exhaustive fashion. o Confusing interactions of multiple errors in widely different parts of the software are eliminated. Unit Test Consideration: The tests that occur as part of unit tests are illustrated schematically in below Figure. The module interface is tested to ensure that information properly flows into and out of the program unit under testing. 3 PARUL UNIVERSITY [MCA] Garima Pathak

4 The local data structure is examined to ensure that data stored temporarily maintains its integrity during all steps in an algorithm s execution. Boundary conditions are tested to ensure that the module operates properly at the boundaries established to limit or restrict processing. All independent paths through the control structure are exercised to ensure that all statements in a module have been executed at least once. And finally, all error-handling paths are tested [Figure of Unit Test] Common errors in computation are: o Incorrect arithmetic precedence o Mixed code operations o Incorrect initialization o Precision inaccuracy o Incorrect symbolic representation of an expression Test cases in unit testing should uncover errors, such as: o Comparison of different data types 4 PARUL UNIVERSITY [MCA] Garima Pathak

5 o Incorrect logical operators or precedence o Expectation of equality when precision error makes equality unlikely o Incorrect comparison of variables o Improper loop termination o Failure to exit when divergent iteration is encountered o Improperly modified loop variables Unit-test Procedure The unit-test environment is illustrated in below Figure. [Figure of Unit Test Environment] In most applications a driver is nothing more than a main program that accepts test-case data, passes such data to the component (to be tested), and prints relevant results. Stubs serve to replace modules that are subordinate (called by) to the component to be tested. 5 PARUL UNIVERSITY [MCA] Garima Pathak

6 A stub uses the subordinate module s interface, may do minimal data manipulation, prints verification of entry, and returns control to the module undergoing testing. Drivers and stubs represent overhead. That is, both are software that must be written but that is not delivered with the final software product. If drivers and stubs are kept simple, actual overhead is relatively low. Unfortunately, many components cannot be adequately unit tested with simple overhead software. In such cases, complete testing can be postponed until the integration test step (where drivers or stubs are also used). Unit testing is simplified when a component with high cohesion is designed. When only one function is addressed by a component, the number of test cases is reduced and errors can be more easily predicted and uncovered. Advantages: o Issues are found at early stage. hence it also helps in reducing the cost of bug fixes. o Codes are more reusable. In order to make unit testing possible, codes need to be modular. This means that codes are easier to reuse. o Development is faster. o The cost of fixing a defect detected during unit testing is lesser in comparison to that of defects detected at higher levels. o Unit testing helps in simplifying the debugging process. o Codes are more reliable. o Due to the modular nature of the unit testing, we can tests parts of project without waiting for others to be completed. Disadvantages: Unit testing can't be expected to catch every error in a program. It is not possible to evaluate all execution paths even in the most trivial programs Unit testing by its very nature focuses on a unit of code. Hence it can't catch integration errors or broad system level errors. Unit Testing Tools: o Rational Software - Rational Software by IBM has a unit test feature known as "Rational Test Realtime". The software contains a complete range of testing tools for much more 6 PARUL UNIVERSITY [MCA] Garima Pathak

7 than just unit testing. It is used for Ada, Java, C and C++. It creates unit tests by reverse engineering the software. Operating systems it supports include Windows, Linux, Solaris, HP-UX and AIX. Go to to learn more. o JavaScript Assertion Unit- Also known as jsasserunit, this Freeware JavaScript unit testing tool can be used on any platform that supports JavaScript. It is available athttp://jsassertunit.sourceforge.net/docs/index.html o CUT - CUT is a Freeware unit test tool for C, C++ and Objective C. It is great for embedded software testing frameworks and desktop applications on Linux and Windows operating systems. Learn more at sourceforge.net by going to o Dotunit - Dotunit is a.net framework Freeware unit testing tool. Part of Junit on the Microsoft.net framework, Dotunit is used for automating unit testing on windows systems. This is another tool from sourceforge.net, so look for it at: 7 PARUL UNIVERSITY [MCA] Garima Pathak

8 :Integration Testing: The second level of testing is called integration testing. Integration testing is a systematic technique for constructing the program structure while at the same time conducting tests to uncover errors associated with interfacing. Integration Testing is a level of software testing where individual units are combined and tested as a group. METHOD: Any of Black Box Testing, White Box Testing, and Gray Box Testing methods can be used. Normally, the method depends on your definition of unit. When is Integration Testing performed? Integration Testing is performed after Unit Testing and before System Testing. Who performs Integration Testing? Either Developers themselves or independent Testers perform Integration Testing. Approaches/methodologies/ strategies of Integration Testing: o Big Bang Approach o Incremental Approach: which is further divided into following Top Down Approach Bottom Up Approach Sandwich Approach - Combination of Top Down and Bottom Up Big Bang Approach: In Big Bang integration testing all components or modules are integrated simultaneously, after which everything is tested as a whole. This approach is generally executed by those developers who follows the Run it and see approach. In this approach individual modules are not integrated until and unless all the modules are ready. As per the below image all the modules from Module 1 to Module 6 are integrated simultaneously then the testing is carried out. Advantages: Convenient for small systems. 8 PARUL UNIVERSITY [MCA] Garima Pathak

9 Everything is finished before integration testing starts. Disadvantages: The major disadvantage is that in general it is very time consuming It is very difficult to trace the cause of failures because of this late integration. The chances of having critical failures are more because of integrating all the components together at same time. If any bug is found then it is very difficult to detach all the modules in order to find out the root cause of it. There is high probability of occurrence of the critical bugs in the production environment [Figure of Big Bang Integration Testing] Incremental Approach: The program is constructed and tested in small increments, where errors are easier to isolate and correct; interfaces are more likely to be tested completely; and a systematic test approach may be applied. This process is carried out by using dummy programs called Stubs and Drivers. Stubs and Drivers do not implement the entire programming logic of the software module but just simulate data communication with the calling module. Stub: Is called by the Module under Test. 9 PARUL UNIVERSITY [MCA] Garima Pathak

10 Driver: Calls the Module to be tested. Incremental approach is further divided into three sub categories: A. Top-down Integration B. Bottom-up Integration C. Sandwich Integration A. Top-down Integration: Top-down integration testing is an incremental approach to construction of the software architecture. Modules are integrated by moving downward through the control hierarchy. Top-Down Integration: There are two approaches: i) Depth-First Integration ii) Breadth-First Integration [Figure-Top-Down Integration] i) Depth-First Integration: o Referring to Figure-E, depth-first integration integrates all components on a major control path of the program structure. o For example, selecting the left-hand path, components M1, M2, M5 would be integrated first. Next, M8 or (if necessary for proper functioning of M2) M6 would be integrated. o Then, the central and right-hand control paths are built 10 PARUL UNIVERSITY [MCA] Garima Pathak

11 ii) Breadth-First Integration: o Referring to Figure-E, incorporates all components directly subordinate at each level, moving across the structure horizontally. o From the figure, components M2, M3, and M4 would be integrated first. The next control level, M5, M6, and so on, follows. Integration Process Steps: 1. The main control module is used as a test driver and stubs are substituted for all components directly subordinate to the main control module. 2. Depending on the integration approach selected (i.e., depth or breadth first), subordinate stubs are replaced one at a time with actual components. 3. Tests are conducted as each component is integrated. 4. On completion of each set of tests, another stub is replaced with the real component. 5. Regression testing may be conducted to ensure that new errors have not been introduced. 6. The process continues from step 2 until the entire program structure is built. Advantages: The tested product is very consistent because the integration testing is basically performed in an environment that almost similar to that of reality Stubs can be written with lesser time because when compared to the drivers then Stubs are simpler to author. Disadvantages: Basic functionality is tested at the end of cycle Needs many Stubs. Modules at lower level are tested inadequately. B. Bottom-Up Integration: Bottom-up integration testing, as its name implies, begins construction and testing with atomic modules (i.e., components at the lowest levels in the program structure). A bottom-up integration strategy may be implemented with the following steps: 11 PARUL UNIVERSITY [MCA] Garima Pathak

12 1. Low-level components are combined into clusters (sometimes called builds) that perform a specific software subfunction. 2. A driver (a control program for testing) is written to coordinate test case input and output. 3. The cluster is tested. 4. Drivers are removed and clusters are combined moving upward in the program structure. [Figure-Bottom-Up Integration] Integration follows the pattern illustrated in Figure -F. Components are combined to form clusters 1, 2, and 3. Each of the clusters is tested using a driver (shown as a dashed block). Components in clusters 1 and 2 are subordinate to Ma. Drivers D1 and D2 are removed and the clusters are interfaced directly to Ma. Advantages: In this approach development and testing can be done together so that the product or application will be efficient and as per the customer specifications. No time is wasted waiting for all modules to be developed unlike Big-bang approach 12 PARUL UNIVERSITY [MCA] Garima Pathak

13 Disadvantages: Critical modules (at the top level of software architecture) which control the flow of application are tested last and may be prone to defects. Early prototype is not possible C. Sandwich Integration: It is an approach to Integration Testing which is a combination of Top Down and Bottom Up approaches. So, it is also called mixed integration testing. In it, the whole system is divided into three layers, just like a sandwich: the target is in the middle and one layer is above the target and one is below the target. The top-down approach is used in the layer that is above the target and the bottom-up approach is used in the layer that is below the target. Testing coverage on the middle layer, chosen on the basis of the structure of the component hierarchy and system characteristics, also combines the advantages of both the top-down and bottom-up approach. It also requires drivers. The ability to plan and control the sequence is hard in it. At the beginning the work parallelism is medium. Stubs are also needed. The ability to test particular paths is medium. 13 PARUL UNIVERSITY [MCA] Garima Pathak

14 Advantages: o Sandwich approach is useful for very large projects having several subprojects.when development follows a spiral model and the module itself is as large as a system, then one can use sandwich testing. o Both Top-down and Bottom-up approach starts at a time as per development schedule. Units are tested and brought together to make a system.integration is done downwards. o It needs more resources and big teams perform both bottom-up and top-down methods of testing at a time or one after the other. Disadvantages: o It require very high cost for testing because one part has Top-down approach while another part has bottom-up approach. o It cannot be used for smaller system with huge interdependence between different modules. It makes sense when the individual subsystem is as good as complete system. o Different skill sets are required for tester at different level as module are separate system handling separate domains like ERP products with modules representing different functional areas. 14 PARUL UNIVERSITY [MCA] Garima Pathak

15 :System Testing: System Testing is a level of the software testing where a complete and integrated software is tested. The purpose of this test is to evaluate the system s compliance with the specified requirements. System testing is actually a series of different tests whose sole purpose is to exercise the full computer based system. System testing should investigate both functional and non-functional requirements of the testing. METHOD Black Box Testing method is used. System testing involves the external workings of the software from the user's perspective. When is it performed? System Testing is performed after Integration Testing and before Acceptance Testing. Who performs it? Normally, independent Testers perform System Testing. Types of System Testing: There are more than 50 types of System Testing. Below listed types of system testing are large software development company would typically use currently: 1. Recovery Testing: Recovery testing is done to demonstrate a software solution is reliable, trustworthy and can successfully recoup from possible crashes. 2. Security Testing: Security testing attempts to verify that protection mechanisms built into a system will, protect it from improper situation. 3. Stress Testing: Stress testing executes a system in a manner that demands resources in abnormal quantity, frequency, or volume. 4. Performance Testing: Performance testing is designed to test the run-time performance of software within the context of an integrated system. Performance testing occurs throughout all steps in the testing process. 15 PARUL UNIVERSITY [MCA] Garima Pathak

16 5. Deployment Testing: Deployment testing, sometimes called configuration testing, exercises the software in each environment in which it is to operate. 6. Usability Testing: Usability testing mainly focuses on the user's ease to use the application, flexibility in handling controls and ability of the system to meet its objectives 7. Load Testing: Load testing is necessary to know that a software solution will perform under real life loads. [Figure-System Testing Types] 16 PARUL UNIVERSITY [MCA] Garima Pathak

17 :Acceptance Testing: Acceptance Testing is a level of the software testing where a system is tested for acceptability. The purpose of this test is to evaluate the system s compliance with the business requirements and assess whether it is acceptable for delivery. METHOD Usually, Black Box Testing method is used in Acceptance Testing. The Acceptance testing is black box tests, means UAT users doesn t aware of internal structure of the code, they just specify the input to the system & check whether systems respond with correct result. When is it performed? Acceptance Testing is performed after System Testing and before making the system available for actual use. There are two types of Acceptance Testing: i) Alpha Testing: o Alpha testing refers to the system testing carried out by the test team within the development organization. o The alpha test is conducted at the developer s site by the customer under the project team s guidance. In this test, users test the software on the development platform and point out errors for correction. o Alpha tests are conducted in a controlled environment. It is a simulation of real-life usage. o It has two phases: In the first phase of alpha testing, the software is tested by in-house developers. They use either debugger software, or hardware-assisted debuggers. The goal is to catch bugs quickly. In the second phase of alpha testing, the software is handed over to the software QA staff, for additional testing in an environment that is similar to the intended use. 17 PARUL UNIVERSITY [MCA] Garima Pathak

18 o Once the alpha test is complete, the software product is ready for transition to the customer site for implementation and development. o The version of the release on which Alpha testing is perform is called Alpha Release. Advantages of Alpha Testing: o Provides better view about the reliability of the software at an early stage o Helps simulate real time user behavior and environment. o Detect many showstopper or serious errors o Ability to provide early detection of errors with respect to design and functionality Disadvantages of Alpha Testing: o In depth functionality cannot be tested as software is still under development stage Sometimes developers and testers are dissatisfied with the results of alpha testing ii) Beta Testing: Beta testing is the system testing performed by a selected group of friendly customers. If the system is com System is installed and all users are asked to use the software in testing mode; this is not live usage. This is called the beta test. Beta tests are conducted at the customer site in an environment where the software is exposed to a number of users. The developer may or may not be present while the software is in use. So, beta testing is a reallife software experience without actual implementation. In this test, end users record their observations, mistakes, errors, and so on and report them periodically. In a beta test, the user may suggest a modification, a major change, or a deviation. Beta testing is also known as Field testing. This is used to get feedback from the market. Developers are taking actions on all issues reported in beta testing after bug triage & then the software application is ready for the final release. The version release after beta testing is called Beta Release. Developers release either a closed beta or an open beta; 18 PARUL UNIVERSITY [MCA] Garima Pathak

19 o Closed beta versions are released to a select group of individuals for a user test and are invitation only, while o Open betas are from a larger group to the general public and anyone interested. The testers report any bugs that they find, and sometimes suggest additional features they think should be available in the final version. This is last stage of the testing where product is sent outside the company or for trial offer to download. Advantages Beta Testing o Reduces product failure risk via customer validation. o Beta Testing allows a company to test post-launch infrastructure. o Improves product quality via customer feedback o Cost effective compared to similar data gathering methods o Creates goodwill with customers and increases customer satisfaction Disadvantages Beta Testing o Test Management is an issue. As compared to other testing types which are usually executed inside a company in a controlled environment, beta testing is executed out in the real world where you seldom have control. o Finding the right beta users and maintaining their participation could be a challenge Alpha Testing Beta Testing 1. It is always performed by the developers 1. It is always performed by the customers at at the software development site. their own site. 2. Sometimes it is also performed by 2. It is not performed by Independent Testing Independent Testing Team. Team. 3. Alpha Testing is not open to the market 3. Beta Testing is always open to the market and public and public. 4. It is conducted for the software 4. It is usually conducted for software product. application and project. 5. It is always performed in Virtual 5. It is performed in Real Time Environment. Environment. 6. It is always performed within the 6. It is always performed outside the organization. organization. 7. It is the form of Acceptance Testing. 7. It is also the form of Acceptance Testing. 8. Alpha Testing is definitely performed and 8. Beta Testing (field testing) is performed 19 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA PATHAK

20 carried out at the developing organizations and carried out by users or you can say people location with the involvement of developers. at their own locations and site using customer data. 9. It comes under the category of both White 9. It is only a kind of Black Box Testing. Box Testing and Black Box Testing. 10. Alpha Testing is always performed at the 10. Beta Testing is always performed at the time of Acceptance Testing when developers time when software product and project are test the product and project to check marketed. whether it meets the user requirements or not. 11. It is always performed at the developer s 11. It is always performed at the user s premises in the absence of the users. premises in the absence of the development team. 12. It is considered as the User Acceptance 12. It is also considered as the User Testing (UAT) which is done at developer s Acceptance Testing (UAT) which is done at area. customers or users area. 20 PARUL UNIVERSITY [MCA] Garima Pathak

21 Static Testing: :Static and Dynamic Testing: Static testing is the testing of the software work products manually, or with a set of tools, but they are not executed. It starts early in the Life cycle [SDLC-V Model] and so it is done during the verification process. The fundamental objective of static testing technique is to improve the quality of the software products by finding errors in early stages of software development life cycle. Following are the main Static Testing techniques used: 1. Informal Reviews- A review not based on a formal (documented) procedure. 2. Walkthrough-A step-by-step presentation by the author of a document in order to gather information and to establish a common understanding of its content. 3. Technical Reviews- A peer group discussion activity that focuses on achieving consensus on the technical 4. Inspection-A type of peer review that relies on visual examination of documents to detect defects, eg., violation of development standards and non-conformance to higherlevel documentation. The most formal review technique and therefore always based on a documented procedure. 5. Static Code Analysis- The code written by developers are analysed (usually by tools) for structural defects that may lead to defects. Advantages: - Capture defects early, so saves rework cost - Checklist-based approach - Focuses on coverage - Group perspective - Highest probability of finding defects Disadvantages: - Time-consuming 21 PARUL UNIVERSITY [MCA] Garima Pathak

22 - Cannot test data dependencies - High skill levels required Dynamic Testing: This testing technique needs computer for testing. It is done during Validation process. The software is tested by executing it on computer. Example of this Dynamic Testing Technique: testing, integration, system testing, acceptance testing Validation, the dynamic testing part is more hands on and happens on the product itself and not on an artifact or a representation of the product. A much formal process of test case/condition identification, coverage considerations, execution and defect reporting all mark the dynamic testing methods. Advantages: 1. Considered High Level Exercise: One of the main and most crucial advantages of Dynamic Testing is that it can always find errors that static testing cannot find and that is the reason why it is considered as high level exercise. 2. Improves Quality: Another main advantage of Dynamic Testing is that it improves the quality of software because Dynamic Testing means executing the software and when we execute the software chances are higher for finds many bugs in the application so it ensures error free software to some extent and hence increases the quality of a product and project. Disadvantages: 1. Time Consuming: Main disadvantage of Dynamic Testing is that it is always considered as a time consuming task because its aim is to execute the application/software or code and hence as a result more test cases are needed to execute, so obviously executing more test cases can result in waste of time means consume lots of time. 22 PARUL UNIVERSITY [MCA] Garima Pathak

23 2. Increases cost of Product: Another disadvantage of Dynamic Testing is that it increases the cost of project/product because it is not starting early in the software life cycle and hence bugs fixed in later stages can result in increase of cost. 3. Execution Costly: One of the main and most important disadvantages of Dynamic Testing is that its execution is costly because as you know Dynamic Testing means execution of software so it requires more man power to complete its task and hence resulted in higher cost to perform Dynamic Testing. Difference between Static and Dynamic Testing: Static Testing Testing done without executing the program This testing does verification process Dynamic Testing Testing done by executing the program Dynamic testing does validation process Static testing is about prevention of defects Static testing gives assessment of code and documentation Static testing involves checklist and process to be followed This testing can be performed before compilation Static testing covers the structural and statement coverage testing Cost of finding defects and fixing is less Return on investment will be high as this process involved at early stage More reviews comments are highly recommended for good quality Dynamic testing is about finding and fixing the defects Dynamic testing gives bugs/bottlenecks in the software system. Dynamic testing involves test cases for execution Dynamic testing is performed after compilation Dynamic testing covers the executable file of the code Cost of finding and fixing defects is high Return on investment will be low as this process involves after the development phase More defects are highly recommended for good quality. 23 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA PATHAK

24 Requires loads of meetings Comparatively requires lesser meetings 24 PARUL UNIVERSITY [MCA] Garima Pathak

25 :Manual and Automation Testing: Testing of software can be done in both Automation and Manual testing method, but it s totally depends on the project requirement, budget associated with the project, and which testing method will be benefited to the project. Manual Testing: In Manual Testing, Testers manually execute test cases without using any automation tools. Manual testing is the most primitive of all testing types and helps find bugs in the software system. Any new application must be manually tested before its testing can be automated. Manual testing requires more effort, but is necessary to check automation feasibility. Manual Testing does not require knowledge of any testing tool. One of the Software Testing Fundamental is "100% Automation is not possible". This makes Manual Testing imperative. Manual Testing Types: o In general any type of software testing type can be executed both manually as well using an automation tool. o Bus some specific manual testing types are as below: o Unit Testing o This initial stage in testing normally carried out by the developer who wrote the code and sometimes by a peer using the white box testing technique. o Integration Testing o This stage is carried out in two modes, as a complete package or as an increment to the earlier package. Most of the time black box testing technique is used. However, sometimes a combination of Black and White box testing is also used in this stage. o System Testing o In this stage the software is tested from all possible dimensions for all intended purposes and platforms. In this stage Black box testing technique is normally used. o User Acceptance Testing 25 PARUL UNIVERSITY [MCA] Garima Pathak

26 o This testing stage carried out in order to get customer sign-off of finished product. A 'pass' in this stage also ensures that the customer has accepted the software and is ready for their use. Advantages: o Short-term cost is lower o More likely to find real user issues o Manual testing is flexible o Manual testing is cost-efficient for tests run only a few times Disadvantages: o Certain tasks are difficult to do manually o Load testing and performance testing is not possible in manual testing. o Running test manually is very time consuming job. o Can t reuse manual tests-regression Test cases are time consuming if it is manual testing. o Manual testing is prone to human error, so may lead to inconsistent or misleading results Automation Testing: Automation Testing means using an automation tool to execute your test case suite. The automation software can also enter test data into the System Under Test, compare expected and actual results and generate detailed test reports. Software Test automation makes use of specialized tools to control the execution of tests and compares the actual results against the expected result. Usually regression tests, which are repetitive actions, are automated. Testers write test scripts and test cases using the automation tool and then group into test suites. Why Automated Testing? Automated testing is important due to following reasons: o Manual Testing of all work flows, all fields, all negative scenarios is time and cost consuming 26 PARUL UNIVERSITY [MCA] Garima Pathak

27 o It is difficult to test for multi lingual sites manually o Automation does not require Human intervention. You can run automated test unattended (overnight) o Automation increases speed of test execution o Automation helps increase Test Coverage o Manual Testing can become boring and hence error prone. Which Test Cases to Auotomate? Test cases to be automated can be selected using the following criterion to increase the automation ROI o High Risk - Business Critical test cases o Test cases that are executed repeatedly o Test Cases that are very tedious or difficult to perform manually o Test Cases which are time consuming The following category of test cases are not suitable for automation: o Test Cases that are newly designed and not executed manually atleast once o Test Cases for which the requirements are changing frequently o Test cases which are executed on ad-hoc basis. Automation Process: o Test tool selection: Test Tool selection largely depends on the technology the Application Under Test is built on. For instance QTP does not support Informatica. So QTP cannot be used for testing Informatica applications. It's a good idea to conduct Proof of Concept of Tool on AUT o Define the scope of Automation: Scope of automation is the area of your Application Under Test which will be automated. o Planning, design and development: During this phase you create Automation strategy & plan. 27 PARUL UNIVERSITY [MCA] Garima Pathak

28 o Test execution: Automation Scripts are executed during this phase. The scripts need input test data before there are set to run. Once executed they provide detailed test reports. Criteria for Tool Selection: For automating any application, the following parameters should be considered. o Data driven capabilities o Debugging and logging capabilities o Platform independence o Extensibility & Customizability o Notifications o Version control friendly o Support unattended test runs Popular Tools that are used for Functional automation: Product Vendor URL Quick Test HP Professional Rational Robot IBM 03.ibm.com/software/products/us/en/robot/ Coded UI Microsoft us/library/dd aspx Selenium Open Source Auto IT Open Source 28 PARUL UNIVERSITY [MCA] Garima Pathak

29 Popular Tools that are used for Non-Functional automation: Product Vendor URL Load Runner HP Jmeter Apache jmeter.apache.org/ Burp Suite PortSwigger Acunetix Acunetix Advantages: o Reliable: Tests perform precisely the same operations each time they are run, thereby eliminating human error o Repeatable: You can test how the software reacts under repeated execution of the same operations. Programmable: You can program sophisticated tests that bring out hidden information from the application. Comprehensive: You can build a suite of tests that covers every feature in your application. o Reusable: You can reuse tests on different versions of an application, even if the user interface changes. o Better Quality Software: Because you can run more tests in less time with fewer resources o Fast: Automated Tools run tests significantly faster than human users. o Cost Reduction: As the number of resources for regression test are reduced. 29 PARUL UNIVERSITY [MCA] Garima Pathak

30 Disadvantages: o Proficiency is required to write the automation test scripts. o Debugging the test script is major issue. If any error is present in the test script, sometimes it may lead to deadly consequences. o Test maintenance is costly in case of playback methods. Even though a minor change occurs in the GUI, the test script has to be rerecorded or replaced by a new test script. o Maintenance of test data files is difficult, if the test script tests more screens. Difference between Manual and Automation Testing: Manual Testing Manual testing is the testing done without the use of any automation tool Manual testing start once the testing team receives a build from the development team. (the zero level testing starts here). If Test Cases have to be run a small number of times it's more likely to perform manual testing Manual tests can be very time consuming Automation Testing Automation testing is the testing done with the help of a tool. Automation testing this depends on the requirements indicated to perform testing. Not all test cases need to be tested using the automation tools. some test cases have to be tested manually. If you have to run a set of tests repeatedly automation is a huge gain Automated Tests can be done in faster manner It allows the tester to perform more ad- Helps performing "compatibility testing" - hoc (random testing) testing the software on different configurations Manual testing is not reliable. Using this Automation testing perform the same method test execution is not accurate all operation each time 30 PARUL UNIVERSITY [MCA DEPARTMENT] GARIMA PATHAK

31 the time. Short term costs are reduced Using manual testing,testing on different machine with different OS platform combination is not possible, concurrently. To execute such task different testers are required. Long term costs are reduced Automation testing can also be done on different machine with different OS platform combination, concurrently. 31 PARUL UNIVERSITY [MCA] Garima Pathak