End-to-End Agile Testing using Incremental Approach for a Leading EIM Solution Provider ATTENTION. ALWAYS.

Transcription

1 End-to-End Agile Testing using Incremental Approach for a Leading EIM Solution Provider ATTENTION. ALWAYS.

2 ABOUT THE CUSTOMER Our Customer is one of the global leaders in Enterprise Information Management (EIM) solutions disrupting the ECM market by eliminating information silos and providing quick and easy access to the right content from any core business system and device. They have offices in United States, United Kingdom, Finland, France, Germany, Sweden and Australia and they have about 5,00,000 global users across 100 countries supporting in 25 different languages. TECHNOLOGY Desktop, QMS & CK : Test Complete & Jscript. Web : Selenium, Java and TestNG; Selenium, C# and NUnit. Mobile : Selenium, Appium & Java Connector API : C# and NUnit THE CHALLENGE Business Challenges: The product supports desktop, web, mobile and cloud and had about tests which resulted in time consuming manual activities & increased costs Ensuring each build release adhered to the high-quality standards Many custom applications built on the base product and each application has its own release schedule.

3 Technical Challenges: Frequent releases caused versioning issues that ultimately affected a wide range of customers Environment compatibility issues against different operating systems (windows 7/8/10) and browsers like Internet explorer 11, Firefox, Chrome, Safari & Opera and platforms (Windows & MAC). Application Server & Client Compatibility issues were challenging to test with the given timeframe especially during releases. Multi-language support and compatibility across different applications in the product. High flow of regression issues with release builds. THE SOLUTION Aspire provided end-to-end product including manual and automation services for UI Dektop, Web UI, Mobile UI and Connector API. Aspire executed automation tests to ensure high quality and provide quicker feedback.

4 1. UI DESKTOP AUTOMATION Implemented shift-left and encouraged running test automation for each developer change before they merged to a stable branch. Implemented Incremental test automation approach and using AFTA (Aspire s Framework for test ) covered most of the repetitive manual processes by automating the test cases. Along with this, Functional, Compatibility, API, Performance and Security test automation were also implemented successfully. Integrated the CI pipeline using Jenkins and TeamCity for continuous execution of test automation for each and every build eliminating manual steps. Developed around 6580 test cases and was also responsible for re and follow up of bugs which were created in earlier builds until they were fixed. On every stable build, almost 9000 tests with the duration of 405 hrs were running against Windows 7, Windows 8 and Windows 10 using 64 bit & 32 bit operating systems. Also cross browser for Chrome, Firefox, IE11, Edge, Mac Safari browsers, Different versions of Android and IOS mobile and tablets were executed. Provided release documentation for multiple languages - English, Finnish, French, and Dutch. Highlights execution was run for almost 3-4 times for every build to ensure quality Smoke Testing was done for compatibility in different language builds. Performed post-release checks.

5 2. WEB UI AUTOMATION Aspire provided web automation services to the customer for their end-to-end product. Open source automation tool was used which is feasible for most of the product requirement to reduce the software costs. Highlights End-to-End automation using continuous integration tools like Jenkins and TeamCity Identified regression issues at an early stage that lead to a smooth release. Executed cross-browser Provided impressive Dashboard reports through open source reporting plugin

6 3. WEB & MOBILE MANUAL TESTING Aspire extended their support on ad-hoc on the builds to identify any defects which are not covered as part of the sprint scope. We developed test cases in web & 2000 cases in mobile and on every sprint and identified an average of defects as part of services. Also tested bugs which were created in earlier builds to makes sure the bug fixes are working as expected. Highlights High test coverage through Smoke, User Acceptance, Regression & ad hoc tests at different levels Covered on-premises & Cloud test environments Ensured zero defect leakage Manual regression was done for every sprint based on the user story. Ad hoc on pre-release builds after system to increase the test coverage. 4. MOBILE UI AUTOMATION Aspire provided mobile automation services to the customer for their native application consisting of tests of the product. Solution and approach: Open source automation tool was used to reduce the software costs Web automation framework was used and hence less effort was required to scale up scripts across platforms like android & ios product. Supported different versions of Android & IoS, Mobile & Tablet, and Portrait & Landscape views. Compatibility was executed for different OS versions of Android and IOS, Server and Mobile applications.

7 5. CONNECTOR API AUTOMATION Aspire provides API automation services to the customer for their UI-Extension applications that helps connecting various external repositories. We are also in the process of automating the tests consisting of tests of the product with 10 connector applications and there will be 10 more connectors developed each year Highlights Open source automation tool used to reduce the software costs End to End automation, Integrated with CI tools Early identification of compatibility issues Basic performance test reports to measure the performance between different UI-Extension applications. Extent reports are used for reporting

8 THE RESULT Aspire s end to end helped the customer streamline their process and accelerate the product quality with faster market releases. More environment test coverage through test Faster feedback using shift left and continuous. Reduced manual effort spent by executing automation at many places, so that QA members could focus more on the User story & UAT tests for a quicker and promising product release. Project Test Suites Full Regression #Test Cases 8000 Automated Cases 5300 Manual Effort (Hours) 884 Effort Frequency (Hours) 75 Every two weeks Saved Hours 809 Desktop UI Sanity Daily 194 Smoke Daily 39 Compatibility Every two weeks 150 Web UI Full Regression Monthly 430 Sanity Daily 150 Smoke Daily 27 Mobile UI Smoke Regression Weekly Monthly Compliance Kit Regression Weekly 96 Connector API Regression Daily 51

9

10 ATTENTION. ALWAYS. ABOUT ASPIRE Aspire Systems is a global technology services firm serving as a trusted technology partner for our customers. We work with some of the world s most innovative enterprises and independent software vendors, helping them leverage technology and outsourcing in our specific areas of expertise. Our core philosophy of Attention. Always. communicates our belief in lavishing care and attention on our customer and employees. SINGAPORE NORTH AMERICA EUROPE INDIA MIDDLE EAST For more info contact info@aspiresys.com or visit