AUTOMATED TEST SYSTEM DEVELOPMENT FROM SCRATCH: THE MAIN PROBLEMS AND THEIR SOLUTIONS. Lilia Sapurina

Transcription

1 AUTOMATED TEST SYSTEM DEVELOPMENT FROM SCRATCH: THE MAIN PROBLEMS AND THEIR SOLUTIONS Lilia Sapurina

2 ABOUT ME 2 years in huge company with delivered QA processes: automated UI testing, CI 4 month in small company without any automated QA processes: development CI processes, automatisation

3 WHO IS AN AUTOMATION ENGINEER? Company with processes Company without processes

4 MAIN STEPS Syntax analysis Documentation Unit tests Rest API autotests UI autotests Autotest code Reports

5 SYNTAX ANALYSIS

6 SYNTAX ANALYSIS: ADVANTAGES Eliminates small typos Creates universal code Simplifies documentation Simplifies testing

7 SYNTAX ANALYSIS: PROBLEMS Not enough time to analyse the rules A large number of invalid code Not enough time to fix

8 SYNTAX ANALYSIS: SOLUTION Selection of basic rules Rules for reconciliation - non-critical errors Validated new code

9 SYNTAX ANALYSIS: TOOLS SonarQube - too complex Tools in the native language (e.x. Javascript - ESLint, php - PHPLint) Plugins/Special settings for IDE

10 DOCUMENTATION

11 DOCUMENTATION: ADVANTAGES Improve team communication Full use of existing functionality Getting rid of duplicates Simplify refactoring Simplify testing (REST API, unit tests, UI Autotests)

12 DOCUMENTATION: PROBLEMS Not enough time to produce detailed comments in new methods A large number of nondocumented code

13 DOCUMENTATION: SOLUTION Prioritisation: REST API methods, methods for unit-tests, helpers Add rules to syntax analysis configuration file

14 DOCUMENTATION: TOOLS Tools in the native language (JSDoc, JavaDoc, PHPDocumentor, EDoc) APIDoc

15 WHY APIDOC? APIDOC Swagger Description In code In Swagger Editor Usage REST API, Autotest (Rest, UI), helpers REST API, Autotest (REST) Language Simple comments Special layout

16 DOCUMENTATION: EXAMPLE

17 UNIT TESTS

18 UNIT TESTS: ADVANTAGES Quick bug fix Can be used like documentation Simplified refactoring

19 UNIT TESTS: PROBLEMS Not enough time to develop tests for old functionality New frameworks A large number of unit tests interfere with the development of the project

20 UNIT TESTS: SOLUTION Partial implementation: minimal test coverage Remote testing infrastructure development Instructions and examples development Tasks for new applicants

21 UNIT TESTS: TOOLS AngularJs: Jasmine + PhantomJS PHP: PHPUnit Erlang: EUnit

22 AUTOTEST: REST API

23 AUTOTEST (REST API): ADVANTAGES Continuous monitoring of the system functioning Quick bug fix Detecting bug causes Vulnerability scanner

24 AUTOTEST (REST API): PROBLEMS Wrong test structure Incomprehensible reports Misunderstanding with developers

25 FIND DEFECT REASON: ALGORITHM Report Documentation Tester

26 AUTOTEST (REST API): SOLUTION Instrumental analysis Parametrisation Resemblance of programming languages

27 AUTOTEST (REST API): TOOLS Java + Junit Allure ApiDoc

28 ADDITIONAL LOGS

29 AUTOTEST: UI

30 AUTOTEST (UI): ADVANTAGES Continuous monitoring of the system functioning Unpredictable bugs search Smoke tests

31 AUTOTEST (UI): PROBLEMS Constant design changes Functionality changes Testing duration

32 AUTOTEST (UI): SOLUTION Basic functionality allocation Basic scenarios for sprint Constant run of the simplest and fastest tests Analysis of reports by the testing team

33 AUTOTEST (UI): TOOLS Protractor + Jasmine Allure ApiDoc Testrail

34 AUTOTEST CODE

35 AUTOTEST CODE: CHECKS Syntax analysis Documentation Spelling

36 REPORTS

37 REPORTS: ADVANTAGES Improve communication between developers and testers Quick bug fix Statistical data

38 REPORTS: PROBLEMS Variety of tools Uninformative notifications Overextended error analysis

39 REPORTS: SOLUTION Similar tools Brief review Isolate reports and continuous integration system Variety of reports

40

41 FIRST RESULTS (4 MONTHS) Jenkins builds + Pull Request System + Notifications + Reports + Load Tests Backend (php) Frontend (AngularJS) Messenger (Erlang) Syntax Analysis 80 % 100 % - Unit Tests 40 % 30 % 10 % Autotest 60 API methods (30% coverage) 20 use cases 10 basic smoke test 3-4 tests per week 100% coverage (use cases)

42 THANK YOU

43 USEFUL LINKS presentations/developing-an-automated-front- end-testing-system-with-protractor-framework