SE 3S03 - Tutorial 2. Zahra Ali. Week of Feb 8, 2016

Transcription

1 SE 3S03 - Tutorial 2 Department of Computer Science McMaster University naqvis7@mcmaster.ca Week of Feb 8, 2016 Traditional Specific Battery Usage Test

2 Outline Traditional Specific Battery Usage Traditional Specific Battery Usage Test

3 Emulator Emulates real mobile device software, hardware and OS Simulator Simulates internal behavior but does not emulate hardware Traditional Specific Battery Usage Test

4 Traditional Product Focused Used to find bugs Process Focused Protect software from bugs Traditional Specific Battery Usage Test

5 Traditional Traditional Specific Battery Usage Test

6 Traditional Traditional Specific Battery Usage Test

7 Traditional Process Focused, languages, tools, guidelines, standards Product Focused Static Dynamic White box Black box Traditional Specific Battery Usage Test

8 Traditional White Box Statement coverage Branch coverage Decision coverage Control Flow Data flow Black Box Boundary values Decision Table Equivalence class partitioning Traditional Specific Battery Usage Test

9 Specific Functional Usability Accesibility Traditional Specific Battery Usage Test

10 Functional Execute test cases Test if features and requirements have been correctly implemented Test acceptance criteria Test all functions Traditional Specific Battery Usage Test

11 Usability Less is more Avoid useless navigation, text/words Self explanatory Clear and understandable elements Pay attention to patterns Check compliance with design pattern Allow mistakes User friendly error messages, phrase messages according to target customer, allow user to correct errors Traditional Specific Battery Usage Test

12 Usability Check the wording Easy wording, free of typos, text size in different languages Check consistency Same look and feel in all views, platforms, corporate identity guidelines Write it down Note down differences, screen capture, categorize bugs Traditional Specific Battery Usage Test

13 Accesibility Visual Impairment Make elements available for screen readers Adjustable font size Allow changing color contrasts Voice recognition Auditory Impairment Vibrations or visual notifications Subtitles for video content Volume control Mono audio Physical Impairment Use voice recognition Processing speed Traditional Specific Battery Usage Test

14 Accesibility Cognitive Impairment Provide easy interface Screen readers, text highlighting Auto text feature Accesibility guidelines Available for Android and ios Traditional Specific Battery Usage Test

15 Battery Usage Fully Charged Battery Foreground Background Proxy tools to check requests to back end systems Low Battery 10% to 15% battery left Completely empty battery Traditional Specific Battery Usage Test

16 Battery Usage Battery Consumption Traditional Specific Battery Usage Test

17 Battery Usage Battery Consumption Traditional Specific Battery Usage Test

18 Manual trigger notifications Automatic Generates events like touch, click, screen rotation, swipe Example: Monkey tool for android Traditional Specific Battery Usage Test

19 Performance Areas Server data networks Device and s Traditional Specific Battery Usage Test

20 Measure launch time Check for delays during user interactions Measure content loading time Test on different hardware Compare live app version with the new candidate version Profiling tools to measure operation time of the code Traditional Specific Battery Usage Test

21 Standby Put app into standby mode Check UI refreshes Check data consistency Absence of internet connection Traditional Specific Battery Usage Test

22 Installation Verify installation on local storage or memory card Check installation with different internet connections Change connection during installation Switch to other Switch internet off Install with less memory remaining Uninstall Traditional Specific Battery Usage Test

23 Update Logged in user should not be logged out after update Existing app data should not be modified Traditional Specific Battery Usage Test

24 Database Validation Integration Performance Procedure and Function Trigger CRUD operation Integration with UI Security Traditional Specific Battery Usage Test

25 Local Storage When storage is full When local storage is full but extended has some space When local has some space but extended is full When both are full When local is nearly full Move app to extended storage When local storage is empty Traditional Specific Battery Usage Test

26 Security Common app security problems Cache storage Unintended local storage Encryption Client side validation Un encrypted communication Unnecessary app permissions Traditional Specific Battery Usage Test

27 Security Security requirement analysis Identify user roles and permissions Impact of user role on back end system Security approaches and tools Traditional Specific Battery Usage Test

28 Platform Guideline Keep platform specific guidelines in mind Include guidelines in design phase Android guidelines Apple ios design guidelines Apple ios approval guidelines Traditional Specific Battery Usage Test

29 Conformance European Telecommunication Standards Institute World Wide Web Consortium Institute of Electrical and Electronics Engineering Traditional Specific Battery Usage Test

30 Bugs Bug ID Bad: 123 Good: AppXYZ-123 Description Bad: App crashed, blank screen Good: Error code 542 on detail message view, Time out when sending a search request Steps to reproduce Bad: I tried to execute a search Good: Start the app and enter into search input field. Press search button and you will see error code 783 Traditional Specific Battery Usage Test

31 Bugs Expected result Bad: It should work, It shouldnt have crashed Good: expected to see search results with maximum 20 entries Actual result Bad: It didnt work Good: The search result page was empty Work-around Bad: I found a work around Good: search results are visible if the device is in landscape mode Traditional Specific Battery Usage Test

32 Bugs Reproducible Bad: Sometimes Good: occurs two out of ten times Operating system, platform, device Bad: Android Good: Android, Version Google Nexus 4 Device specific information Bad: No information Good: Battery state=15%, GPS was activated Traditional Specific Battery Usage Test

33 Bugs Browser version Bad: Google Chrome Good: Google Chrome version Software build version Bad: No information Good: App build version Network condition and environment Bad: Happened while walking Good: Connected to 3G while walking through a place surrounded with buildings Traditional Specific Battery Usage Test

34 Bugs Language Bad: No information Good: German Language Test data Bad: No information Good: Find attached SQL script Severity Bad: No information Good: Critical Traditional Specific Battery Usage Test

35 Bugs Bug category Bad: No information Good: UI, Performance Screenshot or video Bad: Screenshot1 Good: 01SearchResultPage Log files Bad: No information Good: Attached, with full stack trace Tester information Bad: No information Good: Name. Traditional Specific Battery Usage Test

36 Pyramid Traditional Specific Battery Usage Test

37 Pyramid Traditional Specific Battery Usage Test

38 Flipped Pyramid Traditional Specific Battery Usage Test

39 Flipped Pyramid Traditional Specific Battery Usage Test

40 Test Pyramid Traditional Specific Battery Usage Test

41 Test Pyramid Traditional Specific Battery Usage Test

42 Image recognition Embed screenshots into script Example: eggplant, RoutineBot Coordinate based recognition Rely on predefines coordinates to interact with UI elements Examples: MonkeyTalk, Perfecto OCR/text recognition To determine if text is present on screen Examples: eggplant, MonkeyTalk Traditional Specific Battery Usage Test

43 Native object recognition Accesses UI elements using CSS or object ID of elements Examples: Appium, Calabash Capture and replay Captures activity, replays it to test Traditional Specific Battery Usage Test

44 Business critical parts User workflows and scenarios Complex app scenarios Sequences that need repetition Acceptance criteria Automate if economically reasonable! Traditional Specific Battery Usage Test

45 Support for different app types platforms Recognition technology Tests on real device, simulator, emulator Test execution time UI and control elements Gesture support Language tests Open source or closed source Cross platform tests Traditional Specific Battery Usage Test

46 Crowd Cloud Traditional Specific Battery Usage Test

47 Crowd Crowd outsourcing Diverse backgrounds: geographically and technically Progress of the crowd testers is made visible to the client Traditional Specific Battery Usage Test

48 Crowd Process Briefing between crowd provider and client Crowd provider selects crowd based on client requirements Crowd tests the software Testers file reports Provider ensures quality of report Provider writes final test report at the end of the test cycle Report is presented to client Traditional Specific Battery Usage Test

49 Pros Testers with different demographic background and skill sets Different mobile devices App tested in real world conditions by real users Provider filters and categorizes bugs Traditional Specific Battery Usage Test

50 Cons Testers are generally not experts Reports may be of low quality Access to back end systems may be limited due to privacy issues Time taking process Reproducing bugs can be difficult Testers may keep using the app after cycle has ended Traditional Specific Battery Usage Test

51 Cloud Wide range of devices with different hardware and software available on the cloud The cloud uses a dynamic, shared and virtual IT infrastructure The cloud provides on demand self service Priced according to consumption Traditional Specific Battery Usage Test

52 Cloud Types Functional Load Device Cross Browser Traditional Specific Battery Usage Test

53 Pros Easy access to physical devices Easy access to emulators and simulators Fast and easy setup of mobile devices Accessibility from anywhere in the world Lower costs No maintenance cost Traditional Specific Battery Usage Test

54 Cons Less control over devices Network issues Security and privacy issues. Delete app after session complete. Sensors and interfaces cannot be tested No physical contact with the device Traditional Specific Battery Usage Test

55 usability, accessibility and functional Bug reporting Test automation methods Traditional Specific Battery Usage Test

56 References I Appendix References Daniel Knott Hands-On.