SE 3S03 - Tutorial 2 Department of Computer Science McMaster University naqvis7@mcmaster.ca Week of Feb 8, 2016 Traditional Specific Battery Usage Test
Outline Traditional Specific Battery Usage Traditional Specific Battery Usage Test
Emulator Emulates real mobile device software, hardware and OS Simulator Simulates internal behavior but does not emulate hardware Traditional Specific Battery Usage Test
Traditional Product Focused Used to find bugs Process Focused Protect software from bugs Traditional Specific Battery Usage Test
Traditional Traditional Specific Battery Usage Test
Traditional Traditional Specific Battery Usage Test
Traditional Process Focused, languages, tools, guidelines, standards Product Focused Static Dynamic White box Black box Traditional Specific Battery Usage Test
Traditional White Box Statement coverage Branch coverage Decision coverage Control Flow Data flow Black Box Boundary values Decision Table Equivalence class partitioning Traditional Specific Battery Usage Test
Specific Functional Usability Accesibility Traditional Specific Battery Usage Test
Functional Execute test cases Test if features and requirements have been correctly implemented Test acceptance criteria Test all functions Traditional Specific Battery Usage Test
Usability Less is more Avoid useless navigation, text/words Self explanatory Clear and understandable elements Pay attention to patterns Check compliance with design pattern Allow mistakes User friendly error messages, phrase messages according to target customer, allow user to correct errors Traditional Specific Battery Usage Test
Usability Check the wording Easy wording, free of typos, text size in different languages Check consistency Same look and feel in all views, platforms, corporate identity guidelines Write it down Note down differences, screen capture, categorize bugs Traditional Specific Battery Usage Test
Accesibility Visual Impairment Make elements available for screen readers Adjustable font size Allow changing color contrasts Voice recognition Auditory Impairment Vibrations or visual notifications Subtitles for video content Volume control Mono audio Physical Impairment Use voice recognition Processing speed Traditional Specific Battery Usage Test
Accesibility Cognitive Impairment Provide easy interface Screen readers, text highlighting Auto text feature Accesibility guidelines Available for Android and ios Traditional Specific Battery Usage Test
Battery Usage Fully Charged Battery Foreground Background Proxy tools to check requests to back end systems Low Battery 10% to 15% battery left Completely empty battery Traditional Specific Battery Usage Test
Battery Usage Battery Consumption Traditional Specific Battery Usage Test
Battery Usage Battery Consumption Traditional Specific Battery Usage Test
Manual trigger notifications Automatic Generates events like touch, click, screen rotation, swipe Example: Monkey tool for android Traditional Specific Battery Usage Test
Performance Areas Server data networks Device and s Traditional Specific Battery Usage Test
Measure launch time Check for delays during user interactions Measure content loading time Test on different hardware Compare live app version with the new candidate version Profiling tools to measure operation time of the code Traditional Specific Battery Usage Test
Standby Put app into standby mode Check UI refreshes Check data consistency Absence of internet connection Traditional Specific Battery Usage Test
Installation Verify installation on local storage or memory card Check installation with different internet connections Change connection during installation Switch to other Switch internet off Install with less memory remaining Uninstall Traditional Specific Battery Usage Test
Update Logged in user should not be logged out after update Existing app data should not be modified Traditional Specific Battery Usage Test
Database Validation Integration Performance Procedure and Function Trigger CRUD operation Integration with UI Security Traditional Specific Battery Usage Test
Local Storage When storage is full When local storage is full but extended has some space When local has some space but extended is full When both are full When local is nearly full Move app to extended storage When local storage is empty Traditional Specific Battery Usage Test
Security Common app security problems Cache storage Unintended local storage Encryption Client side validation Un encrypted communication Unnecessary app permissions Traditional Specific Battery Usage Test
Security Security requirement analysis Identify user roles and permissions Impact of user role on back end system Security approaches and tools Traditional Specific Battery Usage Test
Platform Guideline Keep platform specific guidelines in mind Include guidelines in design phase Android guidelines Apple ios design guidelines Apple ios approval guidelines Traditional Specific Battery Usage Test
Conformance European Telecommunication Standards Institute World Wide Web Consortium Institute of Electrical and Electronics Engineering Traditional Specific Battery Usage Test
Bugs Bug ID Bad: 123 Good: AppXYZ-123 Description Bad: App crashed, blank screen Good: Error code 542 on detail message view, Time out when sending a search request Steps to reproduce Bad: I tried to execute a search Good: Start the app and enter into search input field. Press search button and you will see error code 783 Traditional Specific Battery Usage Test
Bugs Expected result Bad: It should work, It shouldnt have crashed Good: expected to see search results with maximum 20 entries Actual result Bad: It didnt work Good: The search result page was empty Work-around Bad: I found a work around Good: search results are visible if the device is in landscape mode Traditional Specific Battery Usage Test
Bugs Reproducible Bad: Sometimes Good: occurs two out of ten times Operating system, platform, device Bad: Android Good: Android, Version 4.1.2 Google Nexus 4 Device specific information Bad: No information Good: Battery state=15%, GPS was activated Traditional Specific Battery Usage Test
Bugs Browser version Bad: Google Chrome Good: Google Chrome version 45.4343 Software build version Bad: No information Good: App build version 1.2.3 Network condition and environment Bad: Happened while walking Good: Connected to 3G while walking through a place surrounded with buildings Traditional Specific Battery Usage Test
Bugs Language Bad: No information Good: German Language Test data Bad: No information Good: Find attached SQL script Severity Bad: No information Good: Critical Traditional Specific Battery Usage Test
Bugs Bug category Bad: No information Good: UI, Performance Screenshot or video Bad: Screenshot1 Good: 01SearchResultPage Log files Bad: No information Good: Attached, with full stack trace Tester information Bad: No information Good: Name. Email: name@xyz.com Traditional Specific Battery Usage Test
Pyramid Traditional Specific Battery Usage Test
Pyramid Traditional Specific Battery Usage Test
Flipped Pyramid Traditional Specific Battery Usage Test
Flipped Pyramid Traditional Specific Battery Usage Test
Test Pyramid Traditional Specific Battery Usage Test
Test Pyramid Traditional Specific Battery Usage Test
Image recognition Embed screenshots into script Example: eggplant, RoutineBot Coordinate based recognition Rely on predefines coordinates to interact with UI elements Examples: MonkeyTalk, Perfecto OCR/text recognition To determine if text is present on screen Examples: eggplant, MonkeyTalk Traditional Specific Battery Usage Test
Native object recognition Accesses UI elements using CSS or object ID of elements Examples: Appium, Calabash Capture and replay Captures activity, replays it to test Traditional Specific Battery Usage Test
Business critical parts User workflows and scenarios Complex app scenarios Sequences that need repetition Acceptance criteria Automate if economically reasonable! Traditional Specific Battery Usage Test
Support for different app types platforms Recognition technology Tests on real device, simulator, emulator Test execution time UI and control elements Gesture support Language tests Open source or closed source Cross platform tests Traditional Specific Battery Usage Test
Crowd Cloud Traditional Specific Battery Usage Test
Crowd Crowd outsourcing Diverse backgrounds: geographically and technically Progress of the crowd testers is made visible to the client Traditional Specific Battery Usage Test
Crowd Process Briefing between crowd provider and client Crowd provider selects crowd based on client requirements Crowd tests the software Testers file reports Provider ensures quality of report Provider writes final test report at the end of the test cycle Report is presented to client Traditional Specific Battery Usage Test
Pros Testers with different demographic background and skill sets Different mobile devices App tested in real world conditions by real users Provider filters and categorizes bugs Traditional Specific Battery Usage Test
Cons Testers are generally not experts Reports may be of low quality Access to back end systems may be limited due to privacy issues Time taking process Reproducing bugs can be difficult Testers may keep using the app after cycle has ended Traditional Specific Battery Usage Test
Cloud Wide range of devices with different hardware and software available on the cloud The cloud uses a dynamic, shared and virtual IT infrastructure The cloud provides on demand self service Priced according to consumption Traditional Specific Battery Usage Test
Cloud Types Functional Load Device Cross Browser Traditional Specific Battery Usage Test
Pros Easy access to physical devices Easy access to emulators and simulators Fast and easy setup of mobile devices Accessibility from anywhere in the world Lower costs No maintenance cost Traditional Specific Battery Usage Test
Cons Less control over devices Network issues Security and privacy issues. Delete app after session complete. Sensors and interfaces cannot be tested No physical contact with the device Traditional Specific Battery Usage Test
usability, accessibility and functional Bug reporting Test automation methods Traditional Specific Battery Usage Test
References I Appendix References Daniel Knott Hands-On.