Continuous Integration Testing for IoT Endpoint Integrity. Arthur Code Curmudgeon Hicken Parasoft

Transcription

1

2 Continuous Integration Testing for IoT Endpoint Integrity Arthur Code Curmudgeon Hicken Parasoft

3 Arthur Hicken is Chief Evangelist at Parasoft where he has been involved in automating various software development and testing practices for over 20 years. He has worked on projects including cybersecurity, database development, the software development lifecycle, web publishing and monitoring, and integration with legacy systems and maintains the IoT Hall-of-Shame Follow Blog: Web:

4 Agenda IoT specifics Challenges in quality assurance Effectiveness of end-to-end testing Doing it better

5 Things are everywhere Industrial Automation Smart Health Smart Home Smart City

6 Things are vulnerable The IoT Hall-of-shame

7 Communication Backbone Identification Hardware Sensors Protocols Software Security Machine Learning Data Brokers / Cloud Platforms Aspects of IoT

8 IoT specifics Multi-layered systems Disparate technologies Functionality spanning across many layers

9 Why it is difficult to test IoT Disparate development technologies hamper introducing consistent testing strategies Not enough code to justify testing syndrome Limited user interface make the solution to look deceptively simple Inverted testing pyramid problem Strong temptation to rely entirely on system testing

10 Labor intensive Effectiveness of end-to-end Difficult to automate Difficult to start early Requires inter-team synchronization Difficult to test error handlers testing Failed tests do not provide precise information about failure points

11 Effectiveness of end-to-end testing End-to-End testing for diabetes health system example Difficult to apply stimulation Difficult to validate test result Difficult to test error handlers

12 Effective IoT Testing Avoid manual end-to-end testing Invest more into automated tests at component Follow testing pyramid rules Measure: test results test effectiveness Determine: Risk

13 How to do it better? Give absolute priority to automated tests Invest time into designing interfaces (API) Use API (service) testing tools to cover interfaces Measure the quality of the test

14 How to do it better? Use stubs and mocks to improve: Facilitate test automation Increase code penetration

15 How to do it better? service virtualization to improve test automation Isolate at the message layer Simulate functional scenarios & performance

16 Test scenario example

17 Test Scenario: REQUEST_NEW_BOLUS_UPDATE_PUMP Tested System Tested System

18 Elements of Continuous Testing Process improvement opportunities Risk Assessment Ensure access to complete test environments Test Environment Access Policy Analysis Define actionable practices Expand test coverage and measure test effectiveness Test Optimization Continuous Testing Advanced Analysis Requirements Traceability Connect functional with non-functional requirements Automate defect prevention and Policy measurement

19 Beyond Automation Quality gates: Organizations can automatically promote software through quality gates when business expectations have been met Real-time feedback from Business Stakeholder Software Feature Developer Customer Support Feature Release Objective assessment and go/no go Defects are eliminated at the point that they are easiest to fix Continuous measurements mean continuous refinement of the process

20 Summary IoT adds complexity - It s not just a device Stubs/mocks/virtualization enables testing Continuous testing catches it before the user

21 Q&A Arthur Hicken: Web: Facebook: Blog: IoT Hall-of-Shame: