F-Interop H2020 Project A platform for online interoperability and performance test. Prof. Serge Fdida, Sorbonne Université

Transcription

1 F-Interop H2020 Project A platform for online interoperability and performance test Prof. Serge Fdida, Sorbonne Université 1 - IoT Week 2018 Bilbao, Spain 4 th June 2018

2 EU-China potential for joint activities in: - IoT Interop and Conformance tests - Digital Infrastructures Test Platforms as a Service (including 5G) Serge Fdida --- IoT Week Bilbao, Spain, June

3 IoT Testing interoperability and Conformance Serge Fdida --- IoT Week Bilbao, Spain, June

4 State of the Art: Face-to-Face Events Goals of these events: Make better standards Reduced time-to-market Increase adoption Example: ETSI plugtests 6TiSCH: Jul 15, Feb 16, Jul 16 onem2m: Sep 15, May 16, Nov 16 CoAP: Mar 12, Nov 12, Nov 13, Mar 14 Serge Fdida --- IoT Week Bilbao, Spain, June

5 5 Serge Fdida --- IoT Week Bilbao, Spain, June

6 Challenges with Face-to-Face Events From a user s point of view Serge Fdida --- IoT Week Bilbao, Spain, June

7 IoT 6TiSCH 6TiSCH F-Interop Testing Server GUI IUT sniffer reference implementation Agent Serge Fdida --- IoT Week Bilbao, Spain, June

8 Session model: F-Interop s component User s component Data plane Control plane F-Interop Central Server onem2m Testing Tool GUI GUI agent agent ip tun ip tun User A onem2m ADN (IUT) onem2m CSE (IUT) User B PC PC Remote location IUT: Implementation Under Test Remote location

9 How? Redefining Interoperability tests Turn ETSI plugtests specifications (6TiSCH, CoAP, 6LoWPAN) into F-Interop tests suite Serge Fdida --- IoT Week Bilbao, Spain, June

10 Two communities in motion Serge Fdida --- IoT Week Bilbao, Spain, June

11 Experimentally-driven research Serge Fdida --- IoT Week Bilbao, Spain, June

12 Is this really new? 12 Serge Fdida --- IoT Week Bilbao, Spain, June

13 Can we use this facility for other innovative purposes? 13 Serge Fdida --- IoT Week Bilbao, Spain, June

14 Shared testbeds as a service Isolated testbeds Federated testbeds Shared Testbed as a Services Remote access Interoperability Conformance Security Serge Fdida --- IoT Week Bilbao, Spain, June

15 F-Interop Main Objectives 1. Develop and provide online interoperability and performance testing tools for IoT 2. Support IoT standardization and industry Directly contributing to three global emerging IoT standards: 6TiSCH (IETF) CoAP onem2m Web of Things (W3C) 3. Offer a shared testbed as a service (TaaS) over several FIRE+ testbed federations (Fed4FIRE, OneLab, IoT Lab) Serge Fdida --- IoT Week Bilbao, Spain, June

16 F-Interop in a nutshell Duration 36 months (from Nov to Oct. 2018) EU funding 3 Millions Consortium 10 partners from 6 different countries Serge Fdida --- IoT Week Bilbao, Spain, June

17 Online Interoperability and Conformance Test Tools Serge Fdida --- IoT Week Bilbao, Spain, June

18 Core Idea Implements 6TiSCH CoAP 6LoWPAN Serge Fdida --- IoT Week Bilbao, Spain, June

19 Core Idea sme go.f-interop.eu Tests for: 6TiSCH CoAP 6LoWPAN agent Implements 6TiSCH CoAP 6LoWPAN Conformance Testing Serge Fdida --- IoT Week Bilbao, Spain, June

20 Core Idea smea smeb go.f-interop.eu agent agent Interoperability Testing Serge Fdida --- IoT Week Bilbao, Spain, June

21 Requirements Serge Fdida --- IoT Week Bilbao, Spain, June

22 Requirements F-interop components Serge Fdida --- IoT Week Bilbao, Spain, June

23 Location models User laptop Device(s) Under Test Other devices F-Int. F-interop central services Testbed with devices (IoT, servers,...) -> can also be multiple testbeds, e.g. Cloud testbed to run the test suite/part of the application and IoT testbed for the devices Serge Fdida --- IoT Week Bilbao, Spain, June

24 Identified testing location models A. User downloads all necessary tools and software from F-interop website and executes all locally and processes pcap locally (e.g. for development of new tests) F-Int. F-Int. B. User downloads all necessary tools and software from F-interop website and executes all remotely in a testbed and processes pcap locally or in the testbed (e.g. for dev. new tests) C. User downloads all necessary tools and software From F-interop website and executes all locally and uploads pcap to F-interop (e.g. User wants to orchestrate his own interactions) D. User downloads all necessary tools and software from F-interop and executes all remotely in a testbed and uploads pcap to F-interop (e.g. User wants to orchestrate his own interactions) Serge Fdida --- IoT Week Bilbao, Spain, June

25 Identified testing location models F-Int. E. User downloads all necessary tools and software and tests local DuT against device in testbed (or in central platform) and processes pcap locally F. User downloads all necessary tools and software and tests local DuT against device in testbed (or in central platform) and uploads pcap to F-interop F-Int. F-Int. G. Similar to C, but now a F-interop proxy(s) executes all tests and uploads pcap to F-interop (e.g. Automatic test with minimum interaction of user) F-Int. F-Int. H. Similar to D, but now F-interop proxy(s) runs in remote testbed and executes all tests and uploads pcap to F-interop (e.g. Automatic remote test) Serge Fdida --- IoT Week Bilbao, Spain, June

26 Identified testing location models F-Int. F-Int. I. Similar to F, but now F-interop proxy(s) executes all tests and uploads pcap to F-interop Similar models with 2 remote users We address multiple technologies, but not all previous models are applicable all technologies Serge Fdida --- IoT Week Bilbao, Spain, June

27 Interaction for end-user (finterop session) User registration and login Registration of devices at the user s desk if any User starts by discovering the available test suites and by selecting the one he/she wants to execute. User specifies/selects resources in the F-Interop-Platform that are needed for his/her F-Interop session. The resources selected in the previous step are actually reserved. In case of local resources, the user has to download and execute the Agent locally to talk to the local device. The online F-Interop test campaign is launched and the selected test suites are executed against the IUTs. Test execution information is analysed. The test results and verdicts are provided. Serge Fdida --- IoT Week Bilbao, Spain, June

28 Interaction for a contributor User registration and login User starts by discovering the available test suites and by selecting the one he/she wants to extend or use as template for a new test. The F-Interop platform deploys a full instance of the platform on a testbed User adapts/finetunes/debugs the test or defines a new test and can contribute back by giving the (adapted) code and Ansible Playbooks the F-Interop operators can verify the automatic deployment and test itself and make it available to others as well Serge Fdida --- IoT Week Bilbao, Spain, June

29 Architecture Serge Fdida --- IoT Week Bilbao, Spain, June

30 F-Interop server Architecture user test suite web EventBus orchestrator agent Serge Fdida --- IoT Week Bilbao, Spain, June

31 F-Interop server Contribution #1: Orchestrator user test suite web EventBus orchestrator agent Serge Fdida --- IoT Week Bilbao, Spain, June

32 Contribution #1: Orchestrator Core Idea Wait for users to join Launch (independent) testing tool instances Orchestrate the communication between user(s) and testing tool Design Communication through AMQP protocol (RabbitMQ) One AMQP vhost per testing session (isolation, security) Implementation Working version Deployed on imec premises Starts/stops testing tools instances as Docker containers As part of the testing tools, can start/stop reference implementations on-the-fly Serge Fdida --- IoT Week Bilbao, Spain, June

33 F-Interop server Contribution #2: Web Interface user test suite web EventBus orchestrator agent Serge Fdida --- IoT Week Bilbao, Spain, June

34 Contribution #2: Web Interface Serge Fdida --- IoT Week Bilbao, Spain, June

35 Contribution #2: Web Interface Serge Fdida --- IoT Week Bilbao, Spain, June

36 F-Interop server Contribution #3: Tests user test suite web EventBus orchestrator agent Serge Fdida --- IoT Week Bilbao, Spain, June

37 Contribution #3: Tests CoAP Tests support both CoAP server and client both conformance (reference implementation running on F- Interop platform) and interop Complete test suite with more that 20 test cases for CoAP CORE (RFC 7252) 6TiSCH Architecture built around Wireshark dissector pcap json analysis Online and Offline mode Was used on July at 1 st F- Interop 6TiSCH interop event! Serge Fdida --- IoT Week Bilbao, Spain, June

38 Detailed APIs testing tools API documentation at: Serge Fdida --- IoT Week Bilbao, Spain, June

39 GUI Session orchestrator AMQP accounts F-Interop tests Adding devices AMQP Interop/conformance/ test coordinator + analysis AMQP Resource Repository Result Repository F-Interop central services User 1 premises User IoT devices agent USB User 2 premises User IoT devices agent USB Serge Fdida --- IoT Week Bilbao, Spain, June

40 GUI RSpec REST AMQP accounts F-Interop tests Adding devices AMQP TBaaS engine Interop/conformance/ test orchestration + analysis AMQP server Resource Repository Result Repository F-Interop central services AMQP VMs Testbed VMs Testbed IoT devices Per test on testbed agent device AMQP User premises AMQP User IoT devices agent USB Serge Fdida --- IoT Week Bilbao, Spain, June

41 GUI RSpec REST AMQP accounts F-Interop tests Adding devices AMQP TBaaS engine Interop/conformance/ test Interop/conformance/ orchestration + analysis test Interop/conformance/ orchestration AMQP + analysis server test orchestration AMQP + analysis server AMQP server Resource Repository Result Repository F-Interop central services AMQP VMs AMQP Testbed VMs AMQP VMs VMs Testbed VMs AMQP Testbed VMs AMQP AMQP Testbed IoT agent Testbed devices IoT Testbed devices IoT devices Per test device on Per testbed on Per testbed on testbed agent agent device device AMQP agent User agent User premises agent User premises US premises USB User IoT devices USB User IoT devices User IoT devices Serge Fdida --- IoT Week Bilbao, Spain, June

42 Software Development Kit documentation of the AMQP messages and single test framework at full automatic deployment of the F-Interop software ready for extending Serge Fdida --- IoT Week Bilbao, Spain, June

43 Mutualizing ICT Resources and Test Platform Infrastructures 40 testbeds nodes Serge Fdida --- IoT Week Bilbao, Spain, June

44 SILECS Super Infrastructure for Large-scale Experimental Computer Science Serge Fdida, Sorbonne Université EU ESFRI proposal Serge Fdida --- IoT Week Bilbao, Spain, June

45 Digital transformation, scientific challenges A scalable, robust and secured digital infrastructure Serge Fdida --- IoT Week Bilbao, Spain, June

46 IoT & Cloud Transformation Sébastien Ziegler Serge Fdida --- IoT Week Bilbao, Spain, June

47 Entering ICT Pervasiveness Sébastien Ziegler Serge Fdida --- IoT Week Bilbao, Spain, June

48 Digital Infrastructures - What are the fundamental problems? - What is the purpose/value of an experiment? - Can the experiment be certified ( reproduceability )? Serge Fdida --- IoT Week Bilbao, Spain, June

49 Why do we need SILECS? Research and develop robust, secured and scalable DIGITAL INFRASTRUCTURES SUPPORT RESEARCH on DIGITAL TRANSFORMATION: Scalability (>50 Bio connected devices) Pervasiveness Heterogeneity & Interoperability Virtualization and Cloudification Security and Data Protection TRUSTABLE & REPRODUCIBLE RESULTS Encourage reproducibility and transparency practices in the scientific community, including public access to raw data and protocols, the conduct of replication studies. Serge Fdida --- IoT Week Bilbao, Spain, June

50 Fully Controllable, programmable Virtualized Digital Infrastructure Test Platform Serge Fdida --- IoT Week Bilbao, Spain, June

51 Some background Serge Fdida --- IoT Week Bilbao, Spain, June

52 Building a First Large Internet test Platform Serge Fdida --- IoT Week Bilbao, Spain, June

53 Publications & Achievements Wide variety of publications addressing: R&D for the next generation of Test Platforms Referencing access by Users of Test Platforms Test Platforms as a support to Education and Training Serge Fdida --- IoT Week Bilbao, Spain, June

54 Planetlab cited in more than 16,000 publications Serge Fdida --- IoT Week Bilbao, Spain, June

55 ORBIT testbed cited in more than 25,000 publications Serge Fdida --- IoT Week Bilbao, Spain, June

56 A global initiative towards ICT Test Platforms EU FIRE: Future Internet Research & Experimentation US GENI: Global Environment for Networking Innovations Serge Fdida --- IoT Week Bilbao, Spain, June

57 Funding NSF GENI M$ NSF TIPOFF M$ NSF CloudLab & Chameleon M$ EU FIRE M Serge Fdida --- IoT Week Bilbao, Spain, June

58 International context GAFA Large Scale Infrastructures as a support to the design and validation of systems Example ACM SigComm 2016 scientific publications See Facebook Terragraph Lab No Research Large Scale Infrastructure Serge Fdida --- IoT Week Bilbao, Spain, June

59 Competition PAWR (Platforms Research): for Advanced Wireless NSF + Industry M$ CENI (Chinese Experimental National Infrastructure): : first phase deployment, proof of concept : second phase, scaling up 190 M Serge Fdida --- IoT Week Bilbao, Spain, June

60 The Silecs distributed infrastructure Serge Fdida --- IoT Week Bilbao, Spain, June

61 Motivating Experiment from digital transformation Serge Fdida --- IoT Week Bilbao, Spain, June

62 Physical Architecture Resources distributed in islands Serge Fdida --- IoT Week Bilbao, Spain, June

63 Experiment Execution from a single point of entry / Distributed reservation of resources Serge Fdida --- IoT Week Bilbao, Spain, June

64 Technologies (Resources) IoT Largeclusters of IoT resources generating massively data Connected to the network with state-of-the-art technologies Wireless 4G/5G/WiFi access, Indoor/Outdoor/RF-isolated setups Off-the-shelf and open source equipment fully programmable Massive MIMO, beam stearing and beam forming equipment Mobile nodes with wireless connectivity Cloud Provide access to VM and Bare metal Open cloud testbeds Cloudlab & Chameleon together with NSF Serge Fdida --- IoT Week Bilbao, Spain, June

65 Technologies (examples) Serge Fdida --- IoT Week Bilbao, Spain, June

66 SILECS software SILECS Software Control Plane Virtualized Infrastructure Management Tools SILECS Experimental Plane Software Experiment Control Portal User Interface, single sign-in Serge Fdida --- IoT Week Bilbao, Spain, June

67 Network Operations Center Serge Fdida --- IoT Week Bilbao, Spain, June

68 Serge Fdida --- IoT Week Bilbao, Spain, June

69 Browse Resources Serge Fdida --- IoT Week Bilbao, Spain, June

70 SILECS ESFRI proposal: Supporting governments & organizations Countries FR GR CH ES CY IT DE NL LU BE Gov. Research Industry NRENs Serge Fdida --- IoT Week Bilbao, Spain, June

71 How to develop the concept? Serge Fdida --- IoT Week Bilbao, Spain, June

72 Open API F-Interop Platform Association to maintain the platform Proprietary Online Testing Tools Service Open API Federated Testbed as a Service Open source Module A Module B Module C Module Module SDOs SMEs Researchers CBs Serge Fdida --- IoT Week Bilbao, Spain, June

73 EU & Chinese Perspectives 1. Promote F-Interop concept and platform at SDOs 2. Partner with CENI, Prof Gaogang Xie (CAS) MoU signed in 2015 Joint research activities Serge Fdida --- IoT Week Bilbao, Spain, June

74 Thank you for your attention For more information, please contact: Serge Fdida --- IoT Week Bilbao, Spain, June

75 Back up slides Serge Fdida --- IoT Week Bilbao, Spain, June

76 SILECS IoT Resources Serge Fdida --- IoT Week Bilbao, Spain, June

77 SILECS Wireless Resources Serge Fdida --- IoT Week Bilbao, Spain, June

78 SILECS Cloud Resources Serge Fdida --- IoT Week Bilbao, Spain, June