Kurento Real Time Media Stream Processing. Juan Ángel Fuentes Software Developer. Stream Oriented GE

Transcription

1 Kurento Real Time Media Stream Processing Juan Ángel Fuentes Software Developer. Stream Oriented GE

2 Introducing multimedia infrastructures Peer-to-Peer Application (without media infrastructure) WebRTC video stream Application with media infrastructure Media infrastructure 1

3 Media is here Multimedia infrastructures for the Future Internet Transcoding Media got there MCU Recording Media is here Enrich Augment Adapt Events Media got there Analyse Transform Store Sensors Context 2

4 Enrich, augment, adapt, analyse, transform, store: what s the problem? Complexity 3

5 The Stream Oriented Generic Enabler SO GE Multimedia infrastructure Interoperable media exchange (multiplatform/multiprotocol) WebRTC, RTP, HTTP (video tag), etc. Standard capabilities Transcoding, MCU, recording Advanced capabilities Computer vision, augmented reality, mixing, blending, etc. APIs REST API JavaScript API Java API Is distributed through a flexible FOSS license LGPL 2.1 4

6 Kurento: The equation Future Internet Multimedia Infrastructur e Simple Developmen t APIs 5 5

7 General Architecture 6

8 7

9 Key Concepts: Media Elements Functional unit performing a specific action on a media stream Developers abstract from the low level implementation Able to receive media form other elements Able to send media to other elements 8

10 Key Concepts: Types of Media Elements Input Endpoints: injects media Filters: Transform or analyze media Hubs: Manage multiple media flows Output Endpoints: take the media stream out 9

11 Key Concepts: Media Pipeline Media Pipeline: is a chain of media elements, where the output stream generated by one element (source) is fed into one or more other elements input streams (sinks) 10

12 Sink SRC Key concepts: media elements and pipelines Media Element Provides a specific media functionality Send/receive media. These are the Endpoints Process media Transform media Ready to be used New media elements can be added Media Pipeline Media Element Media pipeline Chain of media elements implementing the desired media logic The Media Server provides the capability of creating media pipelines by joining media elements of the toolbox 11

13 12

14 Agnostic Media adapter In charge of making possible that Stream Oriented GE APIS allow developers to combine media elements to create the desired pipeline Fully abstracts all the complexities of media codecs and formats 13

15 Developers create applications just connecting elements Application 2 Application 1 Application 3 Toolbox of media elements 14

16 Media Server (KMS): The nucleus of Kurento KMS is a middleware for media streams: Receives the stream Process the stream Issues the stream Media Source Send Receive Analyze Augment Enrich Transform Transcode Record Process Replicate Media Sink KMS 15

17 Media plane and Signaling plane 16

18 How to control the media server? REST API JSON RPC Websocket interface in KMS 17

19 API Implementations For now 18

20 Multimedia Application Architecture in SO GE Web application three layers parallelism: Presentation layer: Multimedia presentation and capturing Application logic: Multimedia logic in charge of building the pipeline Service Layer: Kurento Media Server 19

21 Kurento Applications Client Application code Kurento Media Server JavaScript Kurento Client Kurento Protocol JSON-RPC over WS Media Traffic Client Application Server Kurento Media Server Application code Signaling protocol Application code Java Kurento Client Kurento Protocol JSON-RPC over WS Media Traffic 20

22 Application execution flow Client Code Kurento API Media Server (KMS) 1 Pipeline creation phase I want this media in this type What you want is here Specific logic (e.g. AAA) Commands requesting the creation of a pipeline (JSON-RPC request) Media pipeline creation 2 Media exchange phase Media exchange between client and server 21

23 Campus Party Brazil 22

24 Campus Party Brazil 23

25 24

26 Let s get started! Tutorials 25

27 What do we need? An instance of the Kurento Media Server FIWARE-LAB Local installation Maven NPM Bower Your favorite IDE 26

28 Tutorial 1 - Hello world 27

29 Tutorial 1 - Hello world 28

30 Tutorial 1 - Hello world, server 29

31 Tutorial 1 - Hello world, client 30

32 Tutorial 1 - Hello world (Node.js) 31

33 Tutorial 1 - Hello world (Node.js) 32

34 Tutorial 2 - WebRTC magic mirror 33

35 Tutorial 2 - WebRTC magic mirror 34

36 Tutorial 2 - WebRTC magic mirror 35

37 Tutorial 3 - WebRTC one-to-one video call with recording and filtering 36

38 Tutorial 3 - WebRTC one-to-one video call with recording and filtering 37

39 Tutorial 3 - WebRTC one-to-one video call with recording and filtering 38

40 Kurento and the FIWARE-LAB FIWARE-LAB Working instance of FI-WARE enabling free experimentation with technology Creating a Kurento instance from an image Use latest version of Kurento images. Creating a Kurento instance using recipes Use Ubuntu LTS clear image Use latest version of Kurento recipes 39

41 Kurento and the FIWARE-LAB 40

42 To learn more 41

43 Thank you! on Twitter