5th AR Standards Community Meeting March 19-20, Austin, US Marius Preda Institut TELECOM

Transcription

1 MPEG Technologies and Roadmap for AR 5th AR Standards Community Meeting March 19-20, Austin, US Marius Preda Institut TELECOM

2 What is MPEG? A family of standards published by ISO/IEC dealing with: Coding/compression of elementary media: audio/video (MPEG- 1, 2 and 4), 2D/3D graphics () Hybrid (natural & synthehc) scene descriphon, user interachon () Metadata (MPEG- 7) Media management and protechon (MPEG- 21) Sensors and actuators (MPEG- V) Virtual Worlds (MPEG- V) Advanced User interachon (MPEG- U) Media- oriented middleware (MPEG- M) Coding and Delivery in Heterogeneous Environments Dynamic AdapHve Streaming over HTTP

3 What is MPEG? An achvity conhnuing for 24 years Several hundreds of companies from all over the world More than 2300 achve users A proven manner to organize the work in order to deliver useful and used standards - 4 meehngs per year, around the world, 500+ parhcipants - subgroups with clear objechves - well defined procedure Hundreds of millions of devices contain MPEG technologies (cameras, mobile phones, DVD/Blue- Ray players, STB and TVs, )

4 MPEG tools related to AR 1992/ MPEG- 1/2 (AV content) VRML Part 11 - BIFS: - BinarisaIon of VRML - Extensions for streaming - Extensions for server command - Extensions for 2D graphics Part 2 - Visual: - 3D Mesh compression - Face animaion Part 2 Visual - Body animaion First form of augmentaion of broadcast signal

5 MPEG tools related to AR Part 16 - AFX: - A rich set of 3D graphics tools - Compression of geometry, appearence, animaion A rich set of 3D Graphics representaion and compression tools AFX 2 nd EdiIon: - AnimaIon by morphing - MulI- texturing AFX 3rd EdiIon - WSS for terrain and ciies - Frame based animaion AFX 4th EdiIon - Scalable complexity mesh coding

6 MPEG tools related to AR Part 16 - AFX: - A rich set of 3D graphics tools - Compression of geometry, appearence, animaion Part 16 - X3D InteracIve Profile AFX 2 nd EdiIon: - AnimaIon by morphing - MulI- texturing Part 25 - Compression of third- party XML (X3D, COLLADA) AFX 3rd EdiIon - WSS for terrain and ciies - Frame based animaion AFX 4th EdiIon - Scalable complexity mesh coding

7 MPEG tools related to AR x 201x 201x MPEG- V 1st EdiIon - Sensors and actuators - Interoperability between Virtual Worlds MPEG- U Advanced User Interface A rich set of sensors, actuators MPEG- V 2 nd EdiIon: - GPS - Biosensors - 3D Camera 3D Video Compression of video + depth CDVS Feature- points based descriptors for image recogniion

8 MPEG tools related to AR x 201x 201x MPEG- V 1st EdiIon - Sensors and actuators - Interoperability between Virtual Worlds MPEG- U Advanced User Interface Augmented Reality A study about the MPEG offer in the field of AR MPEG- V 2 nd EdiIon: - GPS - Biosensors - 3D Camera 3D Video Compression of video + depth CDVS Feature- points based descriptors for image recogniion

9 MPEG vision on AR Using MPEG technologies it is possible to envision having a generic AR browser. Instead of downloading a new and heavy applicaion for every context, users could simply point to a URL. The browser would retrieve a scenario from the internet, start the video acquisiion, track objects, recognize few of them from visual signatures, recover the camera pose, download 3D graphics, compose a new scene, get inputs from various sensors, and constantly adapt the interacion possibiliies and objects from a remote server to offer an opimal AR experience. Instead of developing a new applicaion for each use case and smart- phone plaaorm, the industry could rely on MPEG- compliant authoring tools, and MPEG- compliant browsers to reach the maximum number of customers.

10 MPEG vision on AR Produce Download Authoring Tool Compression /MPEG- 7/MPEG- 21/ MPEG- U/MPEG- V MPEG Player

11 MPEG Plans for AR Document use-cases Extract requirements Profile existing tools to address the requirements Issue Call for Proposals for new technologies Develop an Application Format for AR as part of MPEG-A

12 AR use-cases Broadcast AR

13 AR use-cases Realistic augmentation

14 AR use-cases AR/MR Games

15 AR Requirements Requirements for Augmented Reality Application Format (N12423) on scene description on media on metadata on remote access on compression on management and protection on integration and communication

16 AR Context

17 Main features of MPEG AR technologies Representation of synthetic and natural objects Real-time BIFS/3D mesh based streaming Objects from remote/local storage, stored/real-time Inherent object scalability User interaction/server generated scene changes Physical context captured/affected by a broad range of sensors/actuators

18 ARAF CfP Call for Proposals for Additional Augmented Reality Technologies (N12588, February 2012, San Jose) - ARAF metadata - ARAF management and protection

19 ARAF WD2.0 Some technical details and a first implementation of ARAF on mobile phone will be presented tomorrow in Session 8