360 MPEG Omnidirectional Media Format (OMAF)

Transcription

1 (JBE Vol. 22, No. 5, September 2017) (Special Paper) 22 5, (JBE Vol. 22, No. 5, September 2017) ISSN (Online) ISSN (Print) 360 MPEG Omnidirectional Media Format (OMAF) a) MPEG Omnidirectional Media Format (OMAF) for 360 Media Sejin Oh a) VR (Virtual Reality) HMD (Head Mounted Displays) 360. MPEG 360 Omnidirectional Media Format. 360 OMAF OMAF MPEG DASH(Dynamic Adaptive Streaming over HTTP) 360. Abstract Virtual Reality (VR) has lately gained significant attention primarily driven by the recent market availability of consumer devices, such as mobile phone-based Head Mounted Displays (HMDs). Apart from classic gaming use cases, the delivery of 360 video is considered as another major application and is expected to be ubiquitous in the near future. However, the delivery and decoding of high-resolution 360 videos in desirable quality is a challenging task due to network limitations and constraints on available end device decoding and processing. In this paper, we focus on aspects of 360 video streaming and provide an overview and discussion of possible solutions as well as considerations for future VR video streaming applications. This paper mainly focuses on the status of the standardization activities, Omnidirectional MediA Format (OMAF), to support interoperable 360 video streaming services. More concretely, MPEG's ongoing work for OMA aims at harmonization of VR video platforms and applications. The paper also discusses the integration in MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH), which is considered as 360 video streaming services with OMAF content. In context of the general OMAF service architecture. Keyword : VR, Omnidirectional Media Format, 360 Video, Streaming a) LG (Advanced Standard R&D Laboratory, LG Electronics) Corresponding Author : (Sejin Oh) sjin.oh@lge.com ORCID: Manuscript received July 31, 2017; Revised September 19, 2017; Accepted September 19, Copyright 2017 Korean Institute of Broadcast and Media Engineers. All rights reserved. This is an Open-Access article distributed under the terms of the Creative Commons BY-NC-ND ( which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited and not altered.

2 : 360 MPEG Omnidirectional Media Format (OMAF) (Sejin Oh: MPEG Omnidirectional Media Format (OMAF) for 360 Media). (Virtual Reality, VR) VR (Oculus Rift, HTC Vive, Samsung GearVR, Sony PlayStation VR Google Daydream HMD (Head Mounted Displays). VR HMD VR ,, Ricoh Theta, Samsung Gear 360 LG 360 Cam Nokia OZO, GoPro Omni Fraunhofer OmniCam YouTube Facebook VR 360. VR MPEG (OMAF) [1]. 3GPP [2] VR W3C WebVR Community Group [3] VR. MPEG 360 Omnidirectional Media Format ( OMAF) OMAF 3 OMAF / OMAF 360 / OMAF Fig. 1. OMAF Architecture for 360 media [1]

3 (JBE Vol. 22, No. 5, September 2017) OMAF ((Bi),, 2D (D). (Ei) (Ev). 360 (Ba) (Ea).,, (F) (Fs) 360. (Fs) DASH. (F') (F's), (E'a, E'v / E'i).,,. 360 (A i) (A a) HMD headphone, D (fisheye). (D),. / 360 (A i) HMD.. OMAF OMAF. 360 (Bi) 3 ( ). 2D (C)., (D) /. 360, 2D. 2D. 1. (Projection) D. Equirectangular Projection (ERP) Cube Map Projection (CMP). ERP 360, 3, OMAF Fig. 2. OMAF video processing (stitching, projection, and region-wise packing) of input images

4 : 360 MPEG Omnidirectional Media Format (OMAF) (Sejin Oh: MPEG Omnidirectional Media Format (OMAF) for 360 Media) Equirectangular projection (ERP) Fig. 3. Equirectangular projection of 360 video 2D 360 2D. ERP 2D., 4, D Cube Map Projection (CMP). CMP ERP 2D ERP OpenGL. CMP 360 HEVC (High Efficiency Video Coding). 2. (Region-wise packing), cube map projection Fig. 4 Cube map projection of 360 video 5. ERP Fig. 5. A working example when region-wise packing is applied into an ERP projected image

5 (JBE Vol. 22, No. 5, September 2017)., 5, ERP,,... /. OMAF. 3. (Recommended Viewport) (Initial Viewpoint) OMAF HMD ( ). TV VR TV , OMAF,,, 360,, 360

6 : 360 MPEG Omnidirectional Media Format (OMAF) (Sejin Oh: MPEG Omnidirectional Media Format (OMAF) for 360 Media) D 360 / / / , DASH (,, ). 360 DASH DASH ( : ERP CMP) DASH / / DASH. /,, DASH / DASH., DASH, 360, DASH / MPD (Media Presentation Description). DASH,., DASH,,..

7 (JBE Vol. 22, No. 5, September 2017) 360 FoV (Field of View) DASH., / DASH DASH. DASH MPD DASH... (E2E) (Random Access Point, RAP) DASH. DASH. DASH SRD (Spatial Relationship Description) [4]. DASH FoV. (E2E) RAP (Random Access Points), HEVC.. MPEG 360 Omnidirectional Media Format OMAF 360 OMAF. MPEG DASH 360. OMAF. (References) [1] w16824, Text of ISO/IEC DIS Omnidirectional MediA Format (OMAF) [2] The 3rd Generation Partnership Project, ( ) [3] WebVR Community Group, ( ) [4] L. D'Acunto, J. van den Berg, E. Thomas, and O. Niamut, "Using MPEG DASH SRD for zoomable and navigable video", ACM MMSys 2016, New York, USA

8 : 360 MPEG Omnidirectional Media Format (OMAF) (Sejin Oh: MPEG Omnidirectional Media Format (OMAF) for 360 Media) : ~ 2016 : LG ~ : LG - ORCID : - :,,,