360 degree test image with depth Dominika Łosiewicz, Tomasz Grajek, Krzysztof Wegner, Adam Grzelka, Olgierd Stankiewicz, Marek Domański

Transcription

1 INTERNATIONAL ORGANISATION FOR STANDARDISATION ORGANISATION INTERNATIONALE DE NORMALISATION ISO/IEC JTC1/SC29/WG11 CODING OF MOVING PICTURES AND AUDIO ISO/IEC JTC1/SC29/WG11 MPEG2017/m41991 January 2018, Gwangju, Korea Source Status Title Author Poznań University of Technology Input 360 degree test image with depth Dominika Łosiewicz, Tomasz Grajek, Krzysztof Wegner, Adam Grzelka, Olgierd Stankiewicz, Marek Domański 1. Introduction MPEG is currently working towards supporting full immersion with 6DoF. In one of the previous documents [1] we have shown that it is possible to render additional omnidirectional view of the scene from monoscopic or stereoscopic omnidirectional video accompanied by omnidirectional depth. Currently limiting factor is lack of high quality content with depth maps available. This documents presents a 3D model of entrance hall of building of Poznan University of Technology, which can be used to render synthetic omnidirectional content. The ability to directly render reference viewpoint in any place in the 3D space allows evaluation of depth estimation + view synthesis pipeline 2. Poznan Hall Scene description Scene presents entrance hall of Faculty of Electronics and Telecommunications of Poznan University of Technology. Main omnidirectional camera is placed in the middle of the all, allowing the user to look around. Additional omnidirectional camera is placed to the left and right of the main camera.

2 Figure 1. Overview of the camera positioning in Poznan Hall 360 scene Depth map rendering There are many depth map formats possible. Some of them have been standardized are used with in MPEG [2]. Rendering engines commonly uses z-buffer with is a form of z-distance depth map. Where distance to the pixel is stored directly. Most of the MPEG software uses normalized disparity format where z-distance is stored as disparity formalized to a range from nearest distance Z near to the farast Z far. Relation of z- distance and normalized disparity stored in depth map is expressed as: 1 Z 1 Z far v = 1 1 v max Z near Z far In order to render depth map for our scene, and store it as normalized disparity map we have used Node editor and implement directly conversion from z-buffer to required format.

3 Figure 2. Node graph of a depth map rendering. It is crucial to turn off all color processing, like white balancing as it may change value of rendered depth before storage in depth file Projection format used All omnidirectional cameras are configured to render images and depth maps in equirectangular projection (ERP) format. The ERP format is the most widely used projection format for representing 360 degree video as a rectangular video. It is the default projection format used by JVET [5] Resolution We have provided rendered images and depth maps in two resolutions: 2048x1024, 8196x4096. Images with other resolution can be rendered out of the provided source blender file.

4 Figure 3. Example frame of rendered Omnidirectional view of the proposed Poznan Hall 360 scene. Figure 4. Example frame of rendered omnidirectional depth map of the proposed Poznan Hall 360 scene

5 Figure 5. Example frame of rendered Omnidirectional view of the proposed Poznan Hall 360 scene. 3. Summary We have provided a 3D model (Blender) of entrance hall of building of Poznan University of Technology, which can be used to render synthetic omnidirectional content. The ability to directly render reference viewpoint in any place in the 3D space allows evaluation of depth estimation + view synthesis pipeline We have also provided computer graphic images rendered with source blender file. 4. Copyrights Individuals and organizations extracting sequences from this archive agree that the sequences and all intellectual property rights therein remain the property of Poznan University of Technology. These materials may only be used for the purpose of developing, testing and promulgating technology standards and for academic usage. Acknowledgement and reference to this source document are required in all documents that report any usage of the materials. The respective owners make no warranties with respect to the materials and expressly disclaim any warranties regarding their fitness for any purpose. 5. Acknowledgement This work has been supported by the public funds as a DS research project for years 2017/2018.

6 6. References [1] K. Wegner, O. Stankiewicz, A. Dziembowski, D. Mieloch, M. Domański "Omnidirectional 6-DoF/3-DoF+ rendering", ISO/IEC JTC1/SC29/WG11, MPEG2017/m40806, Torino, Italy, July [2] K. Wegner, O. Stankiewicz, T. Grajek, M. Domański, Depth map formats used within MPEG 3D frameworks ISO/IEC JTC1/SC29/WG11 MPEG2017/m40019, Geneva, Switzerland, January [2] Y. Ye, E. Alshina, J. Boyce, Algorithm descriptions of projection format conversion and video quality metrics in 360Lib, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 6th Meeting: doc. JVET F-1003, Hobart, AU, 31 March 7 April [3] G. Bang, G. S. Lee, N. Ho H., Test materials for 360 3D video application discussion, ISO/IEC JTC1/SC29/WG11 MPEG2016/M37810, San Diego, USA, February [4] T. Senoh, K. Yamamoto, R. Oi, T. Mishina, M. Okui: Consideration of depth format. ISO/IEC JTC1/SC29/WG11 MPEG m15047, Antalya, Turkey, Jan [5] D. Rusanovskyy, F.-. Chen, L. Zhang, T. Suzuki, "3D-AVC Test Model 8", Joint Collaborative Team on 3D Video Coding Extensions of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 6th Meeting: Geneva, CH, 25 Oct. 1 Nov. 2013, Document: JCT3V-F1003.