3D live Immerse Video Audio interactive 2nd Newsletter. October Project Information. Consortium. Contents. Contact. Page 1.

Transcription

1 Consortium Project Information 3D live Immerse Video Audio interactive multimedia 2nd Newsletter Brunel Univesrity Centre for Research & Technology / Hellas Informatics & Telematics Istitute Institut für Rundfunktechnik Holografika RAI Research Centre Rundfunk Berlin- Brandenburg Instituto de Telecomunicações European Broadcast Union Web site: Contract No: IST STREP Programme: FP7-ICT Start Date: 1st March 2010 Duration: 36 months Coordinator: Brunel University Contact Project Coordinator & Technical Manager: Dr. Amar Aggoun Brunel University School of Engineering and Design Amar.Aggoun@brunel.ac.uk Contents Editorial 1 Consortium 1 Indicative Use Cases 2 System Architecture 4 Examples of 3D 6 Holoscopic Images Dissemination 7 Editorial October 2010 Welcome to the second newsletter from the European Project 3D VIVANT. 3D VIVANT is a research project co-funded by the EU 7th Framework and started its activities on the 1st of March In the period, since the previous newsletter, the consortium has completed the work under work package 2, entitled Use Cases, Architecture and System Design. A number of scenarios and use cases have been produced to showcase the proposed project technology. Also the system architecture of all project components has been defined and the user acceptance metrics have been identified. 3D VIVANT has been disseminated at various international conferences and exhibitions in this period, including the International Broadcast Exhibition 2010 in Amsterdam, Netherland, the Infocomm 2010 in Las Vegas, USA and the Asia and Entertainment Industry Fair 2010 in Korea and several more. Arnold & Richter Cine Technik Administrative Manager: Dr. Emmanuel Tsekleves Brunel University School of Engineering and Design Emmanuel.Tsekleves@ brunel.ac.uk More information can be found on the project web site: 3D VIVANT is a STREP funded by the European Commission under the 7th Framework Programme Page 1

2 Indicative Service Scenarios Introduction One of the main objectives of the 3D VIVANT project, apart from creating a new visual sensation, is to make holoscopic interactive by means of hyperlinks associated to 3D objects segmented and extracted from the video material. Furthermore, it will enable multimodal story-telling, by allowing mixing of 3D holoscopic from a variety of sources, be it real or virtual. The envisaged service scenarios within the context of this project are based on typical genres of TV shows. A scenario tells a story in order to describe a service from an end user s perspective. It also describes roughly the production background from the professional user s perspective. Thus, it describes examples of what happens when someone interacts with the system. These scenarios are presented below, from the en-user perspective. Magazine Scenario - quergelesen quergelesen is a TV magazine show about books for children aged approximately 8 to 14 years. The Lehmann family are keen readers and regularly watch quergelesen. The children enjoy watching the book presentations and it helps them to discover new books to read. Their parents recognise the value of the show as it encourages their children to read more books. Typically, the family sits in various positions around the living room and watches the show on their new holoscopic TV screen. The 3D aspects of the show make for even more exciting viewing for the children. The Bauer family also likes to watch the TV show, but they watch it on their 2D Hybrid Broadcast Broadband TV (HbbTV). By starting the HbbTV application, the family has the option of accessing further information related to the show and hyperlinked objects. The main usage is to watch the transmission on a 3D holoscopic display set on which the embedded hyperlinking functionality allows the user to follow proper links to get further information about the objects which appear in the video. Compatible formats, such as 2D video, are broadcast through traditional channels; the use of HbbTV sets facilitates the user with the hyperlinking functionalities. The audiovisual material is suitably formatted to be streamed on a web server and the appropriate browser/player allows the user to follow the embedded links. The quergelesen show Presenter The quergelesen show demonstarting mixing of real and vrtual Page 2

3 Indicative Service Scenarios Stop-motion - Sandmännchen Sandmännchen (the sandman) is a daily children s show that has been broadcast on German television for over 50 years. It comprises of five minute long shows that are designed to appeal to young children. This scenario is envisaged to be exploited on both broadcast TV and online. In the broadcast version of the scenario, the basic exploitation of the audiovisual product is on a holoscopic 3D TV, used like a usual TV set (i.e., passively). In the online version, through a specific PC player it is possible to both watch the 3D audiovisual material and to follow the hyperlinks associated to the 3D objects embedded in the clip. These links lead to additional information on the subject in the form of further video and audio files and text. Apart from hyperlinking, 3D objects can be used as samples to find similar objects in large 3D object databases. A simpler exploitation mode through a PC platform consists of watching a streamed online version of the programme. In addition, it is possible to watch the programme on a 2D HD HbbTV set. Tommy and his father access the 3D Sandmännchen portal where one can watch all 3D episodes. Tommy chooses yesterday s chapter. Now, he and his father can see marked 3D objects and, by clicking the slightly flashing hot-air balloon, they get access to a website which shows a collection of Sandmännchen vehicles. Now, Tommy can search for his favourite vehicles in Sandmännchen s collection. He can have a look at all the fancy cars, planes and other vehicles which his beloved figure has used over the years. When Tommy clicks the marked house in this 3D episode, he returns to the 3D portal. When Tommy clicks the marked Sandmännchen figure he can access the basic Sandmännchen website and play the easy games there.. Museum Scenario The programme is a documentary show, depicting the evolution of the electronic technology from telegraphy to holoscopy. The story depicts the historical and technological evolution, with references to important historical facts and to the way of living in the corresponding periods. Francesco likes watching the weekly programme dedicated to technology because it is not limited to the description of equipment and their technical characteristics. On the contrary, the authors try to connect the technology with the historical period and the economical and social environments, pointing out the mutual influences. Thus, in the due time, he sits on his armchair in front of the holoscopic TV set and attentively watches the programme. There is also a video-on-demand and online version of this scenario. In the online version Francesco watches TV programmes using an interactive TV set which enables him to follow the hyperlinks associated to selected objects. This way, he can explore the web site associated to the programme where he can find further information both on the selected object itself and about the historical period and the social and economical environment where the object was produced and used. The Sandmännchen show demonstrating mixing of real and vrtual stop-motion Page 3

4 Use Case & System Architecture Use Case In 3D VIVANT there are two kinds of actors interacting with the system, producers and viewers. Producers more or less correspond to TV professionals such as film producers and cameramen, while viewers correspond to home users watching 3D holoscopic and listening to 3D audio. Producers will be able to record real world objects and 3D scenes and create holoscopic. Apart from recording 3D video, producers will be also able to record 3D sound and synchronize the audio and video streams for high quality audiovisual experience. Producers will be also able to process recorded footage and mix holoscopic coming from real and virtual sources, to arrange and edit the separate recorded sound tracks, to set hyperlinks in holoscopic video and to code holoscopic and spatial audio. Holoscopic and spatial audio created and edited by producers will be made available to viewers. They will be able to watch holoscopic on various display devices, such as a light-field display, an integral imaging display and an autostereoscopic display as well as to listen to spatial audio synchronized to video, thus feeling a complete 3D audio-visual experience. 3D VIVANT will allow non-linear video viewing and viewer interactivity. Viewers will be able to follow hyperlinks while watching holoscopic video. Viewers will be also able to retrieve objects and footage stored in databases, that is similar to samples (queries) provided by them to the system. capture holoscopic Producer code holoscopic -spatial audio capture spatial audio create hyperlinks The 3D VIVANT Use Case Diagram synchronize audio and video edit holoscopic edit spatial audio navigate in hypervideo view holoscopic listen to spatial audio search in holoscopic Viewer Page 4

5 Use Case & System Architecture System Architecture The different modules that comprise the 3D VIVANT system are illustrated in the figure below as a conceptual block diagram. As illustrated, five modules can be identified: the Content Generation Unit, Content Processing & Editing Unit, Content Coding Unit, Content Search & Retrieval Unit and Content Terminal. Users interact with the system through the Content Generation Unit and the Content Terminal. As the names suggest, the Content Generation Unit allows users to record holoscopic and 3D sound, while the Content Terminal provides the means to display holoscopic and play back 3D sound. The Content Terminal also serves as a front end to others modules, and allows users to process and retrieve holoscopic, which are capabilities provided by the Content Processing and Editing Unit and the Content Search and Retrieval Unit respectively. Finally, the Content Coding Unit provides coding functionalities that are necessary for storing and transmitting efficiently this new form of audiovisual. 3D VIVANT will allow non-linear video viewing and viewer interactivity. Viewers will be able to follow hyperlinks while watching holoscopic video. Viewers will also be able to retrieve objects and footage stored in databases, that is similar to samples (queries) provided by them to the system. Content Generation Unit Producers Viewers Content Terminal Content Coding Unit Content Processing & Editing Unit Content Search & Retrieval Unit Conceptual overall architecture of the 3D VIVANT system Page 5

6 Examples of 3D Holoscopic Images Examples of 3D Holoscopic Still Images The consortium led by Brunel is working towards the generation of the first 3D holoscopic video. Examples of previously captured 3D Holoscopic still images are shown here. These have been captured using a number of different parameters. The image in the middle column of the page showcases a 3D holoscopic still image that features horizontal parallax whereas the ones on the right column features 3D holoscopic still images with full parallax (both horizontal and vertical). 3D holoscopic image of a fruitball featuring full parallax with a 847 µm lenticular pitch To view these images and get the full 3D effect, a micro-lens array that matches the one used for the capturing of these images must be overlaid in front of the display unit or the printed image. 3D holoscopic image of a horseman featuring horizontal parallax with a 600 µm lenticular pitch 3D holoscopic image of soft toys featuring full parallax with a 250 µm lenticular pitch Page 6

7 Dissemination 3D VIVANT at IBC D VIVANT at Infocomm D VIVANT participated in this year s International Broadcasting Convention (IBC) Exhibition in Amsterdam, Netherlands from the 9th - 14th September IBC is the leading international forum for the electronic media industry. The 3D VIVANT exhibition stand received more than 300 visitors over five days and distributed over 150 project brochures. A lot of interest was shown by leading electronics manufacturing companies (Phillips, Samsung, LG), broadcast production companies (HBO, BBC) as well as broadcasting standard representatives (ETSI, DVB). Most visitors commented on the dynamics and advantages of the proposed 3D holoscopic imaging technology, which offers fatigue-free viewing, single camera-single aperture capture under natural light and many emphasised its future potential in substituting current 3D technologies (stereo and multiview). The project co-ordinator, Dr Aggoun, was also interviewed by a Polish broadcaster about the 3D VIVANT project (see photo). A conference paper related to the 3D VIVANT project was also published under the IBC proceedings. Holografika participated in this year s Infocomm 2010 exhibition in Las Vegas from 9th-11th of June, which attracted more than 32,000 visitors. Holografika showed 3D holoscopic produced by Brunel on its HoloVizio 240P in the 3D Pavillion. In addition to this a presentation was given about the 3D VIVANT project by the company CEO, Mr Tibor Balogh, in the framework of the 3D Comm Seminar organised by Insight Media/Infocomm. Other Dissemination Events & Conferences The project has been also presented at the following conferences and events: - 9th Euro-American Workshop on Information Optics in Helsinki, Finland in July 2010, by Brunel. - Asia Content & Entertainment Industry Fair 2010, 3D Stereoscopic World Conference, in Gwangju, Korea, in September 2010 by Holografika. - 2nd Intelligent Imaging Programme Meeting, in London, UK in September 2010, by Brunel. - Future 4G Mobile Networks Workshop, in Lisbon, Portugal in October 2010 by IT. The 3D VIVANT project co-ordinator being interviewed at the 3D VIVANT IBC stand A 3D holoscopic animation demonstarted on the HoloVizio 240P by the Holografika CEO at Infocomm 2010 Page 7