Challenges and solutions for real-time immersive video communication
|
|
- Horatio Sparks
- 6 years ago
- Views:
Transcription
1 Challenges and solutions for real-time immersive video communication Part III - 15 th of April 2005 Dr. Oliver Schreer Fraunhofer Institute for Telecommunications Heinrich-Hertz-Institut, Berlin, Germany Dr. Oliver Schreer 1
2 Structure of the Short Course Part I (13 th of April 2005) Introduction Scope of immersive video communication Quick tour into projective geometry Part II (14 th of April 2005) Quick tour into camera model and epipolar geometry Analysis of real-time video streams Part III (15 th of April 2005) The concept of real-time immersive video conferencing Quick tour into stereo image processing Hybrid-recursive disparity estimation Dr. Oliver Schreer 2
3 Outline The concept of real-time immersive videoconferencing A quick tour into stereo image processing Hybrid-recursive disparity estimation Dr. Oliver Schreer 3
4 Eye Contact Problem in Video Conferencing Dr. Oliver Schreer 4
5 Application for a Virtual Camera virtual camera set-up stereo camera B virtual camera A Dr. Oliver Schreer 5
6 The 3D Video Processing Chain Head Tracker Foreground/ Background Segmentation Tracking of Hands Rectification MPEG-4 Compositor Depth Estimation 3D View Combining MPEG-4 Encode Transmission 3D Warping of Video Objects MPEG-4 Decode Analysis Synthesis Dr. Oliver Schreer 6
7 Rectification Rectification Rectification Dr. Oliver Schreer 7
8 The 3D Video Processing Chain Head Tracker Foreground/ Background Segmentation Tracking of Hands Rectification MPEG-4 Compositor Depth Estimation 3D View Combining MPEG-4 Encode Transmission 3D Warping of Video Objects MPEG-4 Decode Analysis Synthesis Dr. Oliver Schreer 8
9 Outline The concept of real-time immersive videoconferencing A quick tour into stereo image processing Hybrid-recursive disparity estimation Dr. Oliver Schreer 9
10 General Task Correct assignement of features in both views, which are images of the same 3D point Dr. Oliver Schreer 10
11 The Correspondence Problem different field of view of two cameras low textured regions occlusions periodical patterns Dr. Oliver Schreer 11
12 Similarity Constraints in Stereo epipolar constraint M 1 T T T m2 Fm1 = 0 m1 l1 = 0, m2l2 = 0 M 2 l 1 m 1 e 1 e 2 m 22 m 21 C 1 C 2 search directions I 1 I 2 uniqueness constraint smootheness constraint Dr. Oliver Schreer 12
13 Similarity Constraints in Stereo ordering constraint I 1 I 2 limited search range M max δ=u 1, da u 2 =0 M min m max m 2 m min C 1 u 1,min u 1,max C 2 u 2 =0 Dr. Oliver Schreer 13
14 Pixel-based Block-Matching v I 1 I 2 δ(u,v) u 1 u 2 u 1 Dr. Oliver Schreer 14
15 Block-Matching with Resolution Pyramid left right Dr. Oliver Schreer 15
16 Block-Matching Fine-to-Fine left right Dr. Oliver Schreer 16
17 Two Stage Block-Matching global stage L R local stage L R Dr. Oliver Schreer 17
18 Outline The concept of real-time immersive videoconferencing A quick tour into stereo processing Hybrid-recursive disparity estimation Dr. Oliver Schreer 18
19 General Concept of Depth Estimator (DE) Original Images Disparity Maps L R Fast Depth Estimation Post- Processing L R R L 1st Step: Fast, Simple and Robust Approach hybrid block- and pixel recursive matching algorithm accept that it produces some mismatches Dr. Oliver Schreer 19
20 General Concept of Depth Estimator (DE) Original Images Disparity Maps L R Fast Depth Estimation Post- Processing L R R L 2nd Step: Detection and Masking of Mismatches efficient consistency check between L R and R L match substitute inconsistent disparities by inter/extrapolation Dr. Oliver Schreer 20
21 Hybride Recursive Disparity Estimation (HRM) Objectives Spatial and temporal consistent disparities Real-time processing on full TV resolution CCIR 601 ( 720 x 576 pel) candidates Dr. Oliver Schreer 21
22 Hybride Recursive Disparity Estimation (HRM) Objectives Spatial and temporal consistent disparities Real-time processing on full TV resolution CCIR 601 ( 720 x 576 pel) pixel recursion Dr. Oliver Schreer 22
23 Thank you for your attention in the first part! Coffee break Dr. Oliver Schreer 23
24 Structure of Hybride-Recursive Matching pixelrecursion update vector choice of best disparity start vektor disparity vector left image right image blockrecursion 3 candidates disparity memory Dr. Oliver Schreer 24
25 Block-Recursion test of 3 candidate vectors vertical temporal horizontal previous image current Image no search area around candidates choice of best candidate via DBD (displaced block difference) Dr. Oliver Schreer 25
26 Interleaved Meander Scan even frame odd frame Dr. Oliver Schreer 26
27 Structure of Hybride-Recursive Matching pixelrecursion update vector choice of best disparity start vektor disparity vector left image right image blockrecursion 3 candidates disparity memory Dr. Oliver Schreer 27
28 Pixel-Recursion Initialisation of two pixel-recursive processes Computation of update-vectors by using optical flow principle Choice of best update-vector by DPD (displaced pixel difference) initial vector d incremental update update vector d start vector Dr. Oliver Schreer 28
29 Structure of Hybride-Recursive Matching pixelrecursion update vektor choice of best disparity start vector disparity vector left image right image blockrecursion 3 candidates disparity memory Dr. Oliver Schreer 29
30 General Concept of Depth Estimator (DE) Original Images Disparity Maps L R Fast Disparity Estimation Post- Processing L R R L Dr. Oliver Schreer 30
31 Consistency Check Right-to-left vector Left-to-right vector L->R R->L occluded region Dr. Oliver Schreer 31
32 Simple Post-Processing L->R R->L L->R R->L horizontal linear interpolation Dr. Oliver Schreer 32
33 Adaptive Post-Processing Homogenous areas: linear interpolation Discontinuities: segment-based hole-filling homogenous areas L->R R->L discontinuities Dr. Oliver Schreer 33
34 Adaptive Post-Processing L->R R->L L->R R->L segment-based hole filling Dr. Oliver Schreer 34
35 Segment-based Interpolation homogenous areas discontinuities Homogenous regions Discontinuities bilinear interpolation segment-based approach Dr. Oliver Schreer 35
36 Experimental Results without consistency check & post-processing L->R simple post-processing L->R segment-based post-processing L->R R->L R->L R->L Dr. Oliver Schreer 36
37 Experimental Results without consistency check & post-processing Dr. Oliver Schreer 37
38 Experimental Results simple post-processing Dr. Oliver Schreer 38
39 Experimental Results segment-based post-processing Dr. Oliver Schreer 39
40 Experimental Results segment-based post-processing with perfect mask Dr. Oliver Schreer 40
41 The 3D Video Processing Chain Head Tracker Foreground/ Background Segmentation Tracking of Hands Rectification MPEG-4 Compositor Depth Estimation 3D View Combining MPEG-4 Encode Transmission 3D Warping of Video Objects MPEG-4 Decode Analysis Synthesis Dr. Oliver Schreer 41
42 3D View Combining Texture & Depth at Middle View Position Dr. Oliver Schreer 42
43 The 3D Video Processing Chain Head Tracker Foreground/ Background Segmentation Tracking of Hands Rectification MPEG-4 Compositor Depth Estimation 3D View Combining MPEG-4 Encode Transmission 3D Warping of Video Objects MPEG-4 Decode Analysis Synthesis Dr. Oliver Schreer 43
44 Final 3D Warp m M w m view ψ virtual view ψ m view ψ Dr. Oliver Schreer 44
45 Experimental Results of View Synthesis without post-processing simple post-processing segment-based post-processing Dr. Oliver Schreer 45
46 Results of View Synthesis Fixed view - motion Dynamic view still image Dr. Oliver Schreer 46
47 Comparison to Other Approaches Visual quality Pyramid Two-stage BM HRM Processing speed at TV-resolution ( P IV 1,7 GHz ) 40 4x4 Grid in ms Dr. Oliver Schreer 47
48 Simple Extension Towards Tele-Collaboration Application at Screen Integrated Tools In cooperation with France Telecom Dr. Oliver Schreer 48
49 Extension Towards Scalable System Architecture CAVE-like Terminal CAVE-like Terminal Basic Module Corner Terminal Multi-User Terminal Corner Terminal Scene & Conference Server Content Management Dr. Oliver Schreer 49
50 Conclusion Real-time capable algorithms available in the whole 3D video processing chain segmentation tracking disparity analysis on full TV resolution MPEG-4/H.26x encoding/decoding view synthesis Architecture concept without dedicated hardware Dr. Oliver Schreer 50
51 Acknowledgement I like to thank... Peter Kauff (head of my group) Khalid Elazouzi (segmentation & tracking of hands) Nicole Atzpadin (disparity analysis) Christoph Fehn (immersive TV) Ingo Feldmann (shadow detection) Ralf Tanger (foreground/background segmentation) Contact Oliver.Schreer@hhi.fhg.de web: ip.hhi.de Dr. Oliver Schreer 51
52 ... thank you for your attention END Dr. Oliver Schreer 52
NEW CONCEPT FOR JOINT DISPARITY ESTIMATION AND SEGMENTATION FOR REAL-TIME VIDEO PROCESSING
NEW CONCEPT FOR JOINT DISPARITY ESTIMATION AND SEGMENTATION FOR REAL-TIME VIDEO PROCESSING Nicole Atzpadin 1, Serap Askar, Peter Kauff, Oliver Schreer Fraunhofer Institut für Nachrichtentechnik, Heinrich-Hertz-Institut,
More informationREAL-TIME DISPARITY MAPS FOR IMMERSIVE 3-D TELECONFERENCING BY HYBRID RECURSIVE MATCHING AND CENSUS TRANSFORM
REAL-TIME DISPARITY MAPS FOR IMMERSIVE 3-D TELECONFERENCING BY HYBRID RECURSIVE MATCHING AND CENSUS TRANSFORM E. Trucco 1 K Plakas 1 Nicole Brandenburg 2 Peter Kauff 2 Michael Karl 2 Oliver Schreer 2 1
More information3DPRESENCE A SYSTEM CONCEPT FOR MULTI-USER AND MULTI-PARTY IMMERSIVE 3D VIDEOCONFERENCING
3DPRESENCE A SYSTEM CONCEPT FOR MULTI-USER AND MULTI-PARTY IMMERSIVE 3D VIDEOCONFERENCING O. Schreer, I. Feldmann, N. Atzpadin, P. Eisert, P. Kauff, H.J.W. Belt* Fraunhofer Institute for Telecommunications/Heinrich-Hertz
More informationINTERACTIVE VIRTUAL VIEW VIDEO FOR IMMERSIVE TV APPLICATIONS
INTERACTIVE VIRTUAL VIEW VIDEO FOR IMMERSIVE TV APPLICATIONS C. Fehn, P. Kauff, O. Schreer and R. Schäfer Heinrich-Hertz-Institut, Germany ABSTRACT This paper presents an evolutionary approach on Immersive
More informationINTERNATIONAL ORGANISATION FOR STANDARISATION ORGANISATION INTERNATIONALE DE NORMALISATION ISO/IEC JTC1/SC29/WG11 CODING OF MOVING PICTURES AND AUDIO
INTERNATIONAL ORGANISATION FOR STANDARISATION ORGANISATION INTERNATIONALE DE NORMALISATION ISO/IEC JTC1/SC29/WG11 CODING OF MOVING PICTURES AND AUDIO ISO/IEC JTC1/SC29/WG11 MPEG/M15672 July 2008, Hannover,
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 14 130307 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Stereo Dense Motion Estimation Translational
More informationThe Virtual Meeting Room
Contact Details of Presenting Authors Stefan Rauthenberg (rauthenberg@hhi.de), Peter Kauff (kauff@hhi.de) Tel: +49-30-31002 266, +49-30-31002 615 Fax: +49-30-3927200 Summation Brief explaination of the
More informationRENDERING AND ANALYSIS OF FACES USING MULTIPLE IMAGES WITH 3D GEOMETRY. Peter Eisert and Jürgen Rurainsky
RENDERING AND ANALYSIS OF FACES USING MULTIPLE IMAGES WITH 3D GEOMETRY Peter Eisert and Jürgen Rurainsky Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institute Image Processing Department
More informationRealtime View Adaptation of Video Objects in 3-Dimensional Virtual Environments
Contact Details of Presenting Author Edward Cooke (cooke@hhi.de) Tel: +49-30-31002 613 Fax: +49-30-3927200 Summation Abstract o Examination of the representation of time-critical, arbitrary-shaped, video
More informationStep-by-Step Model Buidling
Step-by-Step Model Buidling Review Feature selection Feature selection Feature correspondence Camera Calibration Euclidean Reconstruction Landing Augmented Reality Vision Based Control Sparse Structure
More information3D Video Formats and Coding Standards
3D Video Formats and Coding Standards Fraunhofer Institute for Telecommunications Heinrich-Hertz-Institut Berlin Einsteinufer 37 10587 Berlin Germany +49 30 310 02 0 info@hhi.fraunhofer.de http://www.hhi.fraunhofer.de
More informationAutomatic 2D-to-3D Video Conversion Techniques for 3DTV
Automatic 2D-to-3D Video Conversion Techniques for 3DTV Dr. Lai-Man Po Email: eelmpo@cityu.edu.hk Department of Electronic Engineering City University of Hong Kong Date: 13 April 2010 Content Why 2D-to-3D
More informationCross Layer Protocol Design
Cross Layer Protocol Design Radio Communication III The layered world of protocols Video Compression for Mobile Communication » Image formats» Pixel representation Overview» Still image compression Introduction»
More informationFlow Estimation. Min Bai. February 8, University of Toronto. Min Bai (UofT) Flow Estimation February 8, / 47
Flow Estimation Min Bai University of Toronto February 8, 2016 Min Bai (UofT) Flow Estimation February 8, 2016 1 / 47 Outline Optical Flow - Continued Min Bai (UofT) Flow Estimation February 8, 2016 2
More informationCHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION
CHAPTER 3 DISPARITY AND DEPTH MAP COMPUTATION In this chapter we will discuss the process of disparity computation. It plays an important role in our caricature system because all 3D coordinates of nodes
More informationReference Stream Selection for Multiple Depth Stream Encoding
Reference Stream Selection for Multiple Depth Stream Encoding Sang-Uok Kum Ketan Mayer-Patel kumsu@cs.unc.edu kmp@cs.unc.edu University of North Carolina at Chapel Hill CB #3175, Sitterson Hall Chapel
More informationDepth. Common Classification Tasks. Example: AlexNet. Another Example: Inception. Another Example: Inception. Depth
Common Classification Tasks Recognition of individual objects/faces Analyze object-specific features (e.g., key points) Train with images from different viewing angles Recognition of object classes Analyze
More informationNew Techniques for Improved Video Coding
New Techniques for Improved Video Coding Thomas Wiegand Fraunhofer Institute for Telecommunications Heinrich Hertz Institute Berlin, Germany wiegand@hhi.de Outline Inter-frame Encoder Optimization Texture
More informationMultiview Image Compression using Algebraic Constraints
Multiview Image Compression using Algebraic Constraints Chaitanya Kamisetty and C. V. Jawahar Centre for Visual Information Technology, International Institute of Information Technology, Hyderabad, INDIA-500019
More informationKey-Words: - Free viewpoint video, view generation, block based disparity map, disparity refinement, rayspace.
New View Generation Method for Free-Viewpoint Video System GANGYI JIANG*, LIANGZHONG FAN, MEI YU AND FENG SHAO Faculty of Information Science and Engineering Ningbo University 315211 Ningbo CHINA jianggangyi@126.com
More informationEpipolar Geometry CSE P576. Dr. Matthew Brown
Epipolar Geometry CSE P576 Dr. Matthew Brown Epipolar Geometry Epipolar Lines, Plane Constraint Fundamental Matrix, Linear solution + RANSAC Applications: Structure from Motion, Stereo [ Szeliski 11] 2
More informationModule 7 VIDEO CODING AND MOTION ESTIMATION
Module 7 VIDEO CODING AND MOTION ESTIMATION Lesson 20 Basic Building Blocks & Temporal Redundancy Instructional Objectives At the end of this lesson, the students should be able to: 1. Name at least five
More informationColour Segmentation-based Computation of Dense Optical Flow with Application to Video Object Segmentation
ÖGAI Journal 24/1 11 Colour Segmentation-based Computation of Dense Optical Flow with Application to Video Object Segmentation Michael Bleyer, Margrit Gelautz, Christoph Rhemann Vienna University of Technology
More informationCorrespondence and Stereopsis. Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri]
Correspondence and Stereopsis Original notes by W. Correa. Figures from [Forsyth & Ponce] and [Trucco & Verri] Introduction Disparity: Informally: difference between two pictures Allows us to gain a strong
More informationNeue Verfahren der Bildverarbeitung auch zur Erfassung von Schäden in Abwasserkanälen?
Neue Verfahren der Bildverarbeitung auch zur Erfassung von Schäden in Abwasserkanälen? Fraunhofer HHI 13.07.2017 1 Fraunhofer-Gesellschaft Fraunhofer is Europe s largest organization for applied research.
More informationSpatio-Temporal Stereo Disparity Integration
Spatio-Temporal Stereo Disparity Integration Sandino Morales and Reinhard Klette The.enpeda.. Project, The University of Auckland Tamaki Innovation Campus, Auckland, New Zealand pmor085@aucklanduni.ac.nz
More informationRange Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation
Obviously, this is a very slow process and not suitable for dynamic scenes. To speed things up, we can use a laser that projects a vertical line of light onto the scene. This laser rotates around its vertical
More informationCS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
More informationStereo. Outline. Multiple views 3/29/2017. Thurs Mar 30 Kristen Grauman UT Austin. Multi-view geometry, matching, invariant features, stereo vision
Stereo Thurs Mar 30 Kristen Grauman UT Austin Outline Last time: Human stereopsis Epipolar geometry and the epipolar constraint Case example with parallel optical axes General case with calibrated cameras
More informationStereo vision. Many slides adapted from Steve Seitz
Stereo vision Many slides adapted from Steve Seitz What is stereo vision? Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape What is
More informationStereo Vision Computer Vision (Kris Kitani) Carnegie Mellon University
Stereo Vision 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University What s different between these two images? Objects that are close move more or less? The amount of horizontal movement is
More informationFundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision
Fundamentals of Stereo Vision Michael Bleyer LVA Stereo Vision What Happened Last Time? Human 3D perception (3D cinema) Computational stereo Intuitive explanation of what is meant by disparity Stereo matching
More informationHybrid Rendering for Collaborative, Immersive Virtual Environments
Hybrid Rendering for Collaborative, Immersive Virtual Environments Stephan Würmlin wuermlin@inf.ethz.ch Outline! Rendering techniques GBR, IBR and HR! From images to models! Novel view generation! Putting
More informationIntra-Stream Encoding for Multiple Depth Streams
Intra-Stream Encoding for Multiple Depth Streams Sang-Uok Kum kumsu@cs.unc.edu University of North Carolina at Chapel Hill CB #3175, Sitterson Hall Chapel Hill, NC 27599 USA Ketan Mayer-Patel kmp@cs.unc.edu
More informationEE795: Computer Vision and Intelligent Systems
EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 12 130228 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Panoramas, Mosaics, Stitching Two View Geometry
More informationDense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Today: dense 3D reconstruction The matching problem
More informationView Synthesis Prediction for Rate-Overhead Reduction in FTV
MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com View Synthesis Prediction for Rate-Overhead Reduction in FTV Sehoon Yea, Anthony Vetro TR2008-016 June 2008 Abstract This paper proposes the
More informationInline Computational Imaging: Single Sensor Technology for Simultaneous 2D/3D High Definition Inline Inspection
Inline Computational Imaging: Single Sensor Technology for Simultaneous 2D/3D High Definition Inline Inspection Svorad Štolc et al. svorad.stolc@ait.ac.at AIT Austrian Institute of Technology GmbH Center
More information5LSH0 Advanced Topics Video & Analysis
1 Multiview 3D video / Outline 2 Advanced Topics Multimedia Video (5LSH0), Module 02 3D Geometry, 3D Multiview Video Coding & Rendering Peter H.N. de With, Sveta Zinger & Y. Morvan ( p.h.n.de.with@tue.nl
More informationAugmented and Mixed Reality
Augmented and Mixed Reality Uma Mudenagudi Dept. of Computer Science and Engineering, Indian Institute of Technology Delhi Outline Introduction to Augmented Reality(AR) and Mixed Reality(MR) A Typical
More informationDEPTH IMAGE BASED RENDERING WITH ADVANCED TEXTURE SYNTHESIS. P. Ndjiki-Nya, M. Köppel, D. Doshkov, H. Lakshman, P. Merkle, K. Müller, and T.
DEPTH IMAGE BASED RENDERING WITH ADVANCED TEXTURE SYNTHESIS P. Ndjiki-Nya, M. Köppel, D. Doshkov, H. Lakshman, P. Merkle, K. Müller, and T. Wiegand Fraunhofer Institut for Telecommunications, Heinrich-Hertz-Institut
More informationPerceptual Grouping from Motion Cues Using Tensor Voting
Perceptual Grouping from Motion Cues Using Tensor Voting 1. Research Team Project Leader: Graduate Students: Prof. Gérard Medioni, Computer Science Mircea Nicolescu, Changki Min 2. Statement of Project
More informationChapter 11.3 MPEG-2. MPEG-2: For higher quality video at a bit-rate of more than 4 Mbps Defined seven profiles aimed at different applications:
Chapter 11.3 MPEG-2 MPEG-2: For higher quality video at a bit-rate of more than 4 Mbps Defined seven profiles aimed at different applications: Simple, Main, SNR scalable, Spatially scalable, High, 4:2:2,
More informationStereo. 11/02/2012 CS129, Brown James Hays. Slides by Kristen Grauman
Stereo 11/02/2012 CS129, Brown James Hays Slides by Kristen Grauman Multiple views Multi-view geometry, matching, invariant features, stereo vision Lowe Hartley and Zisserman Why multiple views? Structure
More informationOverview. Related Work Tensor Voting in 2-D Tensor Voting in 3-D Tensor Voting in N-D Application to Vision Problems Stereo Visual Motion
Overview Related Work Tensor Voting in 2-D Tensor Voting in 3-D Tensor Voting in N-D Application to Vision Problems Stereo Visual Motion Binary-Space-Partitioned Images 3-D Surface Extraction from Medical
More informationINTEGRATING COMPUTER VISION SENSOR INNOVATIONS INTO MOBILE DEVICES. Eli Savransky Principal Architect - CTO Office Mobile BU NVIDIA corp.
INTEGRATING COMPUTER VISION SENSOR INNOVATIONS INTO MOBILE DEVICES Eli Savransky Principal Architect - CTO Office Mobile BU NVIDIA corp. Computer Vision in Mobile Tegra K1 It s time! AGENDA Use cases categories
More informationEECS 442 Computer vision. Stereo systems. Stereo vision Rectification Correspondence problem Active stereo vision systems
EECS 442 Computer vision Stereo systems Stereo vision Rectification Correspondence problem Active stereo vision systems Reading: [HZ] Chapter: 11 [FP] Chapter: 11 Stereo vision P p p O 1 O 2 Goal: estimate
More informationWhat have we leaned so far?
What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging What have we learned so far? Image Filtering Image Warping Camera Projection Model Project 2: Panoramic
More informationLecture 14: Basic Multi-View Geometry
Lecture 14: Basic Multi-View Geometry Stereo If I needed to find out how far point is away from me, I could use triangulation and two views scene point image plane optical center (Graphic from Khurram
More informationMultiple View Geometry
Multiple View Geometry Martin Quinn with a lot of slides stolen from Steve Seitz and Jianbo Shi 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 Our Goal The Plenoptic Function P(θ,φ,λ,t,V
More information732 IEEE TRANSACTIONS ON BROADCASTING, VOL. 54, NO. 4, DECEMBER /$ IEEE
732 IEEE TRANSACTIONS ON BROADCASTING, VOL. 54, NO. 4, DECEMBER 2008 Generation of ROI Enhanced Depth Maps Using Stereoscopic Cameras and a Depth Camera Sung-Yeol Kim, Student Member, IEEE, Eun-Kyung Lee,
More informationLecture 5: Error Resilience & Scalability
Lecture 5: Error Resilience & Scalability Dr Reji Mathew A/Prof. Jian Zhang NICTA & CSE UNSW COMP9519 Multimedia Systems S 010 jzhang@cse.unsw.edu.au Outline Error Resilience Scalability Including slides
More informationIntroduction to Computer Vision
Introduction to Computer Vision Michael J. Black Oct 2009 Motion estimation Goals Motion estimation Affine flow Optimization Large motions Why affine? Monday dense, smooth motion and regularization. Robust
More informationThree-Dimensional Sensors Lecture 2: Projected-Light Depth Cameras
Three-Dimensional Sensors Lecture 2: Projected-Light Depth Cameras Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inria.fr http://perception.inrialpes.fr/ Outline The geometry of active stereo.
More informationMotion Analysis. Motion analysis. Now we will talk about. Differential Motion Analysis. Motion analysis. Difference Pictures
Now we will talk about Motion Analysis Motion analysis Motion analysis is dealing with three main groups of motionrelated problems: Motion detection Moving object detection and location. Derivation of
More informationIMA Preprint Series # 2016
VIDEO INPAINTING OF OCCLUDING AND OCCLUDED OBJECTS By Kedar A. Patwardhan Guillermo Sapiro and Marcelo Bertalmio IMA Preprint Series # 2016 ( January 2005 ) INSTITUTE FOR MATHEMATICS AND ITS APPLICATIONS
More informationCognitive Algorithms for 3D Video Acquisition and Transmission
Department of Information Engineering, University of Padova Cognitive Algorithms for 3D Video Acquisition and Transmission Simone Milani Politecnico di Milano / University of Padova milani@elet.polimi.it,
More informationCS 4495 Computer Vision A. Bobick. Motion and Optic Flow. Stereo Matching
Stereo Matching Fundamental matrix Let p be a point in left image, p in right image l l Epipolar relation p maps to epipolar line l p maps to epipolar line l p p Epipolar mapping described by a 3x3 matrix
More informationColor Segmentation Based Depth Filtering
Color Segmentation Based Depth Filtering Dipl.-Math. Michael Schmeing Prof. Dr. Xiaoyi Jiang Institute of Computer Science University of Münster, Germany Overview Depth Filtering Our Approach Results Qualitative
More informationGeorgios Tziritas Computer Science Department
New Video Coding standards MPEG-4, HEVC Georgios Tziritas Computer Science Department http://www.csd.uoc.gr/~tziritas 1 MPEG-4 : introduction Motion Picture Expert Group Publication 1998 (Intern. Standardization
More informationNotes 9: Optical Flow
Course 049064: Variational Methods in Image Processing Notes 9: Optical Flow Guy Gilboa 1 Basic Model 1.1 Background Optical flow is a fundamental problem in computer vision. The general goal is to find
More informationCONTENTS. High-Accuracy Stereo Depth Maps Using Structured Light. Yeojin Yoon
[Paper Seminar 7] CVPR2003, Vol.1, pp.195-202 High-Accuracy Stereo Depth Maps Using Structured Light Daniel Scharstein Middlebury College Richard Szeliski Microsoft Research 2012. 05. 30. Yeojin Yoon Introduction
More informationA Novel Filling Disocclusion Method Based on Background Extraction. in Depth-Image-Based-Rendering
A Novel Filling Disocclusion Method Based on Background Extraction in Depth-Image-Based-Rendering Zhenguo Lu,Yuesheng Zhu,Jian Chen ShenZhen Graduate School,Peking University,China Email:zglu@sz.pku.edu.cn,zhuys@pkusz.edu.cn
More informationWARP BASED NEAR REGULAR TEXTURE ANALYSIS FOR IMAGE BASED TEXTURE OVERLAY
WARP BASED NEAR REGULAR TEXTURE ANALYSIS FOR IMAGE BASED TEXTURE OVERLAY Anna Hilsmann 1,2 David C. Schneider 1,2 Peter Eisert 1,2 1 Fraunhofer Heinrich Hertz Institute Berlin, Germany 2 Humboldt University
More informationDense 3D Reconstruction. Christiano Gava
Dense 3D Reconstruction Christiano Gava christiano.gava@dfki.de Outline Previous lecture: structure and motion II Structure and motion loop Triangulation Wide baseline matching (SIFT) Today: dense 3D reconstruction
More informationShadows. COMP 575/770 Spring 2013
Shadows COMP 575/770 Spring 2013 Shadows in Ray Tracing Shadows are important for realism Basic idea: figure out whether a point on an object is illuminated by a light source Easy for ray tracers Just
More informationVisual Tracking. Image Processing Laboratory Dipartimento di Matematica e Informatica Università degli studi di Catania.
Image Processing Laboratory Dipartimento di Matematica e Informatica Università degli studi di Catania 1 What is visual tracking? estimation of the target location over time 2 applications Six main areas:
More informationFocus on visual rendering quality through content-based depth map coding
Focus on visual rendering quality through content-based depth map coding Emilie Bosc, Muriel Pressigout, Luce Morin To cite this version: Emilie Bosc, Muriel Pressigout, Luce Morin. Focus on visual rendering
More informationMultimedia Systems Video II (Video Coding) Mahdi Amiri April 2012 Sharif University of Technology
Course Presentation Multimedia Systems Video II (Video Coding) Mahdi Amiri April 2012 Sharif University of Technology Video Coding Correlation in Video Sequence Spatial correlation Similar pixels seem
More information3-D TERRAIN RECONSTRUCTION WITH AERIAL PHOTOGRAPHY
3-D TERRAIN RECONSTRUCTION WITH AERIAL PHOTOGRAPHY Bin-Yih Juang ( 莊斌鎰 ) 1, and Chiou-Shann Fuh ( 傅楸善 ) 3 1 Ph. D candidate o Dept. o Mechanical Engineering National Taiwan University, Taipei, Taiwan Instructor
More informationEpipolar Geometry and Stereo Vision
Epipolar Geometry and Stereo Vision Computer Vision Jia-Bin Huang, Virginia Tech Many slides from S. Seitz and D. Hoiem Last class: Image Stitching Two images with rotation/zoom but no translation. X x
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationMatching. Compare region of image to region of image. Today, simplest kind of matching. Intensities similar.
Matching Compare region of image to region of image. We talked about this for stereo. Important for motion. Epipolar constraint unknown. But motion small. Recognition Find object in image. Recognize object.
More informationStereo matching. Francesco Isgrò. 3D Reconstruction and Stereo p.1/21
Stereo matching Francesco Isgrò 3D Reconstruction and Stereo p.1/21 Structure of a stereo vision system Extract interesting point from each image Determine a set of matching points Compute the fundamental
More informationVisual Tracking. Antonino Furnari. Image Processing Lab Dipartimento di Matematica e Informatica Università degli Studi di Catania
Visual Tracking Antonino Furnari Image Processing Lab Dipartimento di Matematica e Informatica Università degli Studi di Catania furnari@dmi.unict.it 11 giugno 2015 What is visual tracking? estimation
More informationDevelopment of a New MPEG Standard for Advanced 3D Video Applications
MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Development of a New MPEG Standard for Advanced 3D Video Applications Aljoscha Smolic, Karsten Mueller, Philipp Merkle, Anthony Vetro TR2009-068
More informationDATA FORMAT AND CODING FOR FREE VIEWPOINT VIDEO
DATA FORMAT AND CODING FOR FREE VIEWPOINT VIDEO P. Kauff, A. Smolic, P. Eisert, C. Fehn. K. Müller, R. Schäfer Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institut (FhG/HHI), Berlin, Germany
More informationStereo Wrap + Motion. Computer Vision I. CSE252A Lecture 17
Stereo Wrap + Motion CSE252A Lecture 17 Some Issues Ambiguity Window size Window shape Lighting Half occluded regions Problem of Occlusion Stereo Constraints CONSTRAINT BRIEF DESCRIPTION 1-D Epipolar Search
More informationMotion Estimation for Video Coding Standards
Motion Estimation for Video Coding Standards Prof. Ja-Ling Wu Department of Computer Science and Information Engineering National Taiwan University Introduction of Motion Estimation The goal of video compression
More informationFine grain scalable video coding using 3D wavelets and active meshes
Fine grain scalable video coding using 3D wavelets and active meshes Nathalie Cammas a,stéphane Pateux b a France Telecom RD,4 rue du Clos Courtel, Cesson-Sévigné, France b IRISA, Campus de Beaulieu, Rennes,
More informationCS 787: Assignment 4, Stereo Vision: Block Matching and Dynamic Programming Due: 12:00noon, Fri. Mar. 30, 2007.
CS 787: Assignment 4, Stereo Vision: Block Matching and Dynamic Programming Due: 12:00noon, Fri. Mar. 30, 2007. In this assignment you will implement and test some simple stereo algorithms discussed in
More informationMulti-directional Hole Filling Method for Virtual View Synthesis
DOI 10.1007/s11265-015-1069-2 Multi-directional Hole Filling Method for Virtual View Synthesis Ji-Hun Mun 1 & Yo-Sung Ho 1 Received: 30 March 2015 /Revised: 14 October 2015 /Accepted: 19 October 2015 #
More informationMultiple View Geometry
Multiple View Geometry CS 6320, Spring 2013 Guest Lecture Marcel Prastawa adapted from Pollefeys, Shah, and Zisserman Single view computer vision Projective actions of cameras Camera callibration Photometric
More informationA multiresolutional region based segmentation scheme for stereoscopic image compression
Published in SPIE Vol.2419 - Digital Video compression - Algorithms and technologies in 1995 A multiresolutional region based segmentation scheme for stereoscopic image compression Sriram Sethuraman, M.
More informationReminder: Lecture 20: The Eight-Point Algorithm. Essential/Fundamental Matrix. E/F Matrix Summary. Computing F. Computing F from Point Matches
Reminder: Lecture 20: The Eight-Point Algorithm F = -0.00310695-0.0025646 2.96584-0.028094-0.00771621 56.3813 13.1905-29.2007-9999.79 Readings T&V 7.3 and 7.4 Essential/Fundamental Matrix E/F Matrix Summary
More informationImage processing and features
Image processing and features Gabriele Bleser gabriele.bleser@dfki.de Thanks to Harald Wuest, Folker Wientapper and Marc Pollefeys Introduction Previous lectures: geometry Pose estimation Epipolar geometry
More informationFast Lighting Independent Background Subtraction
Fast Lighting Independent Background Subtraction Yuri Ivanov Aaron Bobick John Liu [yivanov bobick johnliu]@media.mit.edu MIT Media Laboratory February 2, 2001 Abstract This paper describes a new method
More informationFinal Exam Study Guide CSE/EE 486 Fall 2007
Final Exam Study Guide CSE/EE 486 Fall 2007 Lecture 2 Intensity Sufaces and Gradients Image visualized as surface. Terrain concepts. Gradient of functions in 1D and 2D Numerical derivatives. Taylor series.
More informationUsing temporal seeding to constrain the disparity search range in stereo matching
Using temporal seeding to constrain the disparity search range in stereo matching Thulani Ndhlovu Mobile Intelligent Autonomous Systems CSIR South Africa Email: tndhlovu@csir.co.za Fred Nicolls Department
More informationA virtual tour of free viewpoint rendering
A virtual tour of free viewpoint rendering Cédric Verleysen ICTEAM institute, Université catholique de Louvain, Belgium cedric.verleysen@uclouvain.be Organization of the presentation Context Acquisition
More informationVideo Coding Standards
Based on: Y. Wang, J. Ostermann, and Y.-Q. Zhang, Video Processing and Communications, Prentice Hall, 2002. Video Coding Standards Yao Wang Polytechnic University, Brooklyn, NY11201 http://eeweb.poly.edu/~yao
More informationOptical Flow Estimation with CUDA. Mikhail Smirnov
Optical Flow Estimation with CUDA Mikhail Smirnov msmirnov@nvidia.com Document Change History Version Date Responsible Reason for Change Mikhail Smirnov Initial release Abstract Optical flow is the apparent
More informationLecture 14: Computer Vision
CS/b: Artificial Intelligence II Prof. Olga Veksler Lecture : Computer Vision D shape from Images Stereo Reconstruction Many Slides are from Steve Seitz (UW), S. Narasimhan Outline Cues for D shape perception
More informationUpcoming Video Standards. Madhukar Budagavi, Ph.D. DSPS R&D Center, Dallas Texas Instruments Inc.
Upcoming Video Standards Madhukar Budagavi, Ph.D. DSPS R&D Center, Dallas Texas Instruments Inc. Outline Brief history of Video Coding standards Scalable Video Coding (SVC) standard Multiview Video Coding
More informationLecture 10 Dense 3D Reconstruction
Institute of Informatics Institute of Neuroinformatics Lecture 10 Dense 3D Reconstruction Davide Scaramuzza 1 REMODE: Probabilistic, Monocular Dense Reconstruction in Real Time M. Pizzoli, C. Forster,
More informationPublic Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923
Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923 Teesta suspension bridge-darjeeling, India Mark Twain at Pool Table", no date, UCR Museum of Photography Woman getting eye exam during
More informationWeek 14. Video Compression. Ref: Fundamentals of Multimedia
Week 14 Video Compression Ref: Fundamentals of Multimedia Last lecture review Prediction from the previous frame is called forward prediction Prediction from the next frame is called forward prediction
More informationLecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15
Lecture 6 Stereo Systems Multi- view geometry Professor Silvio Savarese Computational Vision and Geometry Lab Silvio Savarese Lecture 6-24-Jan-15 Lecture 6 Stereo Systems Multi- view geometry Stereo systems
More informationScalable Video Coding
Introduction to Multimedia Computing Scalable Video Coding 1 Topics Video On Demand Requirements Video Transcoding Scalable Video Coding Spatial Scalability Temporal Scalability Signal to Noise Scalability
More informationVIDEO COMPRESSION STANDARDS
VIDEO COMPRESSION STANDARDS Family of standards: the evolution of the coding model state of the art (and implementation technology support): H.261: videoconference x64 (1988) MPEG-1: CD storage (up to
More information