Computational Photography and Capture: (Re)Coloring. Gabriel Brostow & Tim Weyrich TA: Frederic Besse

Size: px
Start display at page:

Download "Computational Photography and Capture: (Re)Coloring. Gabriel Brostow & Tim Weyrich TA: Frederic Besse"

Transcription

1 Computational Photography and Capture: (Re)Coloring Gabriel Brostow & Tim Weyrich TA: Frederic Besse

2 Week Date Topic Hours 1 12-Jan Introduction to Computational Photography and Capture Jan Intro + More on Cameras, Sensors and Color Jan No lecture! (Go capture bracketed photos?) Jan Blending, Compositing, Poisson Editing Jan Time-Lapse Jan Carving, Warping, and Morphing Feb High-Dynamic-Range Imaging and Tone Mapping Feb Hybrid Images, Flash and Multi-Flash Photography Feb Colorization and Color Transfer Feb Image Inpainting and Texture Synthesis Feb Video Based Rendering of Scenes I Feb Video Based Rendering of Scenes II Mar Video Texture Synthesis Mar Video Sprites Mar Deblurring/Dehazing and Coded Aperture Imaging Mar Image-based Rendering Mar Motion Capture guest lecture by Doug Griffin Mar Capturing Geometry with Active Lighting Mar Intrinsic Images Mar Dual Photography and Reflectance Analysis 2

3 Today s Lecture Colorization using Optimization Levin, Lischinski, Weiss, Siggraph 2004 Color Transfer Between Images Reinhard, Ashikhmin, Gooch, Shirley, CG&A 2001 N-Dimensional Probability Density Function Transfer and its Application to Colour Transfer Pitié, Kokaram, Dahyot, ICCV 2005 (see also Interactive Local Adjustment of Tonal Values 06)

4 Note: Using Anat Levin s slides for structure, with the other papers appearing throughout Colorization Using Optimization Anat Levin Dani Lischinski Yair Weiss SIGGRAPH 2004 Project web page

5 Colorization Colorization: a computer-assisted process of adding color to a monochrome image or movie. (Invented by Wilson Markle, 1970)

6 Motivation Colorizing black and white movies and TV shows Earl Glick (Chairman, Hal Roach Studios), 1984: You couldn't make Wyatt Earp today for $1 million an episode. But for $50,000 a segment, you can turn it into color and have a brand new series with no residuals to pay Hugh O'Brien as Wyatt Earp, 1957

7 Motivation Colorizing black and white movies and TV shows Recoloring color images for special effects

8 Motivation

9 Typical Colorization Process Images from: Yet Another Colorization Tutorial

10 Typical Colorization Process Delineate region boundary Images from: Yet Another Colorization Tutorial

11 Typical Colorization Process Delineate region boundary Choose region color from palette. Images from: Yet Another Colorization Tutorial

12 Typical Colorization Process Delineate region boundary Choose region color from palette. Images from: Yet Another Colorization Tutorial

13 Typical Colorization Process Delineate region boundary Choose region color from palette. Images from: Yet Another Colorization Tutorial

14 Video Colorization Process Delineate region boundary Choose region color from palette. Track regions across video frames

15 Colorization Process Discussion Time consuming and labor intensive Fine boundaries. Failures in tracking.

16 Colorization by Analogy A : A B : B? Hertzmann et al. 2001, Welsh et al. 2002

17 Colorization by Analogy A : A B : B Hertzmann et al. 2001, Welsh et al. 2002

18 Transferring Color To Greyscale Images Welsh et al. 2002

19 Colorization by Analogy - Discussion Indirect artistic control No spatial continuity constraint

20 Levin et al. Approach

21 Levin et al. Approach Artist scribbles desired colors inside regions

22 Levin et al. Approach Colors are propagated to all pixels Nearby pixels with similar intensities should have the same color

23 Propagation using Optimization Y U, V Intensity channel Color channels Neighboring pixels with similar intensities should have similar colors

24 Y U, V Intensity channel Color channels Neighboring pixels with similar intensities should have similar colors

25 Propagation using Optimization Y U, V Intensity channel Color channels 2 J ( U ) r U ( r) s N ( r) w rs U ( s) Minimize difference between color at a pixel and an affinityweighted average of the neighbors

26 Affinity Functions w rs e 2 ( Y ( r) Y ( s)) 2 / r r proportional to local variance

27 Affinity Functions in Space-Time w rs e 2 2 ( Y ( r) Y ( s)) / 2 r i 1 i i 1

28 Minimizing cost function Minimize: 2 J ( U ) r U ( r) s N ( r) w rs U ( s) Subject to labeling constraints Since cost is quadratic, minimum can be found by solving sparse system of linear equations. Using Matlab s least-squares solver for sparse linear systems (code online)

29 Color Interpolation

30 Coloring Stills

31

32 Coloring Stills Original Colorized

33 Progressive Colorization

34 Progressive Colorization

35 Progressive Colorization

36 Coloring Stills

37 Coloring Stills

38 Colorization Challenges

39 Segmentation? NCuts Segmentation (Shi & Malik 97) Segmentation aided colorization Our result

40 Recoloring Affinity between pixels based on intensity AND color similarities.

41 Recoloring

42 Recoloring c.f. Poisson image editing Perez et al. SIGGRAPH 2003

43 Colorizing Video 13 out of 92 frames

44 Colorizing Video 16 out of 101 frames

45

46 Matting as Colorization Page of example results Red channel<->matte

47 Still Needed: Import image segmentation advantages: affinity functions, optimization techniques. Alternative color spaces, propagating hue and saturation differently

48 Other Approaches? Color space was YUV Small amount of user effort needed For film/color grading, can this be automated?

49 Linear Alignment Reinhard, Ashikhmin, Gooch, Shirley, CG&A 2001 (R, G, B) is rubbish: all channels are correlated (L*, a*, b*) is good Algorithm (per channel): Align mean Rescale standard deviation

50

51

52 Linear Alignment Reinhard, Ashikhmin, Gooch, Shirley, CG&A 2001 RGB is rubbish: all channels are correlated L* a* b* is good Algorithm (per channel): Align mean Rescale standard deviation

53

54

55

56 But Needs Some Guidance

57 Non-Linear Alignment Pitié, Kokaram, Dahyot, ICCV 2005 See their project web-page for code + papers See Lab page for code + paper

58 Iterate 1D Solution at Different Rotations 1D solution uses cumulative PDFs: ND solution: Pick a rotation matrix R, apply to both 3D distribs. Project both distribs. onto each axis in turn Apply 1D solution Unproject, unrotate <repeat>

59

60

61

62 Don t Forget Other Problems (Really) automatic colorization Flicker removal Dirt removal Noise removal Supersampling Texture (aren t histograms enough?) Two-scale Tone Management for Photographic Look, by Bae, Paris, Durand, Siggraph 2006

63 Don t Forget Other Problems (Really) automatic colorization Flicker removal Dirt removal Noise removal Supersampling Texture Two-scale Tone Management for Photographic Look, by Bae, Paris, Durand, Siggraph 2006

Overview. Video. Overview 4/7/2008. Optical flow. Why estimate motion? Motion estimation: Optical flow. Motion Magnification Colorization.

Overview. Video. Overview 4/7/2008. Optical flow. Why estimate motion? Motion estimation: Optical flow. Motion Magnification Colorization. Overview Video Optical flow Motion Magnification Colorization Lecture 9 Optical flow Motion Magnification Colorization Overview Optical flow Combination of slides from Rick Szeliski, Steve Seitz, Alyosha

More information

Image Compositing and Blending

Image Compositing and Blending Computational Photography and Capture: Image Compositing and Blending Gabriel Brostow & Tim Weyrich TA: Frederic Besse Vignetting 3 Figure from http://www.vanwalree.com/optics/vignetting.html Radial Distortion

More information

Colorization: History

Colorization: History Colorization Colorization: History Hand tinting http://en.wikipedia.org/wiki/hand-colouring Colorization: History Film colorization http://en.wikipedia.org/wiki/film_colorization Colorization in 1986 Colorization

More information

Examples. Bilateral filter. Input. Soft texture is removed

Examples. Bilateral filter. Input. Soft texture is removed A Gentle Introduction to Bilateral Filtering and its Applications Limitation? Pierre Kornprobst (INRIA) 0:20 Examples Soft texture is removed Input Bilateral filter Examples Constant regions appear Input

More information

Academia Arena 2017;9(11) Colorization of Gray Level Images by Neural Network

Academia Arena 2017;9(11)  Colorization of Gray Level Images by Neural Network Colorization of Gray Level Images by Neural Network * Hossein Ghayoumi Zadeh 1, Hojat Jafari 1, Ali Hayati 2, Alireza Malvandi 1, Mohammad Fiuzy 1, Javad Haddadnia 1 1. Sabzevar Tarbiat Moallem University/Department

More information

Computational Photography and Video: Intrinsic Images. Prof. Marc Pollefeys Dr. Gabriel Brostow

Computational Photography and Video: Intrinsic Images. Prof. Marc Pollefeys Dr. Gabriel Brostow Computational Photography and Video: Intrinsic Images Prof. Marc Pollefeys Dr. Gabriel Brostow Last Week Schedule Computational Photography and Video Exercises 18 Feb Introduction to Computational Photography

More information

Midterm Examination CS 534: Computational Photography

Midterm Examination CS 534: Computational Photography Midterm Examination CS 534: Computational Photography November 3, 2016 NAME: Problem Score Max Score 1 6 2 8 3 9 4 12 5 4 6 13 7 7 8 6 9 9 10 6 11 14 12 6 Total 100 1 of 8 1. [6] (a) [3] What camera setting(s)

More information

Landmark-Based Sparse Color Representations for Color Transfer

Landmark-Based Sparse Color Representations for Color Transfer Landmark-Based Sparse Color Representations for Color Transfer Tzu-Wei Huang and Hwann-Tzong Chen Department of Computer Science, National Tsing Hua University 30013 Hsinchu, Taiwan Abstract We present

More information

Geometric camera models and calibration

Geometric camera models and calibration Geometric camera models and calibration http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2018, Lecture 13 Course announcements Homework 3 is out. - Due October

More information

Introduction to Computer Graphics. Image Processing (1) June 8, 2017 Kenshi Takayama

Introduction to Computer Graphics. Image Processing (1) June 8, 2017 Kenshi Takayama Introduction to Computer Graphics Image Processing (1) June 8, 2017 Kenshi Takayama Today s topics Edge-aware image processing Gradient-domain image processing 2 Image smoothing using Gaussian Filter Smoothness

More information

Image Composition. COS 526 Princeton University

Image Composition. COS 526 Princeton University Image Composition COS 526 Princeton University Modeled after lecture by Alexei Efros. Slides by Efros, Durand, Freeman, Hays, Fergus, Lazebnik, Agarwala, Shamir, and Perez. Image Composition Jurassic Park

More information

Fast Colorization Using Edge and Gradient Constrains

Fast Colorization Using Edge and Gradient Constrains Fast Colorization Using Edge and Gradient Constrains Yao Li Digital Media Lab, Shanghai Jiao Tong university No.800 Dongchuan Road 200240, Shanghai, China yaolily77@sjtu.edu.cn Ma Lizhuang Digital Media

More information

EE795: Computer Vision and Intelligent Systems

EE795: Computer Vision and Intelligent Systems EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 FDH 204 Lecture 14 130307 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Review Stereo Dense Motion Estimation Translational

More information

Targil 10 : Why Mosaic? Why is this a challenge? Exposure differences Scene illumination Miss-registration Moving objects

Targil 10 : Why Mosaic? Why is this a challenge? Exposure differences Scene illumination Miss-registration Moving objects Why Mosaic? Are you getting the whole picture? Compact Camera FOV = 5 x 35 Targil : Panoramas - Stitching and Blending Some slides from Alexei Efros 2 Slide from Brown & Lowe Why Mosaic? Are you getting

More information

Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation

Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation. Range Imaging Through Triangulation Obviously, this is a very slow process and not suitable for dynamic scenes. To speed things up, we can use a laser that projects a vertical line of light onto the scene. This laser rotates around its vertical

More information

IMA Preprint Series # 1979

IMA Preprint Series # 1979 INPAINTING THE COLORS By Guillermo Sapiro IMA Preprint Series # 1979 ( May 2004 ) INSTITUTE FOR MATHEMATICS AND ITS APPLICATIONS UNIVERSITY OF MINNESOTA 514 Vincent Hall 206 Church Street S.E. Minneapolis,

More information

Homographies and RANSAC

Homographies and RANSAC Homographies and RANSAC Computer vision 6.869 Bill Freeman and Antonio Torralba March 30, 2011 Homographies and RANSAC Homographies RANSAC Building panoramas Phototourism 2 Depth-based ambiguity of position

More information

Skin Color Transfer. Introduction. Other-Race Effect. This work

Skin Color Transfer. Introduction. Other-Race Effect. This work Skin Color Transfer This work Introduction Investigates the importance of facial skin color for race differentiation Explore the potential information for other-race race effect Basic Idea Match the average

More information

TOWARDS AUTOMATED COLOUR GRADING

TOWARDS AUTOMATED COLOUR GRADING TOWARDS AUTOMATED COLOUR GRADING F. Pitié, A.C. Kokaram and R. Dahyot University of Dublin, Trinity College, Ireland, fpitie@mee.tcd.ie, anil.kokaram@tcd.ie, dahyot@mee.tcd.ie Keywords: colour grading,

More information

Image Blending and Compositing NASA

Image Blending and Compositing NASA Image Blending and Compositing NASA CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2016 Image Compositing Compositing Procedure 1. Extract Sprites (e.g using Intelligent

More information

Motion Estimation. There are three main types (or applications) of motion estimation:

Motion Estimation. There are three main types (or applications) of motion estimation: Members: D91922016 朱威達 R93922010 林聖凱 R93922044 謝俊瑋 Motion Estimation There are three main types (or applications) of motion estimation: Parametric motion (image alignment) The main idea of parametric motion

More information

Announcements. Mosaics. Image Mosaics. How to do it? Basic Procedure Take a sequence of images from the same position =

Announcements. Mosaics. Image Mosaics. How to do it? Basic Procedure Take a sequence of images from the same position = Announcements Project 2 out today panorama signup help session at end of class Today mosaic recap blending Mosaics Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html

More information

Multimedia Technology CHAPTER 4. Video and Animation

Multimedia Technology CHAPTER 4. Video and Animation CHAPTER 4 Video and Animation - Both video and animation give us a sense of motion. They exploit some properties of human eye s ability of viewing pictures. - Motion video is the element of multimedia

More information

CS6670: Computer Vision

CS6670: Computer Vision CS6670: Computer Vision Noah Snavely Lecture 7: Image Alignment and Panoramas What s inside your fridge? http://www.cs.washington.edu/education/courses/cse590ss/01wi/ Projection matrix intrinsics projection

More information

Automatic Palette Extraction for Image Editing

Automatic Palette Extraction for Image Editing Automatic Palette Extraction for Image Editing Mairéad Grogan, Matis Hudon, Daniel McCormack, Aljosa Smolic School of Computer Science and Statistics, Trinity College Dublin RGB RGB RGB RGB RGB LAB LAB

More information

Computer Vision 2. SS 18 Dr. Benjamin Guthier Professur für Bildverarbeitung. Computer Vision 2 Dr. Benjamin Guthier

Computer Vision 2. SS 18 Dr. Benjamin Guthier Professur für Bildverarbeitung. Computer Vision 2 Dr. Benjamin Guthier Computer Vision 2 SS 18 Dr. Benjamin Guthier Professur für Bildverarbeitung Computer Vision 2 Dr. Benjamin Guthier 3. HIGH DYNAMIC RANGE Computer Vision 2 Dr. Benjamin Guthier Pixel Value Content of this

More information

Course overview. Digital Visual Effects, Spring 2005 Yung-Yu Chuang 2005/2/23

Course overview. Digital Visual Effects, Spring 2005 Yung-Yu Chuang 2005/2/23 Course overview Digital Visual Effects, Spring 2005 Yung-Yu Chuang 2005/2/23 Logistics Meeting time: 1:20pm-4:20pm, Wednesday Classroom: CSIE Room 110 Instructor: Yung-Yu Chuang (cyy@csie.ntu.edu.tw) Textbook:

More information

Mosaics. Today s Readings

Mosaics. Today s Readings Mosaics VR Seattle: http://www.vrseattle.com/ Full screen panoramas (cubic): http://www.panoramas.dk/ Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html Today s Readings Szeliski and Shum paper (sections

More information

Synthesizing Realistic Facial Expressions from Photographs

Synthesizing Realistic Facial Expressions from Photographs Synthesizing Realistic Facial Expressions from Photographs 1998 F. Pighin, J Hecker, D. Lischinskiy, R. Szeliskiz and D. H. Salesin University of Washington, The Hebrew University Microsoft Research 1

More information

Write detailed answers. That way, you can have points for partial answers;

Write detailed answers. That way, you can have points for partial answers; GIF-/7 Computational Photography Winter Midsemester exam February 6, Total time: minutes This exam has 7 questions on 8 pages (including this one), and is worth % of the total grade for the semester. Make

More information

Manifold Preserving Edit Propagation

Manifold Preserving Edit Propagation Manifold Preserving Edit Propagation SIGGRAPH ASIA 2012 Xiaowu Chen, Dongqing Zou, Qinping Zhao, Ping Tan Kim, Wook 2013. 11. 22 Abstract Edit propagation algorithm more robust to color blending maintain

More information

CS664 Lecture #18: Motion

CS664 Lecture #18: Motion CS664 Lecture #18: Motion Announcements Most paper choices were fine Please be sure to email me for approval, if you haven t already This is intended to help you, especially with the final project Use

More information

ELEC Dr Reji Mathew Electrical Engineering UNSW

ELEC Dr Reji Mathew Electrical Engineering UNSW ELEC 4622 Dr Reji Mathew Electrical Engineering UNSW Review of Motion Modelling and Estimation Introduction to Motion Modelling & Estimation Forward Motion Backward Motion Block Motion Estimation Motion

More information

Topics to be Covered in the Rest of the Semester. CSci 4968 and 6270 Computational Vision Lecture 15 Overview of Remainder of the Semester

Topics to be Covered in the Rest of the Semester. CSci 4968 and 6270 Computational Vision Lecture 15 Overview of Remainder of the Semester Topics to be Covered in the Rest of the Semester CSci 4968 and 6270 Computational Vision Lecture 15 Overview of Remainder of the Semester Charles Stewart Department of Computer Science Rensselaer Polytechnic

More information

Automatic Generation of An Infinite Panorama

Automatic Generation of An Infinite Panorama Automatic Generation of An Infinite Panorama Lisa H. Chan Alexei A. Efros Carnegie Mellon University Original Image Scene Matches Output Image Figure 1: Given an input image, scene matching from a large

More information

Today s lecture. Image Alignment and Stitching. Readings. Motion models

Today s lecture. Image Alignment and Stitching. Readings. Motion models Today s lecture Image Alignment and Stitching Computer Vision CSE576, Spring 2005 Richard Szeliski Image alignment and stitching motion models cylindrical and spherical warping point-based alignment global

More information

The Light Field and Image-Based Rendering

The Light Field and Image-Based Rendering Lecture 11: The Light Field and Image-Based Rendering Visual Computing Systems Demo (movie) Royal Palace: Madrid, Spain Image-based rendering (IBR) So far in course: rendering = synthesizing an image from

More information

CS 354R: Computer Game Technology

CS 354R: Computer Game Technology CS 354R: Computer Game Technology Texture and Environment Maps Fall 2018 Texture Mapping Problem: colors, normals, etc. are only specified at vertices How do we add detail between vertices without incurring

More information

CS664 Lecture #19: Layers, RANSAC, panoramas, epipolar geometry

CS664 Lecture #19: Layers, RANSAC, panoramas, epipolar geometry CS664 Lecture #19: Layers, RANSAC, panoramas, epipolar geometry Some material taken from: David Lowe, UBC Jiri Matas, CMP Prague http://cmp.felk.cvut.cz/~matas/papers/presentations/matas_beyondransac_cvprac05.ppt

More information

Temporal Color Morphing

Temporal Color Morphing Temporal Color Morphing Xuezhong Xiao and Lizhuang Ma Department of Computer Science and Engineering Shanghai Jiao Tong University, China Abstract. Many natural phenomena usually appear in the scenes of

More information

Image Morphing. Application: Movie Special Effects. Application: Registration /Alignment. Image Cross-Dissolve

Image Morphing. Application: Movie Special Effects. Application: Registration /Alignment. Image Cross-Dissolve Image Morphing Application: Movie Special Effects Morphing is turning one image into another (through a seamless transition) First movies with morphing Willow, 1988 Indiana Jones and the Last Crusade,

More information

IMAGE DENOISING TO ESTIMATE THE GRADIENT HISTOGRAM PRESERVATION USING VARIOUS ALGORITHMS

IMAGE DENOISING TO ESTIMATE THE GRADIENT HISTOGRAM PRESERVATION USING VARIOUS ALGORITHMS IMAGE DENOISING TO ESTIMATE THE GRADIENT HISTOGRAM PRESERVATION USING VARIOUS ALGORITHMS P.Mahalakshmi 1, J.Muthulakshmi 2, S.Kannadhasan 3 1,2 U.G Student, 3 Assistant Professor, Department of Electronics

More information

Mixture Models and EM

Mixture Models and EM Mixture Models and EM Goal: Introduction to probabilistic mixture models and the expectationmaximization (EM) algorithm. Motivation: simultaneous fitting of multiple model instances unsupervised clustering

More information

Image Stitching. Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi

Image Stitching. Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi Image Stitching Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi Combine two or more overlapping images to make one larger image Add example Slide credit: Vaibhav Vaish

More information

CS559: Computer Graphics. Lecture 1 Introduction Li Zhang University of Wisconsin, Madison

CS559: Computer Graphics. Lecture 1 Introduction Li Zhang University of Wisconsin, Madison CS559: Computer Graphics Lecture 1 Introduction Li Zhang University of Wisconsin, Madison Today Introduction to Computer Graphics Course Overview What is Computer Graphics Using computers to generate and

More information

Course Evaluations. h"p:// 4 Random Individuals will win an ATI Radeon tm HD2900XT

Course Evaluations. hp://  4 Random Individuals will win an ATI Radeon tm HD2900XT Course Evaluations h"p://www.siggraph.org/courses_evalua4on 4 Random Individuals will win an ATI Radeon tm HD2900XT A Gentle Introduction to Bilateral Filtering and its Applications From Gaussian blur

More information

Graphics Hardware and Display Devices

Graphics Hardware and Display Devices Graphics Hardware and Display Devices CSE328 Lectures Graphics/Visualization Hardware Many graphics/visualization algorithms can be implemented efficiently and inexpensively in hardware Facilitates interactive

More information

GRAPH MODELLING OF 3D GEOMETRIC INFORMATION FOR COLOR CONSISTENCY OF MULTIVIEW IMAGES. Manohar Kuse, Sunil Prasad Jaiswal

GRAPH MODELLING OF 3D GEOMETRIC INFORMATION FOR COLOR CONSISTENCY OF MULTIVIEW IMAGES. Manohar Kuse, Sunil Prasad Jaiswal GRAPH MODELLING OF 3D GEOMETRIC INFORMATION FOR COLOR CONSISTENCY OF MULTIVIEW IMAGES Manohar Kuse, Sunil Prasad Jaiswal The Hong Kong University of Science and Technology, Hong Kong ABSTRACT In this paper

More information

Image Warping and Mosacing

Image Warping and Mosacing Image Warping and Mosacing 15-463: Rendering and Image Processing Alexei Efros with a lot of slides stolen from Steve Seitz and Rick Szeliski Today Mosacs Image Warping Homographies Programming Assignment

More information

Computer Vision 5 Segmentation by Clustering

Computer Vision 5 Segmentation by Clustering Computer Vision 5 Segmentation by Clustering MAP-I Doctoral Programme Miguel Tavares Coimbra Outline Introduction Applications Simple clustering K-means clustering Graph-theoretic clustering Acknowledgements:

More information

CSCI 5980: Assignment #3 Homography

CSCI 5980: Assignment #3 Homography Submission Assignment due: Feb 23 Individual assignment. Write-up submission format: a single PDF up to 3 pages (more than 3 page assignment will be automatically returned.). Code and data. Submission

More information

Color Source Separation for Enhanced Pixel Manipulations MSR-TR

Color Source Separation for Enhanced Pixel Manipulations MSR-TR Color Source Separation for Enhanced Pixel Manipulations MSR-TR-2-98 C. Lawrence Zitnick Microsoft Research larryz@microsoft.com Devi Parikh Toyota Technological Institute, Chicago (TTIC) dparikh@ttic.edu

More information

Why is computer vision difficult?

Why is computer vision difficult? Why is computer vision difficult? Viewpoint variation Illumination Scale Why is computer vision difficult? Intra-class variation Motion (Source: S. Lazebnik) Background clutter Occlusion Challenges: local

More information

Lecture 15: Image-Based Rendering and the Light Field. Kayvon Fatahalian CMU : Graphics and Imaging Architectures (Fall 2011)

Lecture 15: Image-Based Rendering and the Light Field. Kayvon Fatahalian CMU : Graphics and Imaging Architectures (Fall 2011) Lecture 15: Image-Based Rendering and the Light Field Kayvon Fatahalian CMU 15-869: Graphics and Imaging Architectures (Fall 2011) Demo (movie) Royal Palace: Madrid, Spain Image-based rendering (IBR) So

More information

Clustering Color/Intensity. Group together pixels of similar color/intensity.

Clustering Color/Intensity. Group together pixels of similar color/intensity. Clustering Color/Intensity Group together pixels of similar color/intensity. Agglomerative Clustering Cluster = connected pixels with similar color. Optimal decomposition may be hard. For example, find

More information

Feature Tracking and Optical Flow

Feature Tracking and Optical Flow Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who 1 in turn adapted slides from Steve Seitz, Rick Szeliski,

More information

Feature Tracking and Optical Flow

Feature Tracking and Optical Flow Feature Tracking and Optical Flow Prof. D. Stricker Doz. G. Bleser Many slides adapted from James Hays, Derek Hoeim, Lana Lazebnik, Silvio Saverse, who in turn adapted slides from Steve Seitz, Rick Szeliski,

More information

Single-view 3D Reconstruction

Single-view 3D Reconstruction Single-view 3D Reconstruction 10/12/17 Computational Photography Derek Hoiem, University of Illinois Some slides from Alyosha Efros, Steve Seitz Notes about Project 4 (Image-based Lighting) You can work

More information

High Dynamic Range Imaging.

High Dynamic Range Imaging. High Dynamic Range Imaging High Dynamic Range [3] In photography, dynamic range (DR) is measured in exposure value (EV) differences or stops, between the brightest and darkest parts of the image that show

More information

Median filter. Non-linear filtering example. Degraded image. Radius 1 median filter. Today

Median filter. Non-linear filtering example. Degraded image. Radius 1 median filter. Today Today Non-linear filtering example Median filter Replace each pixel by the median over N pixels (5 pixels, for these examples). Generalizes to rank order filters. In: In: 5-pixel neighborhood Out: Out:

More information

Non-linear filtering example

Non-linear filtering example Today Non-linear filtering example Median filter Replace each pixel by the median over N pixels (5 pixels, for these examples). Generalizes to rank order filters. In: In: 5-pixel neighborhood Out: Out:

More information

EE795: Computer Vision and Intelligent Systems

EE795: Computer Vision and Intelligent Systems EE795: Computer Vision and Intelligent Systems Spring 2012 TTh 17:30-18:45 WRI C225 Lecture 02 130124 http://www.ee.unlv.edu/~b1morris/ecg795/ 2 Outline Basics Image Formation Image Processing 3 Intelligent

More information

Image Based Rendering. D.A. Forsyth, with slides from John Hart

Image Based Rendering. D.A. Forsyth, with slides from John Hart Image Based Rendering D.A. Forsyth, with slides from John Hart Topics Mosaics translating cameras reveal extra information, break occlusion Optical flow for very small movements of the camera Explicit

More information

Data-driven methods: Video & Texture. A.A. Efros

Data-driven methods: Video & Texture. A.A. Efros Data-driven methods: Video & Texture A.A. Efros 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 Michel Gondry train video http://youtube.com/watch?v=ques1bwvxga Weather Forecasting for Dummies

More information

3D Editing System for Captured Real Scenes

3D Editing System for Captured Real Scenes 3D Editing System for Captured Real Scenes Inwoo Ha, Yong Beom Lee and James D.K. Kim Samsung Advanced Institute of Technology, Youngin, South Korea E-mail: {iw.ha, leey, jamesdk.kim}@samsung.com Tel:

More information

Today. Motivation. Motivation. Image gradient. Image gradient. Computational Photography

Today. Motivation. Motivation. Image gradient. Image gradient. Computational Photography Computational Photography Matthias Zwicker University of Bern Fall 009 Today Gradient domain image manipulation Introduction Gradient cut & paste Tone mapping Color-to-gray conversion Motivation Cut &

More information

What will we learn? Geometric Operations. Mapping and Affine Transformations. Chapter 7 Geometric Operations

What will we learn? Geometric Operations. Mapping and Affine Transformations. Chapter 7 Geometric Operations What will we learn? Lecture Slides ME 4060 Machine Vision and Vision-based Control Chapter 7 Geometric Operations What do geometric operations do to an image and what are they used for? What are the techniques

More information

IMA Preprint Series # 1979

IMA Preprint Series # 1979 INPAINTING THE COLORS By Guillermo Sapiro IMA Preprint Series # 1979 ( May 2004 ) INSTITUTE FOR MATHEMATICS AND ITS APPLICATIONS UNIVERSITY OF MINNESOTA 514 Vincent Hall 206 Church Street S.E. Minneapolis,

More information

CS4670: Computer Vision

CS4670: Computer Vision CS4670: Computer Vision Noah Snavely Lecture 34: Segmentation From Sandlot Science Announcements In-class exam this Friday, December 3 Review session in class on Wednesday Final projects: Slides due: Sunday,

More information

CS559: Computer Graphics. Lecture 6: Painterly Rendering and Edges Li Zhang Spring 2008

CS559: Computer Graphics. Lecture 6: Painterly Rendering and Edges Li Zhang Spring 2008 CS559: Computer Graphics Lecture 6: Painterly Rendering and Edges Li Zhang Spring 2008 So far Image formation: eyes and cameras Image sampling and reconstruction Image resampling and filtering Today Painterly

More information

Lecture 19: Depth Cameras. Visual Computing Systems CMU , Fall 2013

Lecture 19: Depth Cameras. Visual Computing Systems CMU , Fall 2013 Lecture 19: Depth Cameras Visual Computing Systems Continuing theme: computational photography Cameras capture light, then extensive processing produces the desired image Today: - Capturing scene depth

More information

Image-Based Modeling and Rendering

Image-Based Modeling and Rendering Image-Based Modeling and Rendering Richard Szeliski Microsoft Research IPAM Graduate Summer School: Computer Vision July 26, 2013 How far have we come? Light Fields / Lumigraph - 1996 Richard Szeliski

More information

Panoramic Image Stitching

Panoramic Image Stitching Mcgill University Panoramic Image Stitching by Kai Wang Pengbo Li A report submitted in fulfillment for the COMP 558 Final project in the Faculty of Computer Science April 2013 Mcgill University Abstract

More information

Local Image Features

Local Image Features Local Image Features Computer Vision Read Szeliski 4.1 James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial Flashed Face Distortion 2nd Place in the 8th Annual Best

More information

BCC Rays Streaky Filter

BCC Rays Streaky Filter BCC Rays Streaky Filter The BCC Rays Streaky filter produces a light that contains streaks. The resulting light is generated from a chosen channel in the source image, and spreads from a source point in

More information

CS4670: Computer Vision

CS4670: Computer Vision CS467: Computer Vision Noah Snavely Lecture 13: Projection, Part 2 Perspective study of a vase by Paolo Uccello Szeliski 2.1.3-2.1.6 Reading Announcements Project 2a due Friday, 8:59pm Project 2b out Friday

More information

CIS581: Computer Vision and Computational Photography Project 2: Face Morphing and Blending Due: Oct. 17, 2017 at 3:00 pm

CIS581: Computer Vision and Computational Photography Project 2: Face Morphing and Blending Due: Oct. 17, 2017 at 3:00 pm CIS581: Computer Vision and Computational Photography Project 2: Face Morphing and Blending Due: Oct. 17, 2017 at 3:00 pm Instructions This is an individual project. Individual means each student must

More information

Multi-view stereo. Many slides adapted from S. Seitz

Multi-view stereo. Many slides adapted from S. Seitz Multi-view stereo Many slides adapted from S. Seitz Beyond two-view stereo The third eye can be used for verification Multiple-baseline stereo Pick a reference image, and slide the corresponding window

More information

About Phoenix FD PLUGIN FOR 3DS MAX AND MAYA. SIMULATING AND RENDERING BOTH LIQUIDS AND FIRE/SMOKE. USED IN MOVIES, GAMES AND COMMERCIALS.

About Phoenix FD PLUGIN FOR 3DS MAX AND MAYA. SIMULATING AND RENDERING BOTH LIQUIDS AND FIRE/SMOKE. USED IN MOVIES, GAMES AND COMMERCIALS. About Phoenix FD PLUGIN FOR 3DS MAX AND MAYA. SIMULATING AND RENDERING BOTH LIQUIDS AND FIRE/SMOKE. USED IN MOVIES, GAMES AND COMMERCIALS. Phoenix FD core SIMULATION & RENDERING. SIMULATION CORE - GRID-BASED

More information

CS6670: Computer Vision

CS6670: Computer Vision CS6670: Computer Vision Noah Snavely Lecture 5: Projection Reading: Szeliski 2.1 Projection Reading: Szeliski 2.1 Projection Müller Lyer Illusion http://www.michaelbach.de/ot/sze_muelue/index.html Modeling

More information

Computer Graphics Disciplines. Grading. Textbooks. Course Overview. Assignment Policies. Computer Graphics Goals I

Computer Graphics Disciplines. Grading. Textbooks. Course Overview. Assignment Policies. Computer Graphics Goals I CSCI 480 Computer Graphics Lecture 1 Course Overview January 10, 2011 Jernej Barbic University of Southern California Administrative Issues Modeling Animation Rendering OpenGL Programming Course Information

More information

Topic 12: Texture Mapping. Motivation Sources of texture Texture coordinates Bump mapping, mip-mapping & env mapping

Topic 12: Texture Mapping. Motivation Sources of texture Texture coordinates Bump mapping, mip-mapping & env mapping Topic 12: Texture Mapping Motivation Sources of texture Texture coordinates Bump mapping, mip-mapping & env mapping Texture sources: Photographs Texture sources: Procedural Texture sources: Solid textures

More information

Miniature faking. In close-up photo, the depth of field is limited.

Miniature faking. In close-up photo, the depth of field is limited. Miniature faking In close-up photo, the depth of field is limited. http://en.wikipedia.org/wiki/file:jodhpur_tilt_shift.jpg Miniature faking Miniature faking http://en.wikipedia.org/wiki/file:oregon_state_beavers_tilt-shift_miniature_greg_keene.jpg

More information

Image Based Lighting with Near Light Sources

Image Based Lighting with Near Light Sources Image Based Lighting with Near Light Sources Shiho Furuya, Takayuki Itoh Graduate School of Humanitics and Sciences, Ochanomizu University E-mail: {shiho, itot}@itolab.is.ocha.ac.jp Abstract Recent some

More information

Image Based Lighting with Near Light Sources

Image Based Lighting with Near Light Sources Image Based Lighting with Near Light Sources Shiho Furuya, Takayuki Itoh Graduate School of Humanitics and Sciences, Ochanomizu University E-mail: {shiho, itot}@itolab.is.ocha.ac.jp Abstract Recent some

More information

Today: non-linear filters, and uses for the filters and representations from last time. Review pyramid representations Non-linear filtering Textures

Today: non-linear filters, and uses for the filters and representations from last time. Review pyramid representations Non-linear filtering Textures 1 Today: non-linear filters, and uses for the filters and representations from last time Review pyramid representations Non-linear filtering Textures 2 Reading Related to today s lecture: Chapter 9, Forsyth&Ponce..

More information

Data-driven methods: Video & Texture. A.A. Efros

Data-driven methods: Video & Texture. A.A. Efros Data-driven methods: Video & Texture A.A. Efros CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2014 Michel Gondry train video http://www.youtube.com/watch?v=0s43iwbf0um

More information

BSB663 Image Processing Pinar Duygulu. Slides are adapted from Selim Aksoy

BSB663 Image Processing Pinar Duygulu. Slides are adapted from Selim Aksoy BSB663 Image Processing Pinar Duygulu Slides are adapted from Selim Aksoy Image matching Image matching is a fundamental aspect of many problems in computer vision. Object or scene recognition Solving

More information

Texture. Announcements. Markov Chains. Modeling Texture. Guest lecture next Tuesday. Evals at the end of class today

Texture. Announcements. Markov Chains. Modeling Texture. Guest lecture next Tuesday. Evals at the end of class today Announcements Guest lecture next Tuesday Dan Goldman: CV in special effects held in Allen Center (room TBA) Evals at the end of class today Texture Today s Reading Alexei A. Efros and Thomas K. Leung,

More information

Recommender Systems New Approaches with Netflix Dataset

Recommender Systems New Approaches with Netflix Dataset Recommender Systems New Approaches with Netflix Dataset Robert Bell Yehuda Koren AT&T Labs ICDM 2007 Presented by Matt Rodriguez Outline Overview of Recommender System Approaches which are Content based

More information

Image correspondences and structure from motion

Image correspondences and structure from motion Image correspondences and structure from motion http://graphics.cs.cmu.edu/courses/15-463 15-463, 15-663, 15-862 Computational Photography Fall 2017, Lecture 20 Course announcements Homework 5 posted.

More information

arxiv: v1 [cs.cv] 4 Aug 2016

arxiv: v1 [cs.cv] 4 Aug 2016 GONG, FINLAYSON, FISHER: RECODING COLOR TRANSFER AS A COLOR HOMOGRAPHY1 arxiv:1608.01505v1 [cs.cv] 4 Aug 2016 Recoding Color Transfer as a Color Han Gong 1 http://www2.cmp.uea.ac.uk/~ybb15eau/ Graham D.

More information

Texture Synthesis. Darren Green (

Texture Synthesis. Darren Green ( Texture Synthesis Darren Green (www.darrensworld.com) 15-463: Computational Photography Alexei Efros, CMU, Fall 2005 Texture Texture depicts spatially repeating patterns Many natural phenomena are textures

More information

Topic 11: Texture Mapping 11/13/2017. Texture sources: Solid textures. Texture sources: Synthesized

Topic 11: Texture Mapping 11/13/2017. Texture sources: Solid textures. Texture sources: Synthesized Topic 11: Texture Mapping Motivation Sources of texture Texture coordinates Bump mapping, mip mapping & env mapping Texture sources: Photographs Texture sources: Procedural Texture sources: Solid textures

More information

DD2423 Image Analysis and Computer Vision IMAGE FORMATION. Computational Vision and Active Perception School of Computer Science and Communication

DD2423 Image Analysis and Computer Vision IMAGE FORMATION. Computational Vision and Active Perception School of Computer Science and Communication DD2423 Image Analysis and Computer Vision IMAGE FORMATION Mårten Björkman Computational Vision and Active Perception School of Computer Science and Communication November 8, 2013 1 Image formation Goal:

More information

Outline. Segmentation & Grouping. Examples of grouping in vision. Grouping in vision. Grouping in vision 2/9/2011. CS 376 Lecture 7 Segmentation 1

Outline. Segmentation & Grouping. Examples of grouping in vision. Grouping in vision. Grouping in vision 2/9/2011. CS 376 Lecture 7 Segmentation 1 Outline What are grouping problems in vision? Segmentation & Grouping Wed, Feb 9 Prof. UT-Austin Inspiration from human perception Gestalt properties Bottom-up segmentation via clustering Algorithms: Mode

More information

Physics-based Vision: an Introduction

Physics-based Vision: an Introduction Physics-based Vision: an Introduction Robby Tan ANU/NICTA (Vision Science, Technology and Applications) PhD from The University of Tokyo, 2004 1 What is Physics-based? An approach that is principally concerned

More information

Image Warping and Morphing. Alexey Tikhonov : Computational Photography Alexei Efros, CMU, Fall 2007

Image Warping and Morphing. Alexey Tikhonov : Computational Photography Alexei Efros, CMU, Fall 2007 Image Warping and Morphing Alexey Tikhonov 15-463: Computational Photography Alexei Efros, CMU, Fall 2007 Image Warping in Biology D'Arcy Thompson http://www-groups.dcs.st-and.ac.uk/~history/miscellaneous/darcy.html

More information

Matting, Transparency, and Illumination. Slides from Alexei Efros

Matting, Transparency, and Illumination. Slides from Alexei Efros New course focus Done with texture, patches, and shuffling image content. Focus on light, specifically light models derived from multiple photos of the same scene. Matting, Transparency, and Illumination

More information

A Closed-Form Solution to Natural Image Matting

A Closed-Form Solution to Natural Image Matting 1 A Closed-Form Solution to Natural Image Matting Anat Levin Dani Lischinski Yair Weiss School of Computer Science and Engineering The Hebrew University of Jerusalem Abstract Interactive digital matting,

More information