SUV Color Space & Filtering. Computer Vision I. CSE252A Lecture 9. Announcement. HW2 posted If microphone goes out, let me know

Transcription

1 SUV Color Space & Flterng CSE5A Lecture 9 Announceent HW posted f cropone goes out let e now

2 Uncalbrated Potoetrc Stereo Taeaways For calbrated potoetrc stereo we estated te n by 3 atrx B of surface norals scaled by albedo usng lgtng. Uncalbrated nput: Only ages. No lgtng nfo. Wtout sadowng all ages le n 3D subspace of te n- pxel age space spanned by coluns of an n by 3 atrx B*. Fro 3 or ore ages SVD can be used to estate B*. Te n by 3 atrx B of surface norals scaled by albedo dffers fro B* by a 3x3 lnear transforaton BAB*. After enforcng ntegrablty one can only estate sape and albedo B up to a Generalzed Bas elef GB transforaton wc as 3 paraeters dept scalng tlt Generalzed Bas-elef Transforatons Obects dfferng by a GB ave te sae llunaton cone. Wtout nowledge of lgt source locaton one can only recover surfaces up to GB transforatons. CSE5A

3 SUV Color Space for Potoetrc Stereo of Glossy Obects Note: You ll use ts n HW. Labertan Potoetrc Stereo econsstructon wt Albedo Map 3

4 Wtout te albedo ap Motvaton: Labertan Algort Appled to NonLabertan Surface: Potoetrc Stereo CSE5a 4

5 Dcroatc eflecton Model Dffuse Surface Color depends on lgt source color and dffuse color CSE5a Dcroatc eflecton Model Transparent Fl Color of lgt source CSE5a 5

6 Dcroatc eflecton Model Delectrc Surface CSE5a age Foraton: Color Cannel Specular Color Dffuse Color + Were f d and f s are te dffuse and specular BDF CSE5a 6

7 age foraton: 3 color cannels 3 r g b f d ˆn ˆl D r D g D b + f s θˆn ˆl S r S g S b age color les n span of dffuse color D and specular color S f d ˆn ˆl D + f s θ ˆn ˆl S + CSE5a Varyng dffuse color Note: Dffuse color D vares over te age Specular color s ust color of lgt source CSE5a 7

8 Data-dependent SUV Color Space [] [ S U V] t Frst row of s specular color S. Oter rows are ortogonal to S UV spans a plane ortogonal to S CSE5a Propertes of SUV l Data-dependent. l otatonal ence lnear Transforaton. l Te S cannel encodes te entre specular coponent and an unnown aount of dffuse coponent. l Sadng nforaton s preserved n u and v cannels. CSE5a 8

9 Exaple GB S cannel U cannel V cannel CSE5a Mult-cannel Potoetrc Stereo CSE5a 9

10 Mult-cannel Potoetrc Stereo CSE5a Qualtatve esults CSE5a

11 Quanttatve esults CSE5a age Flterng

12 Wat s age flterng? Producng a new age were te value at a pxel n te output age s a functon of a negborood of te pxel locaton n te nput age Sootng by Averagng Kernel:

13 General process: For new age wose pxels are a wegted su of orgnal pxel values usng te sae set of wegts at eac pont. Process s called convoluton Wegts are called te Convoluton Kernel Lnear Flters Propertes Output s a lnear functon of te nput Output s a sft-nvarant functon of te nput.e. sft te nput age two pxels to te left te output s sfted two pxels to te left Local age Data Convoluton Kernel Output value Convoluton * Kernel K age Note: Typcally Kernel s relatvely sall n vson applcatons. 3

14 4 Convoluton: K* K Kernel sze s + by + Convoluton: K* K Kernel sze s + by +

15 5 Convoluton: K* K Kernel sze s + by + Convoluton: K* K Kernel sze s + by +

16 6 Convoluton: K* K Kernel sze s + by + Convoluton: K* K Kernel sze s + by +

17 7 Convoluton: K* K Kernel sze s + by + Convoluton: K* K Kernel sze s + by +

18 8 Convoluton: K* K Kernel sze s + by + Convoluton: K* K Kernel sze s + by +

19 9 Convoluton: K* K Kernel sze s + by + Swped fro Bll Freean

20 Swped fro Bll Freean Swped fro Bll Freean

21 Sfted one Pxel to te left Swped fro Bll Freean Swped fro Bll Freean

22 Swped fro Bll Freean Blur Exaples -Densonal

23 Blur Exaples -Densonal Practce wt lnear flters -? Orgnal Source: Coputer D. Lowe Vson 3

24 Practce wt lnear flters - Orgnal Sarpenng flter - Accentuates dfferences wt local average Source: Coputer D. Lowe Vson 4

25 Sootng by Averagng Kernel: Flters are teplates Applyng a flter at soe age locaton can be seen as tang a dot-product between a negborood and te ernel. Flterng te age s le tang a dot product at eac locaton. nsgt flters loo le te effects tey are ntended to fnd flters fnd effects tey loo le 5

26 Convolutonal Neural Networs Core operaton s not surprsngly convoluton. Can be extended to 3D e.g. age and GB as cannels N x N x 3 Voluetrc data suc as M CT Durng tranng of a CNN te wegts of te convoluton ernels are learned. Propertes of Contnuous Convoluton Holds for dscrete too Let fg be ages and * denote convoluton f * g x y f x u y v g u v dudv Coutatve: f*g g*f Assocatve: f*g* f*g* Lnear: for scalars a & b and ages fg af+bg* af*+bg* Dfferentaton rule f f * g * g f x x g * x 6

27 Flterng to reduce nose Nose s wat we re not nterested n. We ll dscuss sple low-level nose today: Lgt fluctuatons; Sensor nose; Quantzaton effects; Fnte precson Not coplex: sadows; extraneous obects. A pxel s negborood contans nforaton about ts ntensty. Averagng nose reduces ts effect. Addtve nose S + N. Nose doesn t depend on sgnal. We ll consder: s + n wt En s deternstc. n a rando var. n n ndependent for n n dentcally dstrbuted Gaussan nose n drawn fro Gaussan. 7

28 Gaussan nose Gaussan nose n drawn fro Gaussan dstrbuton wt zero ean and standard devaton σ Guassan Nose: sga 8

29 Guassan Nose: sga6 Averagng Flter Convoluton Kernel wt postve entres tat su. Average of te negborood. For effcency 8 adds and one ultply. f all wegts are equal t s called a BOX flter. 9 F 9

30 An sotropc Gaussan Sootng ernel proportonal to e x +y σ wc s a reasonable odel of a crcularly syetrc fuzzy blob Sootng by Averagng Kernel: 3

31 Sootng wt a Gaussan Kernel: Te effects of sootng Eac row sows sootng wt gaussans of dfferent wdt; eac colun sows dfferent realzatons of an age of gaussan nose. 3

32 Effcent pleentaton Bot te BOX flter and te Gaussan flter are separable: Frst convolve eac row wt a D flter Ten convolve eac colun wt a D flter. 3