Aoucemets Assigmet 5: Due Friday, 4:00 III Itroductio to Computer Visio CSE 52 Lecture 20 Fial Exam: ed, 6/9/04, :30-2:30, LH 2207 (here I ll discuss briefly today, ad will be at discussio sectio tomorrow for first 45 miutes. A Rough Spectrum Virtual Ciematography: Makig 'he Matrix' Sequels George Borshukov VFX echology Supervisor, ESC Etertaimet Friday, Jue 4, 2004 :00 p.m. to 2:30 p.m. [Pizza luch will precede the evet from oo to p.m.] Mai Auditorium, Sa Diego Supercomputer Ceter he presetatio will cover the key techologies that had to be developed ad deployed to create the sythetic huma sequeces i the Matrix sequels icludig Uiversal Capture - image-based facial aimatio, realistic huma face rederig, ad use of measured BRDF i film productio. It will also feature a breakdow of he Superpuch shot (pictured above from "he Matrix Revolutios" (the bullet time puch that Neo delivers to Aget Smith durig the film's last face-off. his difficult, importat, expesive, ad challegig shot was etirely computer geerated ad showcased the techological developmets of 3.5+ years at their best by showig a full-frame close-up of a kow huma actor. Appearace-Based (Eigeface, Fisherface Shape Cotexts Local Features + Spatial Relatios Geometric Ivariats Aspect Graphs Icreasig Geerality 3-D Model-Based Image Abstractios/ Volumetric Primitives Fuctio Projectio, ad recostructio A -pixel image x R ca be projected to a low-dimesioal feature space y R m by y = x From y R m, the recostructio of the poit is y he error of the recostructio is: x- x Face detectio usig distace to face space Sca a widow ω across the image, ad classify the widow as face/ot face as follows: Project widow to subspace, ad recostruct as described earlier. Compute distace betwee ω ad recostructio. Local miima of distace over all image locatios less tha some treshold are take as locatios of faces. Repeat at differet scales. Possibly ormalize widows itesity so that ω =.
Sigular Value Decompositio Ay m by matrix A may be factored such that A = UΣV [m x ] = [m x m][m x ][ x ] U: m by m, orthogoal matrix Colums of U are the eigevectors of AA V: by, orthogoal matrix, colums are the eigevectors of A A Σ: m by, diagoal with o-egative etries (σ, σ 2,, σ s with s=mi(m, are called the called the sigular values Sigular values are the square roots of eigevalues of both AA ad A A & Colums of U are correspodig Eigevectors!! Performig PCA with SVD Sigular values of A are the square roots of eigevalues of both AA ad A A & Colums of U are correspodig Eigevectors Ad a iai = [ a a2 L a ][ a a2 L a ] = AA i= Covariace matrix is: Σ = i= r r r r ( x i µ ( x µ So, igorig / subtract mea image µ from each iput image, create data matrix, ad perform (thi SVD o the data matrix. i Result of SVD algorithm: σ σ 2 σ s PCA & Fisher s Liear Discrimiat PCA PCA (Eigefaces χ χ 2 PCA = arg max S Maximizes projected total scatter Fisher s Liear Discrimiat FLD fld SB = arg max S Maximizes ratio of projected betwee-class to projected withi-class scatter Variability: Camera positio Illumiatio Iteral parameters ithi-class variatios A example: surfaces of first 3 coefficiets Parameterized Eigespace
Basic ideas i classifiers Bayesia Classificatio Discussed o blackboard, but slides may be helpful Loss some errors may be more expesive tha others e.g. a fatal disease that is easily cured by a cheap medicie with o side-effects -> false positives i diagosis are better tha false egatives e discuss two class classificatio: L(->2 is the loss caused by callig a 2 otal risk of usig classifier s Basic ideas i classifiers Geerally, we should classify as if the expected loss of classifyig as is better tha for 2 gives Some loss may be ievitable: the miimum risk (shaded area is called the Bayes risk if 2 if Crucial otio: Decisio boudary poits where the loss is the same for either case Example: kow distributios Fidig a decisio boudary is ot the same as modellig a coditioal desity. pxk ( = 2π p 2 Σ 2 exp 2 x µ k ( Σ ( x µ k Assume ormal class desities, p-dimesioal measuremets with commo (kow covariace ad differet (kow πmeas k Class priors are Ca igore a commo factor i posteriors - importat; posteriors are the: pk ( x ( π k p 2 Σ 2 exp 2π 2 x µ k ( Σ ( x µ k
Classifier boils dow to: choose class that miimizes: Mahalaobis distace δ( x, µ k 2 2 log π k where δ x, µ ( k = x µ k ( Σ x µ ( k because covariace is commo, this simplifies to sig of a liear expressio (i.e. Vorooi diagram i 2D for Σ=I 2 Fidig ski Ski has a very small rage of (itesity idepedet colours, ad little texture Compute a itesity-idepedet colour measure, check if colour is i this rage, check if there is little texture (media filter See this as a classifier - we ca set up the tests by had, or lear them. get class coditioal desities (histograms, priors from data (coutig Classifier is Receiver Operatig Curve Figure from Statistical color models with applicatio to ski detectio, M.J. Joes ad J. Rehg, Proc. Computer Visio ad Patter, 999 copyright 999, IEEE Figure from Statistical color models with applicatio to ski detectio, M.J. Joes ad J. Rehg, Proc. Computer Visio ad Patter, 999 copyright 999, IEEE Appearace-Based Visio: Lessos Stregths Posig the recogitio metric i the image space rather tha a derived represetatio is more powerful tha expected. Modelig objects from may images is ot ureasoable give hardware developmets. he data (images may provide a better represetatios tha abstractios for may tasks. Appearace-Based Visio: Lessos eakesses Segmetatio or object detectio is still a issue. o trai the method, objects have to be observed uder a wide rage of coditios (e.g. pose, lightig, shape deformatio. Limited power to extrapolate or geeralize (abstract to ovel coditios.
Model-Based Visio A Rough Spectrum Appearace-Based (Eigeface, Fisherface Shape Cotexts Geometric Ivariats Image Abstractios/ Volumetric Primitives Give 3-D models of each object Detect image features (ofte edges, lie segmets, coic sectios Establish correspodece betwee model &image features Estimate pose Cosistecy of projected model with image. Local Features + Spatial Relatios Aspect Graphs 3-D Model-Based Fuctio by Hypothesize ad est Geeral idea Hypothesize object idetity ad pose Recover camera parameters (widely kow as backprojectio Reder object usig camera parameters Compare to image Issues where do the hypotheses come from? How do we compare to image (verificatio? Simplest approach Costruct a correspodece for all object features to every correctly sized subset of image poits hese are the hypotheses Expesive search, which is also redudat. Correspodeces betwee image features ad model features are ot idepedet. A small umber of correspodeces yields a camera matrix --- the others correspodeces must be cosistet with this. Pose cosistecy Strategy: Geerate hypotheses usig small umbers of correspodeces (e.g. triples of poits for a calibrated perspective camera, etc., etc. Backproject ad verify Scee Iterpretatio he Swig Fragoard, 766 Fial Exam Closed book Oe cheat sheet Sigle piece of paper, hadwritte, o photocopyig, o physical cut & paste. you ca start with sheet from the midterm, if you wat. hat to study Basically material preseted i class, ad supportig material from text If it was i text, but NEVER metioed i class, it is very ulikely to be o the exam Questio style: Short aswer Some loger problems to be worked out.