C 0 C 1. p 1 (x 0,y 0 ) p 2 I 1. p 3. (x 0,y 0,d) C 2 C 3 I 2 I 3

Transcription

1 c 1998 IEEE. Proc. of In. Conference on Compuer Vision, Bombai, January A Maximum-Flow Formulaion of he N-camera Sereo Corresponence Problem Sebasien Roy Ingemar J. Cox NEC Research Insiue Inepenence Way Princeon, NJ 85, U.S.A. Absrac This paper escribes a new algorihm for solving he N-camera sereo corresponence problem by ransforming i ino a maximum-ow problem. Once solve, he minimum-cu associae o he maximum- ow yiels a ispariy surface for he whole image a once. This global approach o sereo analysis provies a more accurae an coheren eph map han he raiional line-by-line sereo. Moreover, he opimaliy of he eph surface is guaranee an can be shown o be a generalizaion of he ynamic programming approach ha is wiely use in sanar sereo. Resuls show improve eph esimaion as well as beer hanling of eph isconinuiies. While he wors case running ime is O(n log(n)), he observe average running ime is O(n 1: 1: ) for an image size of n pixels an eph resoluion. 1 Inroucion I is well known ha eph relae isplacemens in sereo pairs always occur along lines associae o he camera moion, he epipolar lines. These lines reuce he sereo corresponence problem o one imension an he orering consrain allows ynamic programming o be applie [1{]. However, i is clear ha his reucion o 1- is an oversimplicaion of he problem ha is primarily necessary for compuaional eciency. The soluions obaine on consecuive epipolar lines can vary signicanly an creae arifacs across epipolar lines, especially aecing objec bounaries ha are perpenicular o he epipolar lines (e.g. verical bounary wih horizonal epipolar lines). In his paper, we aress he full - problem, replacing he raiional orering consrain wih he more general local coherence consrain. To perform he global - opimizaion, we cas he sereo corresponence problem as a maximum-ow problem in a graph an show how he associae minimum-cu can be inerpree as a ispariy surface. While he heoreical compuaional complexiy is signicanly higher for maximum-ow han ynamic programming, in pracice, he average case performance is similar. We also show how his new paraigm can suppor boh binocular an N-camera sereo conguraions. Sebasien Roy is visiing from Universie e Monreal, Deparemen 'informaique e e recherche operaionnelle, C.P. 618, Succ. Cenre-Ville, Monreal, Quebec, HC J7 There have been several earlier aemps o relae he soluions of consecuive epipolar lines mache wih ynamic programming. In [], ynamic programming is use o rs mach epipolar lines an hen ieraively improve he soluions obaine by using verical eges as reference. In [], a probabilisic approach is use o relae he iniviual machings obaine by ynamic programming o improve he eph map qualiy. As a rs approach, he curren line maching uses he previous epipolar line soluion o improve is own soluion. However, his inrouces a non-esirable verical asymmery. A secon approach is o ieraively improve each epipolar line soluions wih is neighboring lines soluion. While his local approach is no globally opimal, i provies an ecien way o inrouce smoohness consrain across epipolar lines. In [5], a Bayesian approach o he sereo corresponence problem is escribe. The resuling opimizaion problem can be solve ecienly by using ynamic programming along epipolar lines, resuling in he same problem as [, ] of relaing he inepenen soluions. I proposes a heurisic meho calle ierae sochasic ynamic programming ha uses previously compue ajacen epipolar line soluions o ieraively improve ranomly selece soluions. This approach is no opimal an furher more inrouce a large amoun of smoohness ha ens o blur eph isconinuiies. Some muliple camera algorihms have been presene (see [, 6{8]). In [6], a pair of camera is use as a reference or base pair. Oher cameras provie exra informaion o enrich he maching cos funcion of he reference camera pair. The maching hen procee using ynamic programming as in []. In [7] an [8], a muliple-camera real-ime sereo sysem is presene. They use a single reference camera o perform he maching. All he oher cameras provie he informaion perinen o each possible eph of poins in he reference image. While each pixel is inepenenly solve for eph, an implici smoohness consrain is enforce by smoohing he images before processing hem. Secion escribes a general N-camera sereo framework o be use wih muliple images from arbirary viewpoins. In Secion, he sereo problem is exene from maching single epipolar lines o solving for a full ispariy map. The generalizaion of he orering consrain o local coherence consrain is also escribe here. In Secion, he sereo mach-

2 C p I 1 C C 1 p p 1 (x,y ) P (x,y,) p C a poin P i is projece ono he image plane ino he projecive poin p i by he relaion p i = " xi y i = J P i z i where J is he projecion marix ene as " 1 J = 1 1 I Figure 1: Muliple-camera sereo seup. For any ispariy of poin p, you can back-projec (x ; y ; ) in each inspecion camera (C 1 ; C ; C ), obaining he se of poins p 1 ; p ; p. ing problem is formulae as a maximum-ow problem. Deails of he maximum-ow algorihm an performance issues are presene in Secion.. Experimens on boh classic wo-image an muliple-image sereo sequence are presene an iscusse in Secion 5. Sereo Framework In his secion, we presen a general framework o hanle sereo in he conex of muliple images aken uner arbirary camera geomeries. I naurally exens he raiional wo-image, single-baseline framework for sereo. A se of n inspecion cameras C 1 ; : : : ; C n provies n images I 1 ; : : : ; I n of a scene, as epice in Figure 1 (wih n = ). A base camera C provies he view for which we wish o compue he ispariy map (or equivalenly eph map) for every image poin. The base camera oes no have o provie an image; only he inspecion cameras o. In he case of Figure 1, he base camera C is ienical o inspecion camera C 1. A poin P w expresse in he worl coorinae sysem wih homogeneous coorinaes P w = [ x w y w z w 1 ] T can be ransforme o he homogeneous poin P i in he coorinae sysem of camera i by he relaion where P i = W i P w W i = Ri T i T 1 an R i an T i are, respecively, he roaion an ranslaion marices ening he posiion an orienaion of camera i. Assuming he pinhole camera moel, I From a ransforme an projece poin p i, he corresponing image coorinaes p i are obaine from he relaion p i = H(p i ) where H is an homogenizing funcion xy H(" x=h ) = h y=h During he process of sereo maching, each poin p of image I is aribue a eph z or equivalenly a ispariy (ene as = 1=z) an can be expresse as P = " p z1 = 6 x y 7 z1 5 = 6 x y 1 in he base coorinae sysem C. While hese wo formulaions are equivalen, using he ispariy formulaion allows one o express naurally poins ha reach an innie eph. Therefore, we use ispariy insea of eph z. From his poin P, i is possible o projec back o any camera image p i using he previously ene equaions as x i y i p i = H(p i ) = H(J P i ) = H(J W i P w ) = H(J W i W?1 P ) = H(J W i W?1 6 x y ) an herefore obain pixel inensiy informaion from inspecion cameras in orer o perform he maching. During he sereo maching, each base image poin p = [x ; y ] T an is ispariy value generaes a se of reprojece pixel values ha form a pixel inensiy vecor v ene as " p v(p ; ) = fi i (H(J W i W?1 1 ))g; 8i [1; : : : ; n] (1)

3 line B (a,b) (a,) b a line A a line A a (a,l,) l epipolar lines Figure : Epipolar Maching. Lef, gri of all possible maches beween line A an B. Righ, equivalen formulaion of he problem, where B oes no appear irecly. Figure : Maching whole images. All epipolar lines l are sacke ogeher so ha he whole image A is mache wih ispariy range. A poin has eph an posiion a along epipolar line l. This vecor conains all he pixel inensiy informaion from he inspecion cameras for a paricular mach. In orer o perform he acual sereo maching, a maching cos funcion is require. Ieally, i is minimum for a likely mach an large for an unlikely one. Deriving a meaningful maching cos is a non rivial ask. Since his is no he primary purpose of his paper, we will use he simple form escribe nex. If we assume ha surfaces are Lamberian (i.e. heir inensiy is inepenen of viewing irecion) hen he pixel inensiy values of v(p ; ) shoul be ienical when (p ; ) is on he surface of an objec an hus a vali mach. Then, we can ene he maching cos cos(p ; ) as he variance of he pixel inensiy vecor v(p ; ) as cos(p ; ) = 1 n X (v(p ; )? v(p ; )) ().1 Epipolar geomery an Maching I is a well known fac ha for a given camera geomery, each image poin is resrice o move along a single line calle he epipolar line []. This reuces he maching process o a 1-D search along corresponing epipolar lines. A very imporan aiional consrain is he orering consrain. I saes ha he orer of poins along corresponing epipolar lines is preserve. In fac, his correspons o enforcing a smoohness consrain along epipolar lines (also noe in []). In he raiional approach o sereo maching, a single epipolar line A is mache wih is corresponing epipolar line B in he oher image. The esablishe maching beween he wo lines is a pah in he gri of all possible maches (a; b), as shown on he lef of Figure. The allowe saring an ening posiions of he pah are shown as hick black lines. By assuming ha he orering consrain is saise along epipolar lines, i is possible o solve his pah problem very ecienly via ynamic programming [1{]. In orer o be able o use muliple cameras, he maching gri beween lines A an B can be ransforme ino he equivalen formulaion on he righ of Figure, where only line A appears irecly. For ha case, each poenial mach has he form (a; ), where a is a posiion along line A an is is associae ispariy. The coorinaes in image B corresponing o he mach (a; ) are easy o compue from Eq. 1, while he cos funcion is irecly obaine from Eq.. Given a mach (a; ) or (a; b), i is sraighforwar o map i o any number of cameras wih known geomeries an herefore use exra informaion from muliple cameras. However, he represenaion using (a; ) is favore over one using (a; b) because we o nee wo base camera (A an B) as in [6] bu only one (A). Recovering a full ispariy map A naural exension o maching a single pair of epipolar lines a a ime woul be o exen i o he whole image a once, as epice in Figure, by maching all pairs of epipolar lines simulaneously. Every minimum-cos pah ening he maching of a single epipolar line are now assemble ino a single minimum-cos surface. This surface conains all he ispariy informaion of he base image. The goal of his consrucion is o ake avanage of one very imporan propery of ispariy els, local coherence, suggesing ha ispariies en o be locally very similar, in any an all irecions. As iscusse previously, his propery is exploie along epipolar lines by enforcing he orering consrain. However, local coherence occurs in all irecions an hus across epipolar lines. By puing all he epipolar lines ogeher an solving globally for a ispariy surface, i becomes possible o ake full avanage of local coherence an improve he resuling eph map. Noe ha each poenial mach (a; l; ) in Figure is four-connece since i is par of a -D maching gri as presene in Figure. To ake full avanage of local coherence, hey have o be be six-connece o relae each iniviual epipolar line. Unforunaely, oing his makes ynamic programming unusable since here is no sric orer for builing he soluion surface. Many soluions for global ispariy surface maching have been propose [, 5, 6]. Typically, hese algorihm propose an ieraive approach in which a soluion is improve by using he previous maching obaine for neighboring epipolar lines. While his can someimes work in pracice, hese soluions are no very ecien an no opimal.

4 s y x (x,y,) foregroun Flow irecion : s 6 connece ispariy surface backgroun Figure : Image Maching as a Maximum Flow problem..1 Avoiing irec use of epipolar geomery An imporan isincion has o be mae beween he sereo maching problems epice in Figures an. In he rs case, he epipolar lines are simply sacke up one afer he oher. While his migh work for binocular sereo, i oes no exen well o he case of muliple image sereo since he epipolar lines are specic o a single pair of cameras an arbirary camera geomeries will yiel arbirary se of epipolar lines. To alleviae his problem, we iscar he orering consrain alogeher, replacing i wih he local coherence propery menione in Secion, which is similar bu more general. In his new formulaion, we can pick any se of lines in he image o be sacke ogeher. The obvious choice is o ake he se of horizonal lines since his is he naural image layou. This explains why we can refer o a poin in Figure by is image coorinaes (x ; y ) insea of he epipolar line inex l an posiion a in Figure. The epipolar geomery is now only inirecly use in compuing he maching cos for poins wih given ispariy values (in Equaion 1) bu oes no conribue as an explici consrain o he maching process. Sereo maching as a Maximum Flow problem We propose o solve globally for he ispariy surface by aing a source an a sink o he formulaion of Figure, an rea i as a ow problem in a graph, as epice in Figure. Consier he graph G = (V; E) forming a -D mesh as in Figure. The verex se V is ene as V = V [ fs; g where s is he source, is he sink, an V is he mesh V = f(x ; y ; ) : x [ : : : x max]; y [ : : : y max]; [ : : : max ]g where (x max + 1; y max + 1) is he base image size an max + 1 is he eph resoluion. Inernally he mesh is six-connece an he source s connecs o he fron plane while he back plane is connece o he sink. We have ( (u; v) V V : ku? vk = 1 E = ( s ; (x ; y ; ) ) ( (x ; y ; max ) ; ) : x [ : : : x max] y [ : : : y max] Being six-connece insea of four-connece, each verex of he new problem is no only connece o is neighbors along he epipolar line (in eph), bu also across ajacen epipolar lines (see Figure ). Since ynamic programming is no possible in his siuaion, we can insea compue he maximum-ow beween he source an sink. The se of eges ha are saurae by he maximum-ow represen a minimum-cu of he graph. This cu separaes he source an sink an eecively represens he ispariy surface sough. We ene he ege capaciies in he graph in a sraighforwar way. The maching cos is use irecly as a capaciy. Since a likely mach has a low maching cos, he corresponing ege capaciy will be low an ha ege is likely o be saurae by he maximum-ow. Inversely, a high maching cos yiels a high capaciy ege which is unlikely o be saurae. Since a verex in he graph correspon o a poenial mach, we can use Equaion o erive is maching cos. The capaciy of an ege is erive from he maching cos of he wo verices ha i links. We arbirarily ene he ege capaciy funcion c(u; v) beween verices u an v from Equaion as c(u; v) = cos(u) + cos(v) () where cos(u) is use for simpliciy insea of cos(p ; ) since u is a mach an ene by is associae poin p an ispariy. In fac, since an ege links o verices ha each represen a specic -D mach, i correspons iself o a line segmen in each inspecion image. The obvious improvemen o he ege capaciy funcion is o erive i irecly from hese line segmens. The average of wo verices maching cos is jus a heurisic ha works quie well in pracice..1 Expressing smoohness hrough ege capaciy In orer o conrol he level of smoohness of he ispariy map, i is imporan o iereniae beween wo kin of eges. As epice in Figure 5, an ege oriene along he ispariy axis is calle a ispariy ege while all oher ege orienaion are calle occlusion ege. I will be shown laer ha he capaciy of occlusion eges irecly conrols he level of smoohness. Eges ajacen o he source or sink are no classie an have innie capaciies. We have c(u; v) = 8 >< >: 1 if (u; v) = E if u = s or v = c isp (u; v) if (u? v) = (; ; ) c occ (u; v) if (u? v) = ( x ; y ; ) where c isp (u; v) is he capaciy of a ispariy ege ( oriene along he axis ) while c occ (u; v) is an occlusion ege (oriene along he x or y axis). In Figure 5,

5 s (x,y,) y 6-connece x :Occluing ege :Dispariy ege Figure 5: Expressing smoohness hrough ege capaciy. k=1 : ispariy ege : occlusion ege : min-cos pah (ynamic prog.) k = 1, he resuling ispariy surface is a (maximally smooh) an feaures a single ispariy value for he whole image. Seing k =, each column of he graph is inepenenly given a ispariy, herefore achieving maximal isconinuiy in he ispariy surface. For k = 1, a he op of Figure 6, a balance is reache an he minimum-cu correspons very well o he minimum-cos pah compue by ynamic programming.. From a cu o a ispariy surface I is well known ha once he maximum ow is foun, a minimum-cu C separaes he source an sink in such a way ha he sum of ege capaciies of C is minimize. This cu is herefore he opimal way o separae he source an he sink for he paricular cos funcion. Since he source is connece o he closes poins while he sink is connece o he eepes poins, he cu eecively separaes he view volume ino a foregroun an backgroun an yiels he eph map of he scene. The minimum cu is also guaranee o provies a eph esimae for each image poin, as emonsrae by Propery 1. s Propery 1 (cu as a eph map) Consier a cu C associae wih some ow in he graph G = (V; E). For all (x; y), here exis a leas one such ha he ege (x; y; )? (x; y; + 1) is par of C. Proof. For any (x; y), here is a pah s ; in G of he form k= s k= 8 Figure 6: Example cus for ieren smoohness values. k =, maximal isconinuiy. k = 1, inermeiae smoohness. k = 1, innie smoohness. he arker eges (connecing he black verices) are occlusion ege while ligher eges ispariy eges. We ene hese coss from Equaion as c isp (u; v) = cos(u)+cos(v) c occ (u; v) = k c isp (u; v) ( k 1) where k is a smoohness parameer. A higher occlusion cos (i.e. larger k) increases he smoohness of recovere surfaces while, inversely, a lower occlusion cos faciliae eph isconinuiies. To illusrae he eec of he smoohness parameer k, we creae an example -D problems wih a simple cos funcion, as shown in Figure 6. For reference purposes, a minimum-cos pah linking he lef an righ sies of he graph was compue using sanar ynamic programming an is isplaye as a chain of whie os. The maximum-ow was compue in his graph for smoohness values, 1, an 1 an he corresponing minimum-cu are isplaye as ses of hick black eges. These exreme values of he smoohness parameer k have inuiive consequences. When s s! (x; y; )! (x; y; 1)! : : :! (x; y; max )! herefore conaining he se of eges ( s! (x; y; ) (x; y; )! (x; y; + 1) [; max? 1] (x; y; max )! Any cu of G mus break his pah an hus conain a leas one ege of he form (x; y; )? (x; y; + 1) since he eges s! (x; y; ) an (x; y; max )! have innie capaciies. Accoring o propery 1, a eph map can be consruce from he minimum-cu C of graph G as follow. For each poin (x; y), he ispariy is he larges such ha he ege (x; y; )? (x; y; + 1) belongs o C. This resuls in he esire global ispariy surface.. Solving he Maximum Flow problem There is an abunan lieraure on algorihms o solve he maximum-ow problem [9, 1]. For his paper, we implemene a well known algorihm, preow-push lif-o-fron (see [9]). Currenly, he bes maximum-ow algorihm is presene in [1] an is paricularly well suie for sparse graphs like he ones buil for sereo maching. The number of verices v in he graph is equal o he number of image pixels muliplie by he eph resoluion. For an image of size n pixels, i.e. of imension approximaely p n p n, an a eph resoluion of )

6 ime = O(n 1: ) ime = O( 1: ) maximum flow ( ispariy seps) (18 ispariy seps) (~ ispariy seps) log(ime) log(ime) log(n) log() Figure 7: A) Performance as a funcion of image size n in pixels, for xe eph resoluion. B) Performance as a funcion of eph resoluion for a xe size n. Three oe lines show performance levels of O( p n), O(n), an O(n ). seps, we have v = n. Since he graph is a hreeimensional mesh where each verex is six-connece, he number of ege e is e = O(V ) = n. This implies ha he preow-push algorihm use, wih a running ime yiels a running ime of O(ve log(v =e)) O(n log(n)) The new bes boun [1] runs in O(e log(v =e) log(u)) where U is he larges ege capaciy, yiels a running ime of O(n 1:5 1:5 log(n) log(u)) The ynamic programming approach on separae epipolar lines [] requires a oal running ime of (n), which migh seem much beer han he maximum-ow algorihm. However, he opology of he graph, he posiion of he source an sink, an he srucure of ege capaciies all en o make he problem easier o solve, making he average running ime much beer han he wors case analysis. Figure 7 shows he ypical performance as a funcion of oal image size n (in pixels) an eph resoluion. The average running ime is O(n 1: 1: ), which is almos linear wih respec o image size n (in pixels) an compares favorably wih he ynamic programming approach. The ypical running ime for 5656 images is anywhere beween 1 o minues, on a 16Mhz penium machine, epening on he eph resoluion use. While his is consierably slower han [], he algorihm was no opimize for spee. Performance improvemen are expece in he fuure. 5 Experimens an resuls In his secions, resuls of binocular an N-camera sereoscopic maching from maximum-ow are presene an compare wih wo oher algorihms. sanar sereo shrub 15 Figure 8: Dispariy maps for he Shrub a wo precision level ( an 18 ispariy seps). On op, he maximum-ow an resuls. A boom, he original image shrub-15 an resuls for sanar sereo. Firs, he algorihm referre o as sanar sereo uses line-by-line ynamic programming on N-camera wih variable eph resoluions. I iers from he maximum-ow algorihm only in he way i solve he ispariy surface. They are oherwise ienical an heir resuls use he same ispariy scale an are no equalize. Secon, he algorihm referre o as is he ecien ynamic programming implemenaion from [] (for he binocular version) an from [6] (for he N-camera version). I performs an ieraive opimizaion of is ispariy soluion o enforce smoohness across ispariy lines. I shoul be noe ha he resuls from his algorihm use a ieren ispariy scale (gray levels) han maximum-ow or sanar sereo an are equalize o improve heir conras. Shrub Figure 8 shows one image of a pair of he Shrub image sequence (couresy of T. Kanae an T. Nakahara of CMU), along wih some maching resuls. These resuls show how maximum-ow ens o exrac sharp an precise eph isconinuiies, while sanar sereo an prouce many arifacs along verical eph isconinuiies. Two level of eph resoluions are shown ( an 18 seps) wih ieren level of smoohness. I is noable ha even a high smoohness levels, maximum-ow oes no prouce spurious horizonal links across he gap beween he wo larger shrubs. The resuls of muliple-camera analysis is shown in Figure 9. All he images of his sequence share a common horizonal baseline. Even if he algorihms use ieren number of images ( an 7), he oal spanne camera isplacemen is he same an herefore provie abou he same eph iscriminaion. Some image normalizaion is performe for prior o maching. None was use for he oher wo algorihms.

7 images (6 ispariy seps) 7 images pm images images maximum flow Figure 9: Dispariy maps for he an 7 images Shrub sequence. Boh sequences span he same oal horizonal isplacemen an shoul yiel similar resuls. Whie poins on he righ enoe eece occlusions. penagon The image sequence Park meer shown in Figure 11 was analyze for ieren number of images. Here a number of verical objecs pu in evience he iculmaximum flow sanar sereo Figure 11: Dispariy maps for he Park meer sequence. Resuls are shown for an image sequence. The resul is shown for images. casle maximum flow (7 images) (1 images) Figure 1: Dispariy maps for he Roof sequence. Resuls are shown for 7 an 1 images, respecively. Whie poins on he righ enoe eece occlusions. Figure 1: pair. Penagon maximum flow ( ispariy seps) sanar sereo Dispariy maps for he Penagon sereo The lef image of he sereo pair Penagon is shown in Figure 1, along wih he maching resuls. This sereo pair presens some challenge since he rue camera moion is no exacly horizonal an conain some roaion, creaing image moions ha violaes he epipolar consrain. Forunaely, algorihms like resis beer o hese misalignmen since hey allow negaive ispariies as well as posiive. This explains how he highway srucures a he op lef are well recovere for while he oher algorihms prouce some noiceable spurious mismach. A preice, maximum-ow oes prouce a more symmeric resul, wih less spurious horizonal sreaks. Park meer ies ha sanar sereo an have o relae horizonal epipolar lines soluions ogeher. No horizonal sreaks are presen in maximum-ow. Using images (horizonally isplace along a single baseline), he resuls a he boom of Figure 11 improve sensibly from hose a he op. No resuls were available for. Roof The image sequence "Roof" (couresy of T. Kanae an E. Kawamura of CMU) is shown on he lef of Figure 1. I conains 1 images feauring eiher horizonal or verical ranslaions. The resuls for maximum- ow an are presene a he righ. The ispariy map obaine by maximum-ow is very eaile. In paricular, he srucure of he roof is well reconsruce. Noe ha only 7 horizonally separae images were use by maximum-ow because he exac amoun of verical isplacemen of he remaining 6 images was no available. Casle The sequence Casle from CMU is shown on he lef of Figure 1 an conains 11 images wih various combinaions of horizonal, verical an forwar camera

8 c6 max flow (11 images) Figure 1: The Casle image sereo sequence. On he lef, one of he 11 images. On he righ, he resuling maximum-ow ispariy map. k = 1 k = 1/1 k = Figure 1: Dispariy maps for he Shrub sequence for smoohness levels. On he lef, k = 1 enforce high smoohness. In he mile, k = 1=1 is meium smoohness. On he righ, k = enforce no smoohness. moion. The 11 images were use o creae he ispariy map shown on he righ. A high level of eail an very few spurious maches are presen. Noice ha he whie backgroun is recovere correcly regarless of is lack of exure. I is imporan o noe ha his sequence represen a challenge since he acual ispariy range, i.e. he ierence in ispariy beween he closes an he farhes objec, is only.7 pixels. Performe a a eph resoluion of 96 seps, his implies ha he ispariy precision achieve is. pixels. 5.1 Level of Smoohness In his secion, we wish o illusrae how he level of smoohness, represene by he parameer k of Secion.1, can aec he qualiy of he ispariy map recovere. Figure 1 illusraes his for hree level of smoohness, namely k = 1, k = 1=1 an k =. For k =, he capaciy of occlusion eges is zero an herefore each pixel is given a ispariy inepenenly of is neighbors. I is essenially equivalen o ning he bes ispariy by correlaion over a single pixel winow (on he righ of Figure 1). As expece, lowering he occlusion capaciies favors eph isconinuiies an herefore creaes sharper objec eges, a he expense of surface smoohness. I is observe ha large eph isconinuiies en o say sharp as he level of smoohness increases. This is probably ue o he fac ha he smoohness is expresse in all irecion insea of only along epipolar line. This resul iers srongly from mos oher mehos where a high level of smoohness inuces blurre or missing eph isconinuiies. 6 Conclusion We presene a new algorihm for esablishing N- camera sereo corresponence, base on a reformulaion of he sereo maching problem o ning he maximum-ow in a graph. Represening a generalizaion of ynamic programming along epipolar lines o he global maching space, i is able o solve opimally for he full ispariy surface in a single sep, herefore avoiing he usual ispariy inconsisencies across neighboring epipolar lines. The orering consrain, require for ynamic programming, is replace wih a more general local coherence propery ha applies in all irecions insea of along epipolar lines. The new sereo problem formulaion naurally suppors muliple arbirary cameras an can esimae eph for an arbirary virual camera. For any esire level of smoohness, eph isconinuiies are well preserve since smoohness is applie in all irecions insea of only along epipolar lines. As for fuure research, here are many avenues open o improve he maximum-ow formulaion propose in his paper. In paricular, a muli-resoluion approach as well as local smoohness variaions coul be irecly embee in he graph, improving performance an eph map qualiy. The ege capaciy compuaion can also be improve (as iscusse a hen en of Secion ) by irecly comparing image line segmens insea of single pixels. References [1] H. H. Baker. Deph from Ege an Inensiy Base Sereo. PhD hesis, Universiy of Illinois a Urbana- Champaign, [] Y. Oha an T. Kanae. Sereo by inra- an inerscanline using ynamic programming. IEEE Trans. Paern Analysis an Machine Inelligence, 7():19{ 15, [] I. J. Cox, S. Hingorani, B. M. Maggs, an S. B. Rao. A maximum likelihoo sereo algorihm. Compuer Vision an Image Unersaning, 6():5{567, [] O. Faugeras. Three-imenional compuer vision. MIT Press, Cambrige, 199. [5] P. N. Belhumeur. A Bayesian approach o binocular sereopsis. In. J. Compuer Vision, 19():7{6, [6] I. J. Cox. A maximum likelihoo N-camera sereo algorihm. In Proc. of IEEE Conference on Compuer Vision an Paern Recogniion, pages 7{79, 199. [7] S. B. Kang, J. A. Webb, C. L. Zinick, an T. Kanae. An acive mulibaseline sereo sysem wih real-ime image acquisiion. Technical Repor CMU-CS-9-167, School of Compuer Science, Carnegie Mellon Universiy, 199. [8] T. Kanae, A. Yoshia, K. Oa, H. Kano, an M. Tanaka. A sereo machine for vieo-rae ense eph mapping an is new applicaions. In Proc. of IEEE Conference on Compuer Vision an Paern Recogniion, San Francisco, [9] T. H. Cormen, C. E. Leiserson, an R. L. Rives. Inroucion o Algorihms. McGraw-Hill, New York, 199. [1] A. V. Golberg an S. B. Rao. Lengh funcions for ow compuaions. Technical Repor 97-55, NEC Research Insiue, Princeon NJ, 1997.