The Panum Proxy algorithm for dense stereo matching over a volume of interest

Transcription

1 The Panum Proxy algorithm for ense stereo matching over a volume of interest A. Agarwal an A. Blake Microsoft Research Lt. 7 J J Thomson Ave, Cambrige, CB3 FB, UK Abstract Stereo matching algorithms conventionally match over a range of isparities sufficient to encompass all visible 3D scene points. Human vision however oes not o this. It works over a narrow ban of isparities Panum s fusional ban whose typical range may be as little as 1/2 of the full range of isparities for visible points. Points insie the ban are fuse visually an the remainer of points are seen as iplopic that is with ouble vision. The Panum ban restriction is important also in machine vision, both with active (pan/tilt) cameras, an with high resolution cameras an igital pan/tilt. A probabilistic approach is presente for ense stereo matching uner the Panum ban restriction. First it is shown that existing ense stereo algorithms are inaequate in this problem setting. Seconly it is shown that the main problem is segmentation, separating the (left) image into the areas that fall respectively insie an outsie the ban. Thirly, an approximation is erive that makes up for missing out-of-ban information with a proxy base on image autocorrelation. Lastly it is shown that the Panum Proxy algorithm achieves accuracy close to what can be obtaine when the full isparity ban is available. 1. Introuction In attentional stereo vision, the viewer steers a volume of interest aroun the scene. This is a problem that has receive a goo eal of attention in the realms of oculomotor control [5, 8] an sparse stereo e.g. [14]. In the area of ense stereo however e.g. [12, 6, 2, 3, 1, 15] the issue of restricting attention to a volume, with a limite range of epth or equivalently isparity, has not been aresse. It is of consierable importance from the point of view of efficiency, particularly with high resolution or hea-mounte cameras, in restricting computation to a volume of interest which may be only a small fraction of the visible volume. In principle also, it is most unsatisfying that conventional stereo algorithms nee to explore an irrelevant backgroun, simply in orer to establish significiant properties of the foregroun a form of the celebrate frame problem of Artificial Intelligence The Panum ban The geometry of the situation is illustrate in figure 1. For a particular fiel of view of each camera, potential matches between left an right images form a iamonshape region in each epipolar plane. In human vision [13] the space of possible matches is restricte further to the Panum ban (see figure). This is typically aroun 5 mra wie, an cuts own the number of possible foveal matches by aroun an orer of magnitue. High quality stereo cameras with narrow fiels of view can also benefit from a Panum ban restriction in a similar way. The motivation for stuying Panum ban stereo is then threefol. 1. It is conceptually appealing to evelop a stereo algorithm which focuses on a volume of interest, in the manner known to prevail in human vision. Why shoul a stereo algorithm expen neeless attention to the entire backgroun of a scene? 2. Computational cost for stereo matching grows linearly (or faster) with volume of interest. This is true both for both main components of stereo matching: cost computation an global optimization (whether by graph-cut (GC), ynamic programming (DP) or belief propagation (BP)). Restricting the size of the matching volume is therefore critical for efficiency. For stereo geometry similar to human vision, the saving in computational cost is at least an orer of magnitue, ue to the reuce range of epth (isparity). Usually there is a further factor of saving, ue to the concomitant restriction in image area over which matching occurs. The best stereo algorithms (GC, DP or BP [15]) o not currently come close to real time. This is not going to be solve any time soon by Moore s law because camera resolution is increasing faster than processing power. 3. Computational cost, we have argue, necessitates the restriction of stereo matching to a Panum volume. However, existing ense stereo algorithms are not capable of satisfac-

2 nright Panum Proxy Dense Stereo Matching Agarwal an Blake, Proc CVPR 26 2 stereo matching e.g. [3], but restricte to the image regions that have been labelle as in-ban. Left Right Note that the restriction to the Panum ban in the segmentation step 4 is inee essential, because the complexity of segmentation is ominate by the cost of computing stereo match scores, an this is linear in match volume. The resulting stereo isparity map can be use, for example, to synthesise a new view, as in figure 2, in which a) isparity Left b) Figure 1. The space of possible matches restricte to a Panum ban. a) View from above of rays in a single epipolar plane, forming a iamon-shape match space; the Panum ban forms a ribbon across the iamon an thus cuts own the set of possible matches. b) The match-space represents the situation in (a) in a stanarise iagram, in which the iamon-shape match-space becomes a square, whose sies are respectively a left an right epipolar line, an restricte to the Panum ban as shown. A possible matching path is shown ashe. tory operation over a Panum ban, as this paper will show. A new algorithm is neee The Panum Proxy algorithm The principle of the Panum Proxy stereo algorithm is therefore as follows. 1. Compute match scores or likelihoos for isparities within the Panum ban. 2. Aggregate those scores to compute a total likelihoo, at each point, that there is a within-ban (foregroun) match. 3. The same cannot be one for the backgroun likelihoo, as that woul require match scores outsie the ban. However, it is shown that an autocorrelation-like measure can be use to estimate the backgroun likelihoo. 4. Use the true foregroun likelihoo an the estimate backgroun likelihoo, in a graph cut algorithm, to achieve a segmentation. 5. Once segmentation is complete, perform conventional m Figure 2. Fusion an iplopia with the Panum Proxy algorithm. Results of the Panum Proxy algorithm are illustrate here for a frame from one of the six Microsoft stereo atasets. The matche stereogram shows fusion within the Panum ban but iplopia ouble vision elsewhere. case the view is fuse within the Panum ban, but iplopic outsie it, just as in human stereo vision. 2. Probabilistic framework for stereo matching First we outline the notation for probabilistic stereo matching. Pixels in the rectifie left an right images are L = {L m } an R = {R n } respectively, an jointly we enote the two images z = (L, R). Left an right pixels are associate by any particular matching path (fig. 1). Frequently in stereo matching the so-calle orering constraint is impose, an this means that each move in figure 1b) is allowe only in the positive quarant [1, 12]. Stereo isparity is = { m, m =,..., N} an isparity is simply relate to image coorinates as m = m n. In algorithms that eal explicitly with occlusion [1, 7] an array x of state variables x = {x m }, takes values x m {M, O} accoring to whether the pixel is matche or occlue. This sets up the notation for a path in epipolar matchspace which is a sequence (( 1, x 1 ), ( 2, x 2 ),...) of isparities an states. A Gibbs energy E(z,, x; Θ, Φ) can be efine for the posterior over all epipolar paths taken together an notate (, x), given the image ata z. Parameters Φ an Θ relate respectively to prior an likelihoo terms in the posterior. Then the Gibbs energy can be glob-

3 Panum Proxy Dense Stereo Matching Agarwal an Blake, Proc CVPR 26 3 ally minimise to obtain a segmentation x an isparities Prior istribution over matching paths A Bayesian moel for the posterior istribution p(x, z) is set up as a prouct of prior an likelihoo: p(x, z) p(x, )p(z x, ). (1) The prior istribution p(x, ) exp λe (x, ) is frequently ecompose, in the interests of tractability, as a Markov moel. An MRF (Markov Ranom Fiel) prior for (x, ) is specifie as a prouct of clique potentials V m,m over all pixel pairs (m, m ) N eeme to be neighbouring in the left image. The potentials are chosen to favour matches over occlusions, to impose limits on isparity change along an epipolar line, an to favour figural continuity between matching paths in ajacent epipolar linepairs Stereo matching likelihoo The stereo likelihoo is: p(z x, ) m exp U M m (x m, m ) (2) where the pixelwise negative log-likelihoo ratio, for match vs. non-match, is { Um M M(L P (x m, m ) = m, Rn) P if x m = M (3) M if x m = O, where M(...) is a suitable measure of gooness of match between two patches, often base on normalise sumsquare ifference (SSD) or correlation scores [15]. 3. Restricting conventional stereo matching to a Panum ban We looke at two ense stereo matching algorithms which are consiere competitive [15], one referre to as BVZ [4] that uses graph-cut optimization; the other KZ also using graph-cut but also with explicit allowance for occlusion [1]. The question is whether these algorithms can be applie to the Panum problem simply by reucing the isparity range available for matching. Following conventions for stereo testing, we took the four image pairs Tsukuba, sawtooth, venus an map on the Milebury atabase 1, together with supplie groun truth, an calculate error measures. Over foregroun, an error is counte wherever compute isparity is in error by more than 1 pixel. For backgroun regions, the true isparity is of course out of range, so an incorrect isparity is consiere to be as follows: 1 Figure 3. Stereo matching error rates for the KZ algorithm constraine to the Panum ban. The error rate ata show that backgroun error is greatly magnifie when the Panum ban constraint is impose, while foregroun error barely changes. BVZ: not at the enstop of the Panum ban; KZ: wherever the state is not occlue: x m O, an the isparity is not at the enstop of the Panum ban. In each case we use the operating parameters recommene for the algorithms: for BVZ, isparity graient penalty [3] λ = 2, an for KZ, λ = 1 with occlusion penalty [1] K = 5. Results for the KZ algorithm are shown in figure 3. Results for the (simpler) BVZ algorithm are similar, but omitte here. In both cases, isparity error over foregroun regions is not much affecte by the Panum ban restriction (in fact improve slightly because of the ae constraint). Over backgroun regions, error for both algorithms rises substantially. The conventional stereo algorithms simply fail over the backgroun, generating many ranom isparities. The conclusion from this experiment is that the conventional algorithms, when restricte to the Panum ban, work perfectly well over foregroun regions. All that is require to make the algorithms usable, is reliable ientification of those pixels whose isparities fall within the ban. In other wors, successful Panum-ban stereo coul be achieve if only segmentation into foregroun (within ban) an backgroun coul be achieve reliably. Therefore the remainer of the paper consiers the problem of foregroun/backgroun segmentation uner the Panum ban constraint Can graph-cut stereo be aapte for segmentation? One possibility, for a more subtle aaptation of the existing KZ algorithm, is that its ability to label occlusions coul

4 Panum Proxy Dense Stereo Matching Agarwal an Blake, Proc CVPR 26 4 surely be more efficient, since full consieration of isparity nee then only occur within the foregroun region. Figure 4. Segmentation error from conventional graph-cut stereo. The KZ algorithm is tune here to use its occlusion labels to inicate backgroun, but error rates are very high compare with what is attainable using LGC segmentation with the full range of isparities. be extene to label backgroun points. This is reasonable because, given the restricte Panum ban, both occlusions an backgroun points represent failures to obtain a stereo match. In orer to give the KZ algorithm every chance of success, parameter value K was explore to minimise labelling error rate an this yiele parameters λ = 1, K = 1, quite ifferent from the optimal operating point for regular use of KZ for stereo matching. Results are given in figure 4, showing segmentation error for each of the six test vieos in the Microsoft stereo-segmentation atabase 2. Labelling error-rates (equal error-rate) for the 6 atasets vary between 7% an 41%, an are in all cases many times worse than are obtainable from full, unconstraine stereo segmentation, in the form of LGC (Layere Graph Cut) [9]. Since the aim, with Panum-ban stereo, is to approach the quality of full, unconstraine stereo, the performance of KZ in this moe is far from acceptable Segmentation of the in-ban image region Given the results an iscussion so far, the aim of the remainer of the paper is to evelop a segmentation algorithm, to label all foregroun points with an accuracy approaching full LGC, but without any computation out of the Panum ban. Segmentation coul be one in one of two ways. Either it coul procee simultaneously with computation of isparity; or in a separate pass, preceing the computation of isparity. Simultaneous segmentation an isparity etermination perhaps has the attraction of greater elegance. On the other han, separate segmentation coul be achieve by marginalising the stereo likelihoo over isparities, an then performing energy minimisation with respect to labels x only. A separate labelling pass shoul 2 research.microsoft.com/vision/cambrige/i2i 4. Stereo segmentation First we summarise the full LGC (Layere Graph Cut) algorithm [9] for segmentation by marginalisation of stereo likelihoos. Then in the next section the full LGC energy function is approximate to stay within the Panum ban restriction. For LGC, the matche state M is further subivie into foregroun match F an backgroun match B. LGC etermines segmentation x as the minimum of an energy function E(z, x; Θ), in which stereo isparity oes not appear explicitly. Instea, the stereo match likelihoo (2) in section 2.2 is marginalise over isparity, aggregating support from each putative match, to give a likelihoo p(l x, R) for each of the three label-types occurring in x: foregroun, backgroun an occlusion (F, B, O). Segmentation is therefore a ternary problem, an it can be solve (approximately) by iterative application of a binary graph-cut algorithm, augmente for a multi-label problem by so-calle α-expansion [4]. The energy function for LGC is compose of two terms: E(z, x; Θ, Φ) = V (z, x; Θ) + U S (z, x, Φ) (4) representing energies for spatial coherence/contrast an stereo likelihoo Encouraging coherence The coherence energy V (z, x; Θ) is a sum, over cliques, of pairwise energies with potential coefficients F m,m now efine as follows. Cliques consist of horizontal, vertical an iagonal neighbours on the square gri of pixels. For vertical an iagonal cliques it acts as a switch active across a transition in or out of the foregroun state: F m,m [x, x ] = γ if exactly one of the variables x, x equals F, an F m,m [x, x ] = otherwise. Horizontal cliques, along epipolar lines, inherit the same cost structure, except that certain transitions are isallowe on geometric grouns. These constraints are impose via infinite cost penalties: F m,m [x = F, x = O] = ; F m,m [x = O, x = B] =. where [9] γ = log(2 W M W O ) an parameters W M an W O are the mean withs (in pixels) of matche an occlue regions respectively Encouraging bounaries where contrast is high A tenency for segmentation bounaries in images to align with contours of high contrast is achieve by efining prior penalties F k,k which are suppresse where image

5 Panum Proxy Dense Stereo Matching Agarwal an Blake, Proc CVPR 26 5 contrast is high [3, 4, 11], multiplying them by a iscount factor C m,m (L m, L m ) which suppresses the penalty by a factor ɛ/(1 + ɛ) wherever the contrast across (L m, L m ) is high see [9] for etails. Previously, maximal iscounting has been obtaine [3] by setting ɛ =. Here, as in stereo segmentation [1], ɛ = 1 tens to give the best results, though sensitivity to the precise value of ɛ is relatively mil Foregroun likelihoo The remaining term in (4) is U S (z, x) which captures the influence of stereo matching likelihoo on the probability of a particular segmentation. It is efine to be U S (z, x) = Figure 5. Segmentation error using a simple threshol in place Um(x S of backgroun likelihoo. Error curves are shown as a function of threshol θ for six subjects from the Microsoft atabase. m ) (5) m (Error-rates are total foregroun an backgroun error, average where Um(x S over each sequence.) Horizontal ashe lines show corresponing m ) = log p(l m x m = F, R). (6) error rates for full (non-panum) LGC segmentation, as a benchmark. The substantial shortfall suggests that it shoul be possible Now, marginalising out isparity, foregroun likelihoo is p(l m x m = F, R) = to improve consierably on simple thresholing. p(l m m =, R)p( m = x m = F) (7) 4.5. A simple threshol as proxy for the backgroun where, from (2), likelihoo? p(l m m =, R) f(l,, R) = exp U M m (x m, m ), (8) using the log-likelihoo ratio efine in (3). As a shorthan, we write: p(l F) = p(l, R)p( F) (9) an, as before, in terms of likelihoo ratios, this becomes: L(L F) p(l F) p(l O) = f(l,, R)p( F) (1) where f(l,, R) is the match/non-match likelihoo ratio as above Backgroun likelihoo Since the istribution p( m = x m = F) is efine to be zero outsie the Panum fusional area, it is perfectly possible, uner the Panum assumptions, to compute L(L F) in (1). However, the same cannot be sai for L(L B) p(l B) p(l O) = L(L )p( B) (11) since the corresponing summation is entirely outsie the Panum ban D F of isparities, in that p( B) is non-zero only outsie the Panum ban. Each pixel L m woul therefore have to be compare with pixels in the right image R that are unreachable because they are outsie the ban. Before going to some trouble to approximate the backgroun likelihoo, it is worth looking at the simplest possible approach, an treating the problem as novelty etection. In that view, we have a moel L(L F) for the positive class, an no moel of the backgroun class. Then the likelihoo ratio classifier L(L F) > L(L B) is simplifie to a threshol rule, replacing the backgroun likelihoo by a constant L(L B) = θ. Segmentation uner this moel, for variable threshol θ, is exhibite in figure 5. It appears that a constant threshol θ = 1 yiels close to the best error for each of the 6 atasets, so there woul be no nee for an aaptive algorithm. However, the best error rate achieve is between 2 an 8 times higher than the error achieve (ashe lines) by full LGC. Again, therefore, there is strong motivation to look for a moel an an algorithm that performs better uner the Panum-ban restriction. 5. The Panum Proxy algorithm In the previous section, it was shown that the Panumban constraint means that information require for computing backgroun likelihoo L(L B) is missing, an that replacing L(L B) with a simple threshol constant gives poor results. Therefore in this section an approximation for L(L B) is evelope Deriving the approximate likelihoo We assume that p( F) is uniform over the Panum ban so that p( F) = 1/ D F an similarly, for the backgroun,

6 Panum Proxy Dense Stereo Matching Agarwal an Blake, Proc CVPR B F 4 3 B F 4 3 B F f(,l, R) 2 1 f(,l, R) 2 1 f(,l, R) 2 1 f(,l, L) f(,l, L) f(,l, L) a) b) c) Figure 6. Using the left image as a proxy for the right, to approximate likelihoos. Upper: likelihoo function f(l,, R) for three sample image points; Panum ban is 3 6. Lower: autocorrelation-like proxy f(l,, L) for the same three sample points. In the first two examples a,b), the proxy satisfactorily mimics the shape of the true likelihoo. In c) the sample happens to fall on a textureless area, with consequent stereo ambiguity, an the proxy fails, in that f(l,, L) has a ominant peak whereas there is none in f(l,, R). A test will be evelope for this case. p( B) = 1/ D B. Then, efining D = D B D F, we can write S(L) D f(l,, R) (12) = D F L(L F) + D B L(L B), (13) from (1) an (11). If S(L) were known, then it woul be possible, having compute L(L F) as in (1), to compute L(L B) from the constraint (12). Of course S in (12) cannot be compute exactly, because the summation extens outsie the Panum ban. However, an this is the key iea of the Panum Proxy, we can approximate it by using the left image L as a proxy for the right image R in the match likelihoo ratio: S(L) = S = S f(l,, L) (14) see figure 6 for iagrams illustrating how this works. The approximation rests on the assumption that each match is a goo one, since it is matching the left image with itself. Note that the value of f(l,, L) at = is an upper boun on the value of f(l,, R) at the true match value of, since the match of the left image irectly onto itself is of course perfect; S has to be chosen just big enough to capture the peak of the match-likelihoo, but it is reasonably assume that S D F so that the aitional work in computing (14) is smaller than the amount of matching work one alreay in the Panum ban. Note that a factor of 2 can be save in computing (14) by exploiting the symmetry of autocorrelation, that is that f(l m,, L) = f(l m+,, L). Finally, having estimate S(L), we can estimate the backgroun likelihoo-ratio from the approximate constraint S(L) = D F L(L F) + D B L(L B). (15) ( ) giving L(L B) = S(L) DF L(L F) / D B. (16) 5.2. Complementary likelihoo Now given the weakness of evience, resulting from the Panum ban restriction, for istinguishing backgroun match from occlusion, we o not attempt to istinguish the hypotheses B an O. Therefore we lump them together as the complementary hypothesis F = B O, so that p(l F)p(F) = p(l B)p(B) + p(l O)p(O), (17) an again iviing by p(l O): L(L F)p(F) = L(L B)p(B) + p(o). (18) an this is expresse as L(L F) = (1 ν)l(l B)p(B) + ν, (19) where ν = p(o)/(1 p(f)), for which a typical value woul be ν =.1, reflecting the empirical fact that normally a small proportion of backgroun points are occlue.

7 Panum Proxy Dense Stereo Matching Agarwal an Blake, Proc CVPR Kurtosis test Earlier in figure 6, we saw that although f(l,, L) is often a goo preictor of the shape of f(l,, R), as in figure 6a,b), it can fail where there is no clear peak in f(l,, L), as in figure 6c). The kurtosis k = k(l, L), of f(l,, L) as a function of, is compute as a iagnostic. Figure 7 shows that high kurtosis is associate with low error in the proxy. Therefore the likelihoo estimate is preicte to be reliable if k > k. Mean segmentation error (%) LGC Full LGC Full+ colour LGC PP LGC PP + colour 9.5% AC IU IU JW JM MS VK Figure 8. Segmentation error for Panum Proxy approaches that of full stereo. The Panum Proxy (LGC PP) algorithm achieves error rates that approach very nearly the level achieve by LGC segmentation with the full range of isparities (LGC full), at consierably reuce computational cost. For one test set (MS) the error rate for (LGC PP) is relatively high, though this is restore by aing in colour information. Figure 7. Kurtosis of f(l,, L) as an inication of proxy accuracy. High kurtosis k is associate with reuce magnitue of error S S, suggesting a valiity check base on kurtosis. In fact low kurtosis occurs in practice over relatively textureless image areas, just the situation that gives rise to ambiguous isparity, as in figure 6c). A threshol value of k = 2.5 has prove effective, catching 86% of points on the tails of the error istribution (efine to be those outsie 1 stanar eviation). Then the efinition of S from (14) is replace by S S(L) = r(k) f(l,, L) + (1 r(k)) D L(L F ), = S (2) where r(k) is soft threshol function, taking the value r(k) = 1 when k an r(k) = when k. In this way, the estimate complementary likelihoo L(L F) (19) is unchange in the reliable case r(k) = 1. In the unreliable case r(k) =, S(L) = D L(L F ), an L(L B) = L(L F) from (16), an then (19) efaults towars the no-information conition L(L F) = L(L F) as r(k) Positivity check The other conition that must be ealt with is the possible negativity of the estimate L(L B) (16). In the case of negativity, we simply replace (16) with L(L B) = L(L F)/η (21) an use this to evaluate the complementary hypothesis (19). The value of η is set using the statistics of L(L F)/L(L B) in the negativity conition, collecte from a variety of images, an this gives a working value of η = Results First we show mean error rates, average over the entire stereo vieo sequence for each of the six subjects from the Microsoft atabase. These are extensive tests, representing measurements taken from several hunre stereo pairs. Figure 8 shows that error for the Panum Proxy algorithm approaches quite closely that for full Layere Graph Cut (LGC), with unrestricte stereo isparity. Note that error rates are mostly an orer of magnitue better than for the conventional graph cut stereo algorithm KZ (figure 4). Compare with the naive thresholing scheme (figure 5), in which backgroun likelihoo is replace by a constant, error rates for the Panum Proxy algorithm are lower by factors ranging from 1.5 to 7 across the six subjects. Error rates fall even further when colour information is use in the segmentation following the paraigm, use in LGC [9]. These results can be examine in more etail along their timelines, an we show this here just for the VK ataset, in figures 9 an 1. Discussion We have shown that stereo within a Panumban can be solve effectively using a conventional stereo algorithm, together with a pre-segmentation step that selects those pixels that are within the ban. This allows stereo to operate within a volume of interest, fusing over that volume, an with iplopic vision elsewhere, as in figure 2. Results

8 Panum Proxy Dense Stereo Matching Agarwal an Blake, Proc CVPR 26 8 [3] Y.Y. Boykov an M-P. Jolly. Interactive graph cuts for optimal bounary an region segmentation of objects in N-D images. In Proc. Int. Conf. on Computer Vision, pages , 21. 1, 2, 3, 5 [4] Y.Y. Boykov, O. Veksler, an R.D. Zabih. Fast approximate energy minimization via graph cuts. IEEE Trans. on Pattern Analysis an Machine Intelligence, 23(11), 21. 3, 4, 5 [5] C.M. Brown, D. Coombs, an J. Soong. Real-time smooth pursuit tracking. In A. Blake an A.L. Yuille, eitors, Active Vision, pages MIT, Figure 9. Segmentation error for Panum Proxy, over time, for one subject (VK). The Panum Proxy (LGC PP) algorithm achieves error rates close to those achieve by LGC segmentation with the full range of isparities (LGC full), especially when colour information is fuse in with stereo. See also figure 1. [6] I.J. Cox, S.L. Hingorani, an S.B. Rao. A maximum likelihoo stereo algorithm. Computer vision an image unerstaning, 63(3): , [7] A. Criminisi, J. Shotton, A. Blake, an P.H.S. Torr. Gaze manipulation for one to one teleconferencing. In Proc. Int. Conf. on Computer Vision, pages , [8] T. Uhlin K. Pahlavan an J-O. Eklunh. Dynamic fixation. In Proc. Int. Conf. on Computer Vision, pages , Best case: frame 12 Worst case: frame 9 Dataset VK Figure 1. Some examples of segmentation. Segmentations are shown for the frames with lowest an highest error, for the ataset of fig 9. Results are for LGC PP plus colour. of the Panum Proxy algorithm are close in quality to what is obtainable uner unconstraine conitions, using the full range of available isparity. It remains for future work to test the algorithm uner more stringent circumstances, with greater ranges of isparity in the scenes. Acknowlegements We acknowlege helpful iscussions with A. Criminisi, A. Fitzgibbon an V. Kolmogorov. References [1] H.H. Baker an T.O. Binfor. Depth from ege an intensity base stereo. In Proc. Int. Joint Conf. Artificial Intelligence, pages , [2] P.N. Belhumeur, J.P. Hespanha, an D.J. Kriegman. Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. In Proc. European Conf. Computer Vision, number 8 in Lecture notes in computer science, pages Springer-Verlag, [9] V. Kolmogorov, A. Criminisi, A. Blake, G. Cross, an C. Rother. Bi-layer segmentation of binocular stereo vieo. In Proc. Conf. Computer Vision an Pattern Recognition, 25. 4, 5, 7 [1] V. Kolmogorov an R. Zabih. Computing visual corresponences with occlusions using graph cuts. In Proc. Int. Conf. on Computer Vision, 21. 1, 2, 3, 5 [11] V. Kolmogorov an R. Zabih. Multi-camera scene reconstruction via graph cuts. In Proc. European Conf. Computer Vision, pages 82 96, [12] Y. Ohta an T. Kanae. Stereo by intra- an interscan line search using ynamic programming. IEEE Trans. on Pattern Analysis an Machine Intelligence, 7(2): , , 2 [13] T. Poggio an W. Reichart. Visual control of orientation behaviour in the fly. Quart. Rev. Biophys., 9(3): , [14] I.D. Rei an D.W. Murray. Tracking foveate corner clusters using affine structure. In Proc. Int. Conf. on Computer Vision, pages 76 83, [15] D. Scharstein an R. Szeliski. A taxonomy an evaluation of ense two-frame stereo corresponence algorithms. Int. J. Computer Vision, 47(1 3):7 42, 22. 1, 3