Theoretical Analysis of Graphcut Textures

Transcription

1 Theoretical Analysis o Grahcut Textures Xuejie Qin Yee-Hong Yang {xu yang}@cs.ualberta.ca Deartment o omuting Science University o Alberta Abstract Since the aer was ublished in SIGGRAPH 2003 the grahcut textures has become a oular and imortant techniue or both static and dynamic image texture synthesis. However the discussion on the theoretic side o the grahcut textures aears lacking with little inormation rovided in the original aer to convince the reader the correctness o the aroach. This aer addresses the imortant theoretic issues related to the techniue o grahcut textures using the comlete exansion grah which is an extension o the exansion grah. The main contribution o this aer is to correct or clariy some statements in the original grahcut textures aer and to give mathematical suort and vigorous roos or the aroach. Index Terms Labeling roblem grah cuts comlete exansion moves grahcut textures. Introduction Image texture synthesis i.e. texture synthesis in 2D sace) is an active and imortant research toic in comuter grahics. Given an inut texture samle the goal o texture synthesis is to generate a new image whose textures look similar to the inut texture. Among the various

2 aroaches the atch-based texture synthesis techniue [3 5 6] generates a synthesized image outut) by coying small atches rom the inut image to the outut. The challenging roblem in this aroach is how to remove or reduce) the visible seams between atches when they are laced into the outut. As a oular and imortant atch-based texture synthesis techniue the grahcut textures method [5] treats the roblem o image texture synthesis as a min-cost grah cut roblem and emloys grah cuts e.g. [2 8]) to remove or reduce) the visible seams between atches so that the outut results can be reined as reuired. In act grahcut textures is a grah-based texture synthesis techniue. Although the techniue o grahcut textures has been successully alied to image and video texture synthesis [5] there is a lack o a theoretical analysis o the techniue which is deinitely desirable in both understanding and in using the techniue or uture research. We oint out that it is not straightorward to give such a theoretical analysis. According to the general ramework o grah cuts roosed by Boykov et al. [2 8] deending on dierent roblems i.e. alications) each grah-based energy minimization method reuires a dierent mathematical ormulation o its grah in the sense that the energy unction must be deined dierently and that the design o a roer energy unction to encode the constraints o a roblem is not trivial [2 8]. In addition the cuts and the min-cost cut o the grah have dierent mathematical roerties or dierent roblems which lay a crucial role in establishing the 2

3 euivalence relationshi i.e. the key to solving a grah cut roblem [2 8]) between the set o certain tye o cuts and the set o all labelings o the grah. Furthermore secial consideration must be taken into account in assigning a correct cost or each edge in the grah as suggested by Veksler in her Ph.D. thesis [8]. For examle in the exansion grah when assigning a cost to an edge it is stated that We had to develo a secial trick or the case when the original labels or and do not agree ) in order to get the same eect that lemma 6 establishes or the simler situation when = [8]. More recisely to give a theoretical analysis o grahcut textures one has to address the ollowing imortant issues: ) what is the labeling roblem o grahcut textures? Namely what are the set o sites the set o labels the set o labelings and the energy unction o a labeling? 2) What tye o grah is reuired or the labeling roblem? 3) How to construct the grah? Namely what are the vertices the edges and the costs o edges in the grah? 4) What mathematical roerties a cut o the grah has? 5) What seciic roerties a min-cost cut has? 6) Given a cut how to deine its corresonding labeling? 7) Is it true that inding the otimal labeling is euivalent to inding the min-cost cut in the grah? Excet or the limited inormation rovided on how to construct the grah none o the above imortant theoretical issues has been addressed or well addressed) in the aer o grahcut textures [5] or in any other ublicly available ublications that we are aware o. 3

4 Furthermore there are some mathematical errors and conusions in the statements made regarding the grahs o grahcut textures see the next section or the details) in the original aer which must be clariied. This aer addresses the above imortant issues in order to rovide a comlete and correct theoretical analysis o grahcut textures. The mathematical errors and conusions in Kwatra et al. s original aer [5] are also clariied in this aer. To achieve this goal we roose a new tye o exansion grahs called comlete exansion grahs which is an extension o the exansion grahs introduced by Boykov et al. [2 8]. Using this new tye o exansion grahs all o the above imortant theoretical issues related to grahcut textures can be well ormulated. It is shown that the objective o grahcut textures is to ind an otimal labeling within a comlete exansion move rom the initial labeling. The major theorem roved in this aer to suort the grahcut textures is as ollows: in a comlete exansion grah there is a one to one corresondence between the set o all elementary cuts and the set o all labelings within one comlete exansion move o an initial labeling. This theorem indicates that inding the otimal labeling o grahcut textures is euivalent to inding the mincost cut in a comlete exansion grah which can be eiciently comuted by using grah cut techniues [ ]. 4

5 Another contribution o this aer is to rovide a novel examle o alying the general ramework o the grah-based otimization methods roosed by Boykov Veksler and Zabih [2 8] to solve an alication-deendent roblem. The rest o the aer is structured as ollows. Section 2 rovides the background knowledge on grah cuts and grahcut textures. Section 3 gives the mathematical analysis and roos or grahcut textures using the comlete exansion grahs. Finally conclusions are drawn in Section 4. 2 Background Knowledge 2. Grah uts In this aer we use G =< V E > to reresent a weighted grah where V and E are the set o vertices and the set o edges resectively. There are two secial vertices called the terminals) in the grah: the source and the sink. A cut E is a set o edges such that the terminals are searated in the induced grah G ) =< V E > and no roer subset o searates the terminals in G ). The cost o a cut denoted by is deined as the sum o its edge weights. A min-cost cut in the grah is a cut with the smallest cost which can be eiciently comuted using the max-low theorem [4 7]. 5

6 Among the various otimization techniues o grah cuts [] the methods introduced by Boykov Vesler and Zabih [2 8] have become oular or eiciently solving labeling roblems in comuter vision and comuter grahics. In a labeling roblem a set o sites P and a set o labels L are given the objective is to ind the global or nearly global) otimal labeling a labeling is a ma rom P to L) which minimizes the energy o. In general as indicated by Veksler in her Ph.D. thesis the design o a good energy unction is not trivial [8] and the energy unction must roerly incororates the constraints o a roblem [2 8]. Another major diiculty in solving a labeling roblem is that generally the roblem is intractable. For examle or an image o size i.e. 024 sites) with 256 gray levels i.e. 256 labels) or each ixel there are labelings. Boykov Vesler and Zabih [2 8] have shown that grah cuts can be used to eiciently ind the global or nearly global otimal solutions or labeling roblems with energy unctions that are everywhere smooth iecewise constant or iecewise smooth. The key idea o using grah cuts to solve a labeling roblem is to construct a weighted grah in a way such that there is a one to one corresondence between the set o certain tye o cuts in the grah and the set o all labelings that are in the move sace o an initial labeling [2 8]. With the one to one corresondence established the labeling roblem is converted to ind the min-cost cut in the grah [2 8]. However both the construction o the grah and the establishment o the one to one corresondence are roblem-deendent and secial tricks have to be develoed as suggested by 6

7 Veksler [8]. In act there are similar diicult situations in grahcut textures [8] which are described below. 2.2 Grahcut Textures The grahcut textures [5] is one o the state-o-art techniues in atch-based texture synthesis e.g. [3 6]). Given an inut texture image the atch-based texture synthesis generates an outut texture image that is oten larger than and looks similar to the inut texture by coying small atches rom the inut to the outut. Figure gives an examle o a simle atch-based texture synthesis in which small atches are randomly chosen rom the inut and tiled together in the outut. The roblem o this simle method is that there are noticeable seams between the tiled atches see Figure ). In act the main goal o the atch-based texture synthesis is to ind techniues to remove or reduce) the visible seams between atches when they are laced into the outut. In grahcut textures [5] the techniue o grah cuts roosed by Boykov et al. [2] is used to remove the visible seams between atches. small atch visible seams inut texture random selection and tiling Figure : An examle o simle atch-based texture synthesis. 7

8 Instead o tiling along their boundaries atches are laced into the outut with overlas i.e. a new atch is laced into the outut such that it artially overlas with existing i.e. old) atches. To ensure that the textures in the newly laced atch maximally agree with those in the existing atches in the overla region a minimum error ath in the overla region i.e. a ath through the ixels in the overla region where the new and old atches have the smallest intensity dierence see Figure 2 a)) is calculated and used as a cut line between the new and old atches. overla region atch atch A B minimum error ath a) atch A r min-cost cut x y z u v w b) atch B Figure 2: An examle o grah ormulation o inding the minimum error ath. The roblem o inding the minimum error ath in the overla region can be considered as a grah cut roblem. Suose a new atch B is inserted into the outut that overlas with an existing atch A. A simle grah can be constructed as shown in Figure 2 b). The existing atch in the outut is reresented by a source node with label atch A in the let and the new atch is reresented by a sink node with label atch B in the right. Both the source node and the sink 8

9 node are called terminal nodes. For simlicity reasons it is assumed that there are only 9 ixels in the overla region between the new atch B and the existing atch A. For each edge e in the overla region where and are the neighboring nodes connected by the edge a weighted cost w A B) is assigned to it which is deined as: w A B) = A ) B ) + A ) B ) ) where A ) is the gray level o ixel rom atch A and B ) is the gray level o rom atch B. I an edge between a terminal node A or B) and a non-terminal node is assigned an ininite cost the non-terminal node will be orced to assume the label rom the atch reresented by the terminal node. For examle in Figure 2 both edges e A and e zb have costs which imly that node retains its old atch label and node z is assigned to the new atch B. The minimum error ath in Figure 2 a) is euivalent to the min-cost cut o the grah shown in Figure 2 b) which can be calculated eiciently using standard grah cuts techniues e.g. [4 7]). The above grah cut roblem does not consider the visible seams between the old atches called old seams) in the outut. To remove or reduce) the old seams when laying down a new atch into the outut a dierent tye o grah is constructed as illustrated in Figure 3. The old atches which are already laced down in the outut are reresented by the source node with the label existing atches A and the new atch to be inserted is reresented by the sink node 9

10 with the label new atch B. Let A reresent the articular atch that a ixel in the overla region comes rom. For each air o neighboring ixels and in the overla region i A A i.e. and come rom dierent existing atches) then there is an old seam between and and a seam node s is created between and. Three edges e s e s and e sb are also created at node s each with dierent weights assigned to them which are w A B) w B A ) and w A A ) see E. or the deinition o w.) ) resectively. For the case o A = A no inormation is rovided in Kwatra s aer [5]. In act in this case node and have the same initial atch label and a weight o w A B) should be assigned to edge e as discussed in the next section. overla region existing new atches atch B A min-cost cut old seams existing atches A s r s 2 s 3 x y z u v w s 4 new atch B min-cost cut old seams Figure 3: An examle o grah with seam nodes added to incororate old seam constraints. For examle in Figure 3 there is an old seam between nodes and then a seam node s is created between them. In addition the seam node s is connected to the sink node B by a weighted edge whose weight is w A A ). The edge between and s is assigned a weight 0

11 w A B) and the edge between s and is assigned a weight w B A ). There is no old seam between node x and y which imlies x and y come rom the same old atch denoted by A = A x = A y the weight or edge e xy should be w x y A B) = A x) B x) + A y) B y). Other seam nodes shown in Figure 3 are s 2 s 3 and s 4 between nodes and y y and z and v and w resectively. Kwatra et al. [5] have argued that once the min-cost cut is calculated some old seams e.g. the one between node v and w in Figure 3) are removed rom the grah. On the other hand new seams e.g. the ones between node and r y and z y and v and u and v in Figure 3) are also introduced into the grah by the min-cost cut. However Kwatra et al. have not shown in their aer that the total seam cost i.e. the overall intensity dierence along the boundaries o atches) is reduced even with new seams introduced which imlies that the overall visible seams are reduced by the min-cost cut. Addressing the above roblem reuires the roo o the euivalence between the min-cost cut and the otimal labeling o the grah because the seam cost is reresented as the energy o a labeling in the overla region. In addition there are some mathematical errors in the statements made regarding the grahs constructed or grahcut textures see the last aragrah in Section 3. o the original aer [5]) which are misleading. First Kwatra et al. claim that there exists an euivalence relationshi between the seam cost and the min-cost cut o the grah which is inaccurate. In act

12 as roved in Theorem and orollary in the next section there exists an euivalence relationshi between the min-cost cut and the otimal labeling o the grah not the seam cost) which indicates that the labeling roblem in grahcut textures can be eiciently solved using existing grah cut techniues [ ]. The relationshi between the seam cost and the mincost cut o the grah is not meaningul. Furthermore there is no theoretical suort to demonstrate that the roosed method can eiciently solve the labeling roblem in grahcut textures using grah cuts. seam node e s s e e r s s x y z old atches A u v w new atch B an imossible situation or the min-cost cut Figure 4: An examle to demonstrate the situation when a min-cost cut cuts the edges at a seam node. It is imossible or the min-cost cut to cut any two edges at a seam node. Secondly Kwatra et al. have made an incorrect comment on the min-cost cut which says that icking two arcs originating rom a seam node is always costlier than icking just one o them hence at most one arc is icked in the min-cut. The correct situation is that the min-cost 2

13 cut can never ick two arcs rom the same seam node not to mention icking one o the two candidate arcs. As an examle shown in Figure 4 the min-cost cut cannot cut any two e.g. e s and e sb ) o the three edges e s e s and e sb at the seam node s. In act as roved in Lemma 3 and Lemma 4 in the next section can only cut either the single edge e s or e sb or none o the three edges at s and cannot even cut e s. Lastly the grah constructed or grahcut textures is not euivalent to the one used in Boykov et al. s work [2] as claimed by Kwatra et al. in their aer [5]. By simly ollowing Boykov s ramework without reormulating the grah mathematically one cannot rove the correctness o the grahcut textures articularly the imortant euivalence relationshi between the min-cost cut and the otimal labeling o the grah used in grahcut textures. In the next section we show that the grah constructed as shown in Figure 3 is in act a secial tye o exansion grah [2 8] by which we call a comlete exansion grah. We rove that there is a one to one corresondence between the set o all elementary cuts and the set o all labelings within one comlete exansion move o the initial labeling in the grah and that the otimal labeling in grahcut textures is euivalent to the min-cost cut in the grah. The mathematical roerties o the cuts the min-cost cuts the elementary cuts and the corresonding labeling o a cut in a comlete exansion grah are ormulated in details in order to rovide mathematical roos or grahcut textures. 3

14 3 Mathematical Proos The exansion grahs deined by Boykov et al. [2 8] cannot be directly used to rove the correctness o the grahs constructed or grahcut textures. The reason is that the exansion sace A in an exansion move can be any subset o the set o sites P. However in grahcut textures each time when a new atch is laid down over existing atches in the outut a comlete subregion o the new atch where there are no holes is used to relace a art o the overla region. In act we need a new tye o exansion grahs that are constructed rom comlete exansion moves whose deinition is given below. As shown later in this section this articular tye o exansion moves lays an imortant role in deining the one to one corresondence between the set o elementary cuts and the set o all labelings o the grah constructed or grahcut textures. For clarity the deinition o exansion move which is taken rom Boykov et al. s work [2 8] is restated irst. Deinition [2 8] Let P be a set o sites L be a set o labels and F be the set o all labelings where a labeling is a ma rom P to L. Given a label L a air ) F F is an exansion move i there exits A P such that = or A and = or A. 4

15 One can see that can be obtained rom by switching all labels in A to and this means that the region o all sites with labels has been exanded by. We call one exansion move rom. Deinition 2 Without the loss o generality assume our-nearest neighborhood connections are used. An exansion move rom is comlete i the set A is comlete in the sense that both the set A and its comlement A c = P \ A are connected under our-nearest neighborhood connections. a) b) c) Figure 5: Examles o comlete and non-comlete sets. In all the igures the set A is the set o all sites with black dots. The set A in a) is comlete while the set A in either b) or c) is not comlete. In b) A is connected but c A is not. In c) c A is connected but A is not. A set A is connected i any two sites in it are connected by a ath using our-nearest neighborhood connections. Examles o comlete and non-comlete sets are shown in Figure 5. The set A in Figure 5 a) is comlete since both A and c A are connected. In Figure 5 b) A is 5

16 connected but c A is not thus the set A is not comlete. Similarly the set A in Figure 5 c) is not comlete either. The comlete set deined in Deinition 2 naturally corresonds to the comlete subregion where there are no holes) o a new atch to be laced in the outut image during the synthesis o grahcut textures. With the comlete exansion deined the labeling roblem in grahcut textures [5] can be stated as ollows. Given a new atch a set o ixels P in the overla region between the existing atches and the new atch and an initial labeling the task is to ind an otimal comlete exansion move rom. We assume an initial labeling as the one shown in Figure 3 where always initializes some ixels in the overla region e.g. ixels r z and w in Figure 3) to the new atch. For any given labeling the energy o denoted by E ) is given by: E ) = w ) 2) ) N where is the label o ixel that tells which atch comes rom w ) is deined as in E. and N denotes the set o all neighboring airs { } in the overla region. The energy deined in E. 2 measures the seam cost as described in the revious section. In act i two ixels and come rom dierent atches i.e. ) then there is a seam between them. In this case we want to know how closely the textures in the two atches match with each other at ixel and. This can be measured by the color dierence between and which is exactly w ) according to E. note that a zero value o w ) 6

17 means no dierence in color and thus it is a erect match). I and come rom the same atch i.e. = ) then these is no seam between them which imlies that the measure w ) should give a zero value. This is actually true by E. and =. atch node o s e s s e e r s s x y z old atches A u v w new atch B Figure 6: The structure o the grah built or grahcut textures. The structure o the grah G =< V E > is shown in Figure 6. The set o vertices includes the terminals and as well as all the ixels P in the overla region where reresents the label or the new atch and stands or the old labels assigned to ixels in the overla region by the initial labeling. For each air o neighboring ixels and i they have dierent labels i.e. ) then an auxiliary node [2 8] called the seam node in grahcut textures) s s = is created and we call ixels and the atch nodes o the seam node s. Thereore the set o vertices is V = { } { } P s. 3) { } N 7

18 An edge that connects a ixel P to one o the terminal nodes is called a t-link [2 8] e.g. edges e and r e in Figure 6) and the set o all t-links is reresented by T. For each air o neighboring ixels { } N with = we connect them by an edge e which is called an n-link [2 8]. For each air o neighboring ixels { } N with a trile o edges is created: E = e e e } 4) { s s s where s is the seam node between and the edge e s connects to s e s connects s to and e connects s to. Edges in E = e e e } are called s-links. Thus the set o all edges in s { s s s the grah G =< V E > is: E = T E { e }. 5) { } N { } N = Table : The weights assigned to edges in the grah shown in Figure 6. Edge Weight omments t T t-link e s w ) { } N s is the seam node e s w ) between and and is the label or the new atch. e s w ) e ) { } N = w By the reuirement o the grahcut texture synthesis rocess [5] each t-link rom a ixel P is assigned a weight o ininity which revents ixel s label rom changing during comlete exansion moves see Deinition 2). The weights o n-links and t-links in the grah 8

19 are assigned such that the results in Lemma 4 - Lemma 7 can hold which are essential or the roos o Theorem and orollary described later. The weights or all edges in the grah are summarized in Table. Lemma Let s be a seam node in G =< V E > and and be the atch nodes o s. For any cut in G =< V E > e. s e s e s e e s s s r old atches A new atch B Figure 7: The smallest subgrah o the grah shown in Figure 6 that contains the terminals the node s and and all the corresonding edges among the nodes. Proo: Let G =< V E > be the smallest subgrah o G =< V E > that contains the terminals and the node s and and all the corresonding edges among the nodes see Figure 7). Now suose e s then cannot contain e s. Otherwise the cut has a roer subset e } \ ) that searates the terminals in the induced grah G ) which is a = { s E contradiction to the deinition o a cut see Section 2.). On the other hand since is a cut in G must cut each ath between the terminals which imlies that e s. Thus we have roved that e s e s. 9

20 Lemma 2 Given a seam node s in G =< V E >. Let and be its atch nodes and E be the set o s-links at s. Any cut in G =< V E > satisies exactly one o the three roerties: ) E = φ 2) E = e } and 3) I e then E = e e } else E = { }. e s { s s { s s Proo: Let G =< V E > is the subgrah o G as deined in the roo o Lemma see Figure 7). Suose φ then at least one edge in E = e e e } must be in. I E { s s s e s then cannot contain any o e s and e and thus E = e }. Otherwise there exists a roer subset e } \ ) o that searates the terminals in the induced = { s E s { s grah G ) which contradicts the deinition o a cut. I none o roerties ) and 2) is satisied by then the last roerty must hold or according to Lemma. Given an initial labeling or the ixels in the overla region the objective is to ind an otimal comlete exansion move rom. Now we rove that the otimal comlete exansion move is just the min-cost cut in the grah G =< V E >. Beore roving this we give the deinition o an elementary cut in a grah. Deinition 3 An elementary cut in G =< V E > is a cut that cuts at most one o the three s-links in E = e e e } at any seam node s. { s s s 20

21 Lemma 3 Let s be a seam node in G =< V E > and be its atch nodes and E = e e e } be the set o s-links at s. Any elementary cut in G satisies exactly one o { s s s the three roerties: ) = φ 2) E = e } and 3) E = }. Thus any E { s { e s elementary cut cannot contain the s-link e s in E = { e s es es } at any seam node s. Proo: The results immediately ollow rom Lemma 2 and Deinition 3. Lemma 4 The min-cost cut in G =< V E > is an elementary cut. Proo: By Lemma 2 as a cut the min-cost cut satisies exactly one o three roerties: ) E = φ 2) E = e } and 3) I e then E = e e } else E { s s { s s = { }. To rove is an elementary cut we only have to show that i satisies e s roerty 3) then E = }. Suose this is not true i.e. the else art does not hold or { e s ) then the i art holds or. Thus we have E = e e } and this is imossible by { s s the minimality o the cost o and by the act that the costs see Table ) o the three edges in E satisies the triangle ineuality note that the cost unction w.) deined in E. is metric) which imlies that cutting two edges e s and one i.e. e s ). e s together in E is costlier than cutting the third An elementary cut is not necessarily a min-cost cut. For examle in Figure 8 the cut is an elementary cut but not the min-cost cut which is the cut 2. In order to establish a one to 2

22 one corresondence between the set o all elementary cuts and the set o all labelings within one comlete exansion move o the initial labeling we deine the corresonding labeling o a cut as ollows. 5 3 s r old atches A x 0 y z u 0 v w new atch B 2 Figure 8: An examle o an elementary cut that is not the min-cost cut. cut cut r r x y z u v w x y z u v w G =< V E > G =< V E > G =< V E > Figure 9: An examle o two subgrahs G and G o G =< V E > divided by cut. For a given cut not necessarily an elementary cut) let G =< V E > and G =< V E > be the two subgrahs o G divided by i.e. the subgrahs ater removing all 22

23 the edges in rom G see Figure 9) the corresonding labeling o can be deined as ollows: = i V i V. 6) In other words a ixel is assigned label i it belongs to the subgrah G =< V E > otherwise it retains its old label. Lemma 5 I { } N N is the set o all neighboring airs) such that = then any cut in the grah G =< V E > satisies { e } = w ) where the unction w.) is deined in E.. Proo: There are only two cases: ) e } = φ and 2) e } = { e }. Let { { G =< V E > and G =< V E > be the two subgrahs divided by as shown in Figure 9. I { e } = φ then and must belong to the same subgrah i.e. either { } V or { } V. In either case we have = = = by E. 6 and this imlies that { e } = φ = 0 = w ) = w ) by E.. I e } = { e } then { and cannot belong to the same subgrah. Suose V and V then we have = and =. From the weights shown in Table we have { e } = e = w ) which is eual to w ) since = and =. 23

24 Lemma 6 I { } N such that then any elementary cut in the grah G =< V E > satisies E = w ) where E is given in E. 4. e s e s e s x s y old atches A Elementary cut new atch B Figure 0: The smallest subgrah that contains terminal and the node s and and all the corresonding edges among the nodes. Proo: By Lemma 3 satisies exactly one o the three cases: ) E = φ 2) E = { e s } and 3) E = }. We give the roo or case 3 and the roos or the { e s other two cases are similar. Suose E = } let G =< V E > be the smallest { e s subgrah as shown in Figure 0 that contains terminal and the node s and and all the corresonding edges among the nodes. One can see that cuts all the aths between the terminals in G. Let G =< V E > and G =< V E > be the two subgrahs o G divided by as shown in Figure 9. Since is a cut and E = } it can only cut an edge ater ixel { e s see Figure 0) thus and must belong to the subgrah G i.e. { } V. By E. 6 we have = and =. On the other hand by Lemma 3 and Table we have 24

25 s E = e = w ) which is eual to w ) since = and =. The results in Lemma 6 only hold or an elementary cut. For a non-elementary cut it is ossible that contains both s-links e s and E w ) rather than E = w ). e s at a given seam node s. In that case we have Lemma 7 For any cut in the grah G =< V E > its corresonding labeling deined in E. 6 is a comlete exansion move rom the initial labeling. Moreover or any elementary cut its cost is eual to the energy o i.e. = E ) 7) where E ) is given by E. 2. Proo: It is obvious that is an exansion move rom the initial labeling. As shown in Figure 9 both subregions V and V are connected thus by Deinition 2 is comlete. Now suose is an elementary cut its cost is calculated as ollows: = { } + E. 8) { } N = By Lemma 5 the irst term in E. 8 is { } N = e { } N { e } = w ). { } N = By Lemma 6 the second term in E. 8 is 25

26 26 = w } { } { ) N N E. Plugging the above two euations into E. 8 we have ) ) ) ) } { } { } { E w w w = = + = = N N N. Now we resent our main result in the ollowing theorem and give a corollary o the theorem which roves that the otimal comlete exansion move * rom an initial labeling is the min-cost cut in the grah > =< E V G. Theorem There exists a one to one corresondence between the set o all elementary cuts in > =< E V G and the set o all labelings within one comlete exansion move o the initial labeling. Moreover or any elementary cut the cost o is eual to the energy o i.e. ) E =. Proo: Let Ω be the set o all elementary cuts in > =< E V G and F be the set o all labelings within one comlete exansion move o the initial labeling. We deine a ma between Ω and F as ollows: F Ω = ) : ϕ ϕ a 9) where is deined in E. 6. We show that the ma ϕ is a one to one corresondence i.e. both an injection and surjection.

27 We irst rove that ϕ is an injection i.e. or any two distinct elementary cuts and 2 2 we want to show that their corresonding labelings = and = are dierent i.e. 2. Let G ) =< V ) E ) > and G ) =< V ) E ) > be the two 2 subgrahs see Figure 9 or an examle) divided by and G 2) =< V 2) E 2) > and G 2) =< V 2) E 2) > be the two subgrahs divided by 2. Since 2 their corresonding subgrahs must be dierent and this imlies that V ) V ) and 2 V ) V 2 ). Thereore there exists at least one ixel such that V ) and V 2) which imlies that ) = 2 ). Thus we have roved that 2. Elementary cut r r s s 2 x y z x y z s 3 u v w u v w s 4 G =< V E > G =< V E > G =< V E > Figure : An examle o how to construct an elementary cut given a comlete exansion rom the initial labeling. To rove ϕ is a surjection or any comlete exansion rom the initial labeling we need to show that there exists an elementary cut Ω such that ϕ ) =. The elementary cut can be constructed as ollows. Since is a comlete exansion so the set o all 27

28 ixels with labels are connected and the set o all ixels with old labels i.e. ) are also connected according to Deinition 2. Let G =< V E > be the subgrah where all ixels have label and G =< V E > be the subgrah where all ixels have old labels see the let igure in Figure ). For each air o neighboring ixels { } N where V and V let s be the seam node between and then we deine the set o all edges e s as the reuired cut e.g. the set o all dashed lines in the right igure in Figure ) i.e. = { es es E = { es es es }{ } N V V }. 0) learly the cut deined in E. 0 is an elementary cut and satisies = i.e. ϕ ) =. The moreover art o Theorem has been roved in Lemma 7. As discussed in the beginning o this section the objective o grahcut textures is to ind an otimal labeling within a comlete exansion move rom the initial labeling. The ollowing corollary indicates that inding the otimal labeling is euivalent to inding the mincost cut in the grah which can be eiciently comuted by using grah cut techniues [2 8]. orollary Let be the min-cost cut in G =< V E >. Then the otimal labeling * = arg mine ) among is given by * = where is a comlete exansion move rom the initial labeling. 28

29 Proo: Let F = { is a comlete exansion rom } and Ω be the set o all elementary cuts in G. By Theorem there exists a one to one corresondence between G and F. Since is the min-cost cut in G it is an elementary cut by Lemma 4 thus Ω. The cost o is = E ) again by Theorem. The corresonding labeling o is otimal since is the min-cost cut i.e. * = arg ' F mine ) =. 4 onclusions Although the techniue o grahcut textures [5] has been successully used in image and video texture synthesis since it was introduced in 2003 the mathematical ormulation and mathematical roerties o the grahs or the labeling roblem o grahcut textures are not well resented in the original aer which make the techniue unconvincing rom the theoretical oint o view. This aer gives the essential mathematical suort and roos or grahcut textures and clariies the mathematical errors and conusions in the original grahcut textures aer. We rove that the labeling roblem i.e. inding the otimal labeling) o grahcut textures is euivalent to inding the min-cost cut in a comlete exansion grah. Acknowledgements The authors would like to thank the anonymous reviewers and aroriate agencies. 29

30 Reerences. Ahija K. T.L. Magnati and J.B. Orlin Network Flows: Theory Algorithm and Alications. Prentice Hall Boykov Y. O. Veksler and R. Zabih Eicient Aroximate Energy Minimization via Grah uts. IEEE PAMI also in IV 99) ): Eros A.A. and W.T. Freeman Image Quilting or Texture Synthesis and Transer. AM SIGGRAPH 200: Ford L. and D. Fulkerson Flows in Networks. Princeton University Press Kwatra V. et al. Grahcut Textures: Image and Video Synthesis using Grah uts. AM SIGGRAPH ): Liang L. et al. Real-Time Texture Synthesis by Patch-Based Samling. AM Transactions on Grahics ): Sedgewick R. Algorithms in Part 5: Grah Algorithms. Addison-Wesley Reading Massachusetts Veksler O. Eicient Grah-Based Energy Minimization Methods in omuter Vision. Ph. D. Thesis ornell University