Adaptive Exploration of Locally Satisfiable Proposals for Design Layout

Transcription

1 Adaptive Exploation of Locally Satisfiable Poposals fo Design Layout Figue 1: Rule-based design layout algoithms can solve eveything fom laying out domino tiles, though aanging funitue in a oom to supeimposing metadata on a photogaph following aesthetic pinciples. We fomulate the poblem as an iteative move-making algoithm and pesent a new appoach to geneating locally-satisfiable poposals fo each object Abstact We pesent a new appoach fo ule-based design layout poblems, examples of which include automated gaphic design, funitue aangement and synthesizing vitual wolds. We fomulate this as a weighted constaint-satisfaction poblem which is high-dimensional and equies non-convex optimization. We use a hype-gaph to epesent a set of objects and the design ules applied to them. Although infeence in gaphs with highe ode edges is computationally expensive, we pesent a hybid solution-space exploation algoithm that adaptively epesents highe ode elations compactly. We exploit the fact that fo many types of ules, satisfiable assignments can be found efficiently. In othe wods, these ules ae locally satisfiable. Theefoe we can sample fom the known patial pobability distibution function fo each ule. Expeimental esults demonstate that this sampling technique educes the numbe of samples equied by othe algoithms by odes of magnitude. We demonstate usage via diffeent layout examples such as supeimposing textual and visual elements on photogaphs. Ou adaptive seach algoithm is applicable to othe optimization poblems and genealizes many peviously poposed local seach based algoithms like gaph cuts and iteated conditional modes. CR Categoies: F.4.1 [Mathematical Logic]: Logic and Constaint Pogamming [G.3]: Pobability and Statistics Makov Pocesses; Keywods: weighted constaint optimization, layout synthesis, gaphic design 1 Intoduction Compute-assisted (o automated) design, layout synthesis and object aangement poblems consist of assigning values to object popeties, guided by a set of ules. These ules ae domain specific and may fo example be geometic, examining the spatial elations between objects; colo-based, matching the object s appeaance to the palette of the design o enhancing the contast of specific objects; o semantic, taking into account the functionality of the object. Regadless of thei oigin, they ae epesented as cost functions that measue the quality of any configuation of one o moe objects. This pape focuses on instances of these poblems in which a single good solution is equied, with low computational and time costs, mostly tageted at mobile devices. Gaphic design equies an eye fo aesthetics and is moe at than science. In most cases, ceating a design combining textual and visual elements is a job bette left to pofessionals. Howeve, thee ae applications in which thee is need eal time layout of textual and gaphical elements, based on pedetemined ules. In these cases it is not possible to design each and evey instance. Fo example, when bowsing though a photo collection, it is useful to be able to view the image meta-data (title, exposue infomation, histogam), both on the camea and on vaious othe devices. Howeve, simply supeimposing elements on a photo can obstuct faces and othe salient egions and poduce an unattactive layout. We aim to define ules that efeence an image s saliency map, colo palette and majo edges, and lay out the data by obseving symmety, saliency, photogaphic pinciples and othe quantifiable measues. We fomulate design layout as a weighted constaint-satisfaction poblem. Ou fomulation incopoates both low-ode and highode inteactions between design elements. Low-ode ules ae defined ove one o two design elements. Fo example a ule that states that two paticula objects should be close to each othe. On the othe hand, high-ode inteactions ae defined ove lage numbe of design elements. Fo example a ule specifying that a goup of objects needs to be collinea. We epesent a goup of design ules opeating on a set of objects using a hype-gaph. In this gaph, each node epesents an object and each edge is a ule, connected to all the objects it efeences. Infeence in gaphs with highe ode edges is computationally expensive. We popose an adaptive solution space exploation algoithm that iteatively exploes the solution space and in doing so is able to epesent highe ode elations compactly. We apply ou adaptive seach algoithm to design layout poblems, howeve it is applicable to othe optimization poblems and genealizes many peviously poposed local seach based algoithms like Gaph cuts [Boykov et al. 21; Szeliski et al. 26; Woodfod et al. 28; Gould et al. 29; Lempitsky et al. 21] and Iteated conditional modes [Szeliski et al. 26; Jung et al. 29]. The key idea that dives ou exploation algoithm is an obsevation that fo many types of ules, satisfiable assignments can be found efficiently. In othe wods, these ules ae locally (individually) satisfiable i.e. we geneate poposals fo objects that locally satisfy individual ules with no need fo blind sampling. Ou method gene- 1

2 ates, ove seveal iteations, locally satisfiable and andom poposals fo each object (node), until it conveges to a solution that cannot be impoved. Expeimental esults demonstate that ou locally consistent sampling technique is vey efficient and equies substantially fewe numbe of samples compaed to othe algoithms. We demonstate ou appoach on 2D layout poblems involving supeimposing textual and visual elements on photogaphs as well as compaing to pevious esults in funitue aangement. 2 Related Wok feence algoithm extending simulated annealing with local steps. 3 Layout Specification We define a design as a set of ules that ae ceated by an atist o designe. The design is used to find a layout, an assignment of values to a set of objects, that satisfy the ules of the design. Given an envionment in which we need to lay out the objects, we call the space of all possible assignments the layout solution space. We define a cost function fo the design Constaint Satisfaction Design and layout synthesis consist of ules efeencing a set of objects. An assignment to each object can be measued by how well the ules ae met, whethe they ae satisfied o violated. In essence these ae constaint satisfaction poblems (CSP) [Mackwoth 1977], fundamental in Atificial Intelligence and Opeations Reseach. A vaiant of the poblem, weighted CSP defines a cost function assigned to each constaint, and the objective is to minimize the oveall cost. A lage majoity of CSP algoithms [Kuma 1992] use a seach paadigm ove a limited set of possible object assignments. These appoaches ae elatively igid, and do not offe inteactive pefomance c(s) := i i(ŝ) (1) whee i : (O i O) R is a cost function (ule) opeating on a subset of the objects, s S is a specific solution, and ŝ is a slice of the solution containing only the objects in O i. Typically each ule applies only to a small subset of the objects. Facto gaph Hype gaph MAP Infeence In compute vision, many tasks such as segmentation of an image can be fomulated as image labeling poblems whee each vaiable (pixel) needs to be assigned the label which leads to the most pobable (o lowest cost/enegy) joint labeling of the image. The models fo these poblems ae usually specified as facto-gaphs in which the facto nodes epesent the enegy potential functions that opeate on the vaiables [Kschischang et al. 21]. In most vision models, the enegy function is composed of unay and binay tems and the inteactions between objects ae geneally limited to vaiables in a 4 o 8 neighbohood gid. The spase gid-like stuctue of the object inteactions and the limited numbe of labels allows fo fast solution of image labeling poblems using techniques such as gaph-cuts [Boykov et al. 21; Gould et al. 29; Lempitsky et al. 21; Szeliski et al. 26; Woodfod et al. 28], belief-popagation [Peal 1982], and tee message-passing [Wainwight et al. 25; Kolmogoov 26]. In ou case ules can be defined ove multiple vaiables, and ceate complex facto gaphs which these appoaches do not handle well. Futhe, each object typically has a lage space of possible configuations, which inceases the complexity in multi-object inteactions. Futhemoe, in all but the simplest scenaios the facto gaph contains cycles that makes the poblem NP-had even if the label space fo each object is small. Layout Synthesis Ultaviolet [Boning and Feeman-Benson 1998] uses a constaint satisfaction algoithm famewok fo inteactive gaphics. The constaints fo use inteface layout usually fom a non-cyclic gaph, ae hieachical in natue and containe based and ae theefoe less complex. In [Yu et al. 211; Meell et al. 211] a set of ules and spatial elationships fo optimal funitue positioning ae established fom examples and expet-based design guidelines. These ules ae then enfoced as constaints to geneate funitue layout in a new oom. [Yu et al. 211] employed a simulated annealing method which is effective but takes seveal minutes, while [Meell et al. 211] sample a density function using the Metopolis-Hastings algoithm implemented on a GPU. They evaluate a lage numbe of assignments and achieve inteactive ates (equiing a stong GPU). Both papes wok with a small numbe of objects in elatively small ooms and in static scenaios. In [Yeh et al. 212], the motivation is to populate a scene with a vaiable numbe of objects (open univese). They pesent a pobabilistic in o 1 o 2 o 3 o 4 o = f(o 1, o 2 ) 2 = f(o 1, o 2, o 3 ) 3 = f(o 1, o 2, o 4, o 5 ) o 1 1 o 2 o 4 o 5 Figue 2: A design (in this case consisting of thee ules ove five objects) can be epesented as a Facto Gaph, a bipatite epesentation connecting object nodes to facto nodes. Altenatively it can be epesented as a hype-gaph in which each node is an object, and an edge epesents common ules between connected nodes. A use specifies a design using declaative pogamming. Fist objects ae defined, each identified by a unique name and belonging to one of seveal pedefined classes. An object s class defines its popeties which may be initialized to a specific value, and may be fixed (not allowed to change in the optimization pocess). Fo example an object of class Title might have popeties such as position, otation, font, font size, and colo. Rules ae witten using simple algebaic notation and a libay of pedefined outines, eithe as cost functions o as Boolean conditions (in which case we automatically assign a cost function). A ule can efeence any of the objects defined (and thei popeties), as well as the envionment. Fo example when aanging objects in a oom the designe might efeence wall and floo positions, in a 2D poste design the designe might efeence the colo palette of the backgound image. The numbe of objects included in a ule classify it as unay (one object), binay (two objects), tenay (thee objects) o multiple. 3.1 Gaph Repesentation A common gaph epesentation fo MAP poblems is the Facto Gaph [Yeh et al. 212] which has two node goups: object nodes and facto (ule) nodes. Edges connect factos to the objects they efeence (Figue 2 left). A facto epesenting a unay ule will have one edge, a binay ule will have two edges and so on. We epesent a design as a gaph G = (V, E). Each object o has an associated node v o whose cost function is φ(v o) = { : {o} R} o 3 2 2

3 1 2 1 = f(o 1, o 2 ) 2 = f(o 1, o 2, o 3 ) A 1 (o 1, o 2 ) Tansfoming High-ode ules into Paiwise Inteactions 3 o 3 1 o 1 1 o o 2 1 o 2 3 = f(o 1, o 2, o 4, o 5 ) A 1 (o 1, o 2 ) 1 o 1 o o 3 A 2 (o 4, o 5 ) o 4 o 5 Figue 3: We constuct a gaph fo a design incementally. Rules which efeence moe than two objects ae tansfomed into paiwise inteactions via auxiliay nodes: 1 is a binay ule, 2 is a tenay ule tiggeing the ceation of A 1 which epesents a pai of values (fo o 1 and o 2). 3 involves fou vaiables, in which case we equie two auxiliay nodes (we euse peviously ceated A 1) To simplify the gaphical epesentation of the design, we tansfom hype-edges into paiwise gaph inteactions by intoducing auxiliay nodes. We divide the set of objects associated with a hypeedge e into two goups A 1 and A 2. Fo each goup consisting of moe than one object we add an auxiliay node (othewise we use the oiginal object node). An assignment to auxiliay node A i is an assignment to all vaiables associated with this node. The cost function of an auxiliay node is φ(a i) = (no unay cost). We connect the two nodes with an edge ê such that ψê = ψ e. We connect auxiliay nodes with thei associated object nodes. The cost function fo these edges ψ({o, A i}) is if the assignment to object o matches the assignment to A i and abitaily high othewise. The addition of the auxiliay vaiables ensues that thee ae only binay inteactions between nodes. Fomally, this coesponds to a cost function: E(x) = φ i(x i) + ψ ij(x i, x j) (2) i V ij E (sum of all unay ules on object o). We connect an edge e between v o1 and v o2 if thee exists at least one ule associated with these objects. Its cost function is ψ e = { : {o 1, o 2} R}. Given an assignment to all of the objects, the summed cost ove the nodes and edges of the gaph is equal to the design cost (equation 1). Note that a ule may efe to moe than two objects, and theefoe an edge can connect moe than two nodes, ceating a hype-gaph (Figue 2 ight). 4 Finding an Optimal Layout An optimal layout is the global minimum in the scala field defined by the design cost function. Finding the optimal layout o even a good one is difficult: Rule cost functions may be non-convex, ules might be unsatisfiable, fo example if they conflict with the envionment o with themselves, theefoe we cannot know the lowe bound on the cost and it is difficult to specify a stopping citeia. And finally, the high-dimensional natue of the space and the assumed spasity of feasible solutions educe the effectiveness of stochastic sampling whee V and E epesent the set of nodes and the set of edges between these nodes espectively, x i epesents the label taken by a paticula node, and φ i and ψ ij ae functions that encode unay and binay costs. In figue 3 we demonstate how the intoduction of 2 dives the ceation of auxiliay node A 1, while the cost function is associated with the edge (A 1, o 3). Adding 3 euses A 1 while adding an additional node A 2 and connecting them. 4.2 Adaptive Layout Space Exploation A simple method to find a low-cost solution unde the function defined in equation 2 is to exploe the solution space by local seach i.e. stat fom an initial solution and poceed by making a seies of changes which lead to solutions having lowe enegy. At each step, this move-making algoithm exploes the neighboing solutions and chooses the move which leads to the solution having the lowest enegy. The algoithm is said to convege when no lowe enegy solution can be found. An example of this appoach is the Iteated Conditional Modes (ICM) algoithm that at each iteation optimizes the value of a single vaiable keeping all othe vaiables fixed. Howeve, this appoach is highly inefficient due to the lage label space of each vaiable. Instead we could pefom a andom walk algoithm, in each iteation we select a new value fo one of the objects and evaluate the cost function. If the cost impoves we save the new configuation: Simila to [Meell et al. 211; Yu et al. 211] we focus on a discetized vesion of the solution space. In some instances the application domain necessitates this (fixed esolution image backgounds) while in othe instances thee is a specific esolution equiement which we need to meet. Given N objects in the design and k possible assignments pe object, the size of the solution space k N makes pefoming an exhaustive seach pohibitively expensive. Pevious methods have attempted to sample fom the undelying pobability distibution function, using Metopolis-Hastings [Hastings 197] algoithm coupled with concepts fom simulated annealing. These methods still equie a pohibitively lage numbe of samples (and of couse evaluations of the cost function), theefoe equiing a long un time o eliance on massively paallel GPU implementations [Meell et al. 211]. In many applications pefomance is an issue, and in some platfoms such as mobile devices, computing is costly. Ou appoach theefoe focused on educing the numbe of evaluations equied to find a feasible solution. 252 Algoithm 1 Random Walk minsolution RandomAssignment() cuentsolution minsolution mincost Evaluate(minSolution) fo i 1, nites do fo all O Objects do p O nextp oposal cuentsolution p cost Evaluate(cuentSolution) if cost < mincost then minsolution cuentsolution mincost cost end if end fo end fo Geneating poposals fo this algoithm is key to its pefomance. 3

4 The most staight-fowad appoach is to sample unifomly ove the object popeties. Anothe appoach is to stat with unifom sampling (lage steps) and ove time educe step size, sampling nomally aound the pevious object value which we tem multiesolution andom walk. Othe heuistics exist and have been tied befoe, such as swapping between objects. 4.3 Exponential-sized Seach Neighbohoods Using a bigge move space has a highe chance of eaching a good solution. This obsevation has been fomalized by [Jung et al. 29] who give bounds on the eo of a paticula move making algoithm as the size of the seach space inceases. [Boykov et al. 21] showed that fo many classes of enegy functions, gaph cuts allow the computation of the optimal move in a move space whose size is exponential in the numbe of vaiables in the oiginal function minimization poblem. These move making algoithms have been used to find solutions which ae stong local minima of the enegy (as shown in [Boykov et al. 21; Komodakis and Tziitas 25; Kohli et al. 27; Szeliski et al. 26; Veksle 27]). While the taditional move making methods only dealt with vaiables with small label sets, thei use has ecently been extended to minimizing functions defined ove lage o continuous labels spaces [Lempitsky et al. 21; Woodfod et al. 28; Gould et al. 29]. An example of this wok is the Fusion move method that in pinciple allows fo the minimization of functions defined ove continuous vaiables. Given a set of vaiables, the fusion-move algoithm woks by poposing a labeling fo all vaiables. It chooses fo each vaiable whethe to etain its pevious label o take the new poposed label. Ou method genealizes the above algoithms as, in each iteation, instead of poposing a single new label, it poposes multiple new poposals fo each vaiable. Unlike [Veksle 27], a lage majoity of these poposals ae abitay values in the label space of each vaiable that ae locally satisfiable (see section 4.4). Howeve, the bigge diffeence is the fact that ou method adaptively selects the numbe of vaiables included in the move. In this way it can smoothly exploe the whole spectum of choices between iteated conditional modes on one end, and the full multi-poposal fusion move, that involves changing the label of all vaiables Solving a single iteation We fomulate the poblem of jointly 291 selecting the best poposals fo all vaiables that satisfy the most 292 ules as a discete optimization poblem. Moe fomally, let P i = 293 {p 1 i, p 2 i,..., p k i } be a set of k poposal configuations fo vaiable x i. 294 We intoduce indicato vaiables t l i, i V, l {1...k} whee 295 t l i = 1 indicates that vaiable x i takes the popeties in poposal 296 l. Similaly, we intoduce binay indicato vaiables t l, ij E, l, {1...k} whee t l ij = 1 indicates that vaiables x i and x j 298 take the position poposed in poposal l and espectively. Given 299 the above notation, the best assignment can be computed by solving 3 the following optimization poblem: min t l iφ i(p l i) + t l ijψ ij(p l i, p j) i V l ij E l, s.t. i, t l i = 1 i, j, l, i, j, l, l t l ij = t l i t l i, t l ij {, 1} The above optimization poblem in itself is NP-had to solve in geneal. Instead, we solve its LP-elaxation and ound the fac- (3) tional solution. Fo this pupose, we could use geneal pupose linea pogamming solves. Howeve, we used an implementation of the sequential tee e-weighted message passing algoithm (TRW-S) [Wainwight et al. 25; Kolmogoov 26] that ties to efficiently solve the linea pogam by exploiting the spase natue of the inteactions between vaiables. TRW-S guaantees a nondeceasing lowe bound on the enegy, howeve it makes no assuances egading the solution (See [Szeliski et al. 26] fo detailed compaisons). Algoithm 2 Lage Moves pocedue LARGEMOVES(O, R) Objects, Rules G ConstuctGaph(O, R) minsolution RandomAssignment(O) mincost Evaluate(minSolution) fo i 1, nites do A ObjectsT ooptimize(g, cuentsolution) fo all o i A do P i P oposecandidates() end fo fo all R do UpdateGaphCosts(G, Evaluate(, {P 1, P 2,...})) end fo cuentsolution T RW S(G) cost Evaluate(cuentSolution) if cost < mincost then minsolution cuentsolution mincost cost end if end fo end pocedue Theefoe ou evised algoithm woks as follows (see algoithm 2): Given a design we constuct a gaph as descibed in subsection 4.1. In each iteation we geneate a set of candidates fo all objects to be optimized. Candidates ae sampled andomly o though locallysatisfiable poposals as descibed in subsection 4.4. We evaluate the cost of each ule, fo each tuple of values associated with it. Theefoe a unay ule is evaluated k times (k being the numbe of candidates), a binay ule k 2 times (if each of its efeenced objects has k candidates) and so on. These costs ae tansfeed to the gaph nodes and edges as descibed above. Note that fo complex ules, this ceates a challenging numbe of evaluations, which can go up to k n (whee n is the numbe of objects in the design). We found that by limiting the set of candidates fo auxiliay nodes to O(k) tuple values did not detact fom the efficiency of the algoithm, and kept ou complexity at O(nk 2 ) oveall. We then attempt to find an impoved assignment fo ou objects, based on the populated gaph. In each iteation we use eithe Random-Walk o TRW-S dependent on the numbe of objects to optimize. We epeat this pocedue, caying fowad the cuent best assignment, until a maximum numbe of iteations o evaluations is eached, o the cuent solution is of acceptable cost. 4.4 Locally Satisfiable Poposals The space of possible placements (style, colo etc.) of a design element (object) is vey lage and it may take a lage numbe of poposals to obtain a good assignment fo the object [Ishikawa 29]. We ovecome this poblem by guiding the mechanism though which new poposals ae geneated. Fo many types of ules, assignments that satisfy these ules can be found efficiently. In othe wods, these ules ae locally satisfiable. In simple tems, given an assignment to some of the objects efeenced by we can geneate good poposals fo the est, without esoting to blind sampling in the 4

5 (a) Minimal distance Maximal distance Allowed deviation in diection Allowed deviation in facing (a) A 1 (x 1, x 1 ) d x i, x i+1 > C x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 x 1 (b) 2 < d x 1, x 2 < 5 x 2 x 1, x 2. font.97 x 2. font, x 1. font.9 x 1 x 2 x 3 x 4 2 < d x 2, x 3 < 5 x 3 x 2, x 3. font.97 x 3. font, x 2. font.9 (b) A 1 (x 1, x 2 ) A 2 (x 2, x 3 ) A 3 (x 3, x 4 ) x 3 x 1 x 2 x 4 x i = x i+1 x i 1 2 x 5 (c) x 1 x 9 x 8 x 7 x 6 Figue 4: A set of domino tiles laid out on a cuve (a) Each domino tile must be within a set distance fom its two neighbos, face in oughly the same diection, and emain oughly in line (b) Resulting gaph is a chain whee each each edge epesents a compound ule between neighboing tiles (c) A esult fo ten tiles. Figue 6: (a) Ten objects ae aanged such that each object attempts to be nea the aveage of its two neighbos, and yet maintain a minimal distance fom both. (b) The solution to this design is a least-squaes one, satisfying none of the constaints fully. 362 LSP using two stategies: x 1 x 2 c (a) α 25 x 5 x 3 x 4 c c, x 1, x 2 25 d c, x 1 = d(c, x 2) A 2(c, x 2) x 2 A 1(c, x 1) c, x 2, x 3 25 (b) d c, x 1 = d(c, x 3) c, x 1, x 3 25 Figue 5: A set of objects aanged in a cicle aound a cental object (top ight) The distance fom each object to c is equal, the angle between each pai of objects x i, x j and c is at least 25 degees (b) A patially labeled gaph which shows auxiliay nodes ceated to epesent tenay constaints. The dashed edges ae auxiliay constaints, popagating the selected value of an object to its auxiliay nodes. This gaph has high connectivity. layout solution space. Ou appoach could be seen as pefoming Gibbs sampling [Casella and Geoge 1992], taking advantage of a known patial pobability function, to sample fom the whole solution space. Fo example a geometic constaint dist(a, b) < 4 is locally satisfiable as given a we geneate poposals fo b within the cicle centeed aound a with adius 4. A colo constaint complementay(a, b) is locally satisfiable as given colo a, b is easy to calculate. A unay constaint saliency(a, bg) <.1 attempting to place a 2D element a on a backgound image bg can geneate poposals fo a based on a pe-calculated saliency map of bg. When a designe declaes ules in ou declaative language, he can define a ule as locally satisfiable. This invese function geneates poposals fo the ule efeenced objects, given one o moe object assignments. Rules which have not been declaed explicitly can still be emulated as locally satisfiable by andomly sampling a lage numbe of value tuples fo the ule efeenced objects, and picking the best tuples (which comes with a computational cost). A locally satisfiable poposal (LSP) is a candidate fo object o which was poposed by a locally satisfiable ule. We geneate x 3 x Local: Given a cuent assignment to all objects, and a set of active objects to be optimized in this iteation, we iteate ove all ules efeencing these objects. Fo each ule and fo each object efeences, we fix that object value and poduce one o moe LSP fo the othe objects. Geedy: Given the hype-gaph stuctue of the design, we apply a BFS stating fom a andomly selected node. As we discove new nodes, we geneate LSP fo them, based on the nodes aleady visited, and the edges by which we discove these nodes. Repeating this algoithm ceates a seies of geedy assignments. 5 Expeimental Evaluation We measue the pefomance and quality of a layout optimization algoithm by counting ule evaluations. Fo example calculating the cost of a specific layout fo a design is i, the numbe of ules in the design. Pevious appoaches have counted the numbe of samples the algoithm pefoms fo all objects in all iteations. Howeve, this measue favos algoithm which pefom an exhaustive seach ove limited combinations of values. Anothe measue consists of counting numbe of evaluations of complete layouts. Howeve, this is not epesentative of belief-popagation algoithms (such as TRW-S) in which patial evaluations ae combined togethe. Designs diffe in the type and numbe of ules they contain, and by how constained the solution is. These diffeences ae eflected in the undelying gaph stuctue, and in ou ability to ceate locally satisfiable poposals. We tested ou move-making algoithm on a lage numbe of diffeent designs, of which we pesent a few achetypes hee. Fo each design we pesent the esults of seveal algoithm configuations, vaying ove the numbe of objects optimized in each step, size of candidate set and usage of locallysatisfiable poposals. The esults demonstate that the connectivity of the gaph is a good indicato fo tuning the move-making algoithm, and that locally-satisfiable poposals educed the numbe of evaluations equied in all scenaios. In ou expeiments the ules ae geometic, and each objects in the design can be assigned po- 5

6 sition, otation and scale in 2D. We apply the configuations descibed in table 1 on the following designs. In each design we an each vaiant algoithm 3 times and took the median of the esults. Ou algoithm is cuently a non-optimized single-theaded CPU implementation, its aveage un time is 2-3 seconds Domino - 3 Tiles LS TRW (Geedy) LS TRW (Local) Domino - Thity tiles aanged in a cuve i.e. each tile is at a cetain distance fom the next, and facing in a simila diection. The design and gaph stuctue ae visualized in figue 4. The ceated gaph is a chain stuctue which can be optimally solved by belief popagation algoithms such as TRW-S. Moeove, non-cyclic gaphs appea to wok extemely well with geedy LSP, which is eflected in the esults (figue 7) Random Walk MultiRes RW LS MultiRes RW Cicle - In ode to test a highly connected gaph with cycles, we ceated a design fo nine tiles aanged in a cicle (non-fixed adius) aound a cental tile. The minimal angle between any two tiles is at least 25 o. The design and esulting gaph stuctue ae shown in figue 5, while the expeiment esults ae in figue 8. All ules in this design ae tenay, and the ules enfocing a minimal angle between all objects ceate a gaph with high connectivity. In this scenaio evaluating poposals fo multiple objects at once is disadvantageous. We found that TRW fails to find a solution that appoximates the minimum. Howeve, exhaustive evaluation of each candidate set is not ealistic. Theefoe we educed the numbe of candidates as shown in the gaphs, to 4 and 6 candidates, and saw impovement. Still, constaining the algoithm to one poposal fo a single object in each iteation gives the best esults fo this scenaio. Locally satisfiable poposals pove to be an asset in this scenaio, achieving an x2 facto fom 2 evaluations and up until 12 evaluations. Laplacian Cycle - Finally, to challenge the candidate poposal pocess, we attempt to aange ten tiles t 1..t 1 such that t i = (t i 1 + t i+1)/2 and d(t i, t i+1) > C. Since the ules wap aound t 1 the cost can neve be and the best possible solution is a leastsquaes oval stuctue. The design and esulting gaph stuctue ae in figue 6, while the expeiment esults ae in figue 9. We applied seveal vaiants of TRW on this design, stating fom 2 poposals fo each object each iteation (one LSP, one andom) and up to 1 candidates. Additionally we tied multi-esolution andom walk with and without LSP. We found that a small numbe of candidates ove a lage numbe of iteations poduced the best esult, and that LSP wee of benefit. Algoithm #Obj #Cands Random Walk 1 2 Multi-Res RW 1 2 LS MR RW 1 2 LS TRW (Geedy) n k LS TRW (Local) n k Table 1: Algoithms applied in ou expeiments. #Obj is the numbe of objects optimized in each iteation, #Cands is the numbe of candidates assigned to each (optimizable) object in each iteation. The TRW vaiants have k candidates, 5% of which ae locally satisfiable poposals (if applicable). 5.1 Funitue Aangement We eceated the design ules descibed in [Meell et al. 211], ewiting them to be locally satisfiable. We then used ou algoithm to find funitue layout in seveal oom configuations (figue 1). Ou appoach poduced compaable esults in 5 evaluations, Figue 7: Domino: The chain-like stuctue of the gaph woks optimally fo belief popagation algoithms such as TRW. Combining TRW with geedy LSP conveges on a solution quickly. compaed to 5M 1M evaluations (extapolated fom figue 7 in thei pape). Note that while we poduce a single feasible solution in each un, they attempt to poduce a vaiety of solutions. 5.2 Photo Ovelay When eviewing a lage collection of photogaphs, it is often useful to see the coesponding meta-data (title, date, exposue infomation, geo-tags and use supplied tags) alongside the oiginal image. In smalle fom-factos, thee is not enough sceen eal estate to display the image alongside the infomation. We pesent an application to ovelay textual and visual infomation on an image, based on geometical and aesthetic design ules. Given an image, we extact textual meta-data fom its EXIF such as title, camea and lens model, apetue, shutte speed as well as calculate a luminance histogam. We then calculate a saliency map [Peazzi et al. 212], un an edge detection filte and extact the colo palette of the image [Mose et al. 27] to which we add black and white (Figue 11. We define ou design such that the supeimposed elements ae positioned on the non-salient egions in the image and thei colos ae taken fom the image extended colo palette. Additionally, the title is lage than the exposue infomation, and the exposue infomation elements ae odeed vetically and attempt to align (left o ight depending on thei position within the image). The esults of ou design can be seen in figue 12 and figue Conclusions We pesented a new algoithm fo ule-based design layout poblems. Ou appoach unifies and expands on peviously poposed local seach based methods in the sense that it adaptively detemines the seach neighbohood accoding to the undelying gaph stuctue. We intoduced the concept of locally satisfiable poposals and demonstated that thei use damatically educes the numbe of evaluations equied fo finding a ule-consistent layout. In cases whee LSP fails, ou algoithm degades to a andom sampling appoach. While the efficacy of ou method was demonstated on a 2D layout poblem, ou solution genealizes to a wide ange of layout pob- 6

7 Online Submission ID: 179 Figue 12: Applying ou design ules to vaious images poduces pleasing esults automatically and in eal time, allowing a use to bing up infomation about an image without inteupting the flow of images. Cicle - 9 Objects aound Cente Laplacian - 1 objects 14 Random Walk 14 MultiRes RW 12 LS TRW (Geedy) 4 8 LS TRW (Local) Figue 8: Cicle: The highly connected gaph is a hindance when using a lage numbe of candidates. In this scenaio inceasing the numbe of optimizable objects in each iteation, and the size of the candidate set did not pove beneficial. The best pefoming vaiant was a multi-es andom walk appoach, in which the candidates wee LSP. lems in two o highe dimensions. In futue wok we seek to apply ou algoithm on layout synthesis in 2.5D (such as placing objects on a topogaphical map) and in 3D. Moeove, we ae woking towads a massively paallel implementation of ou appoach that we hope would be beneficial fo the gaphics community Refeences B ORNING, A., AND F REEMAN -B ENSON, B Ultaviolet: A constaint satisfaction algoithm fo inteactive gaphics. Con Figue 9: Laplacian Cycle: The optimal solution in this design is a least-squaes one, and ceates a elatively complex gaph. Still we find that LSP helps convege quickly, especially with a small set of candidates in each iteation (half of which ae andom) LS MR RW MultiRes RW LS TRW (Geedy) LS TRW (Geedy) 8 LS TRW (Geedy) LS TRW (Geedy) 6 1 LS TRW (Local) LS TRW (Geedy) 4 12 LS MR RW LS TRW (Geedy) staints 3, 1, B OYKOV, Y., V EKSLER, O., AND Z ABIH, R. 21. Fast appoximate enegy minimization via gaph cuts. PAMI 21. C ASELLA, G., AND G EORGE, E. I Explaining the gibbs sample. The Ameican Statistician 46, 3, G OULD, S., A MAT, F., AND KOLLER, D. 29. Alphabet soup: A famewok fo appoximate enegy minimization. In CVPR 29, H ASTINGS, W. K Monte calo sampling methods using makov chains and thei applications. Biometika 57, 1,

8 Online Submission ID: 179 Figue 1: We follow the design ules descibed in [Meell et al. 211] and apply ou move-making algoithm to geneate these funitue aangements. Figue 13: In this example, in addition to the descibed design ules, we align the meta-data elements along the most pominent hoizontal edges, extacted using edge-detection. 527 Figue 11: Fo each image we extact a saliency map [Peazzi et al. 212] and a colo palette [Mose et al. 27] I SHIKAWA, H. 29. Highe-ode gadient descent by fusionmove gaph cut. In ICCV 29, J UNG, K., KOHLI, P., AND S HAH, D. 29. Local ules fo global map: When do they wok? In NIPS, KOHLI, P., K UMAR, M. P., AND T ORR, P. H. S. 27. P3 & beyond: Solving enegies with highe ode cliques. In CVPR. KOLMOGOROV, V. 26. Convegent tee-eweighted message passing fo enegy minimization. IEEE Tans. Patten Anal. Mach. Intell. 28, 1 (Oct.), KOMODAKIS, N., AND T ZIRITAS, G. 25. A new famewok fo appoximate labeling via gaph cuts. In ICCV K SCHISCHANG, F. R., F REY, B. J., AND L OELIGER, H.-A. 21. Facto gaphs and the sum-poduct algoithm. IEEE Tansactions on Infomation Theoy 47, 2, K UMAR, V Algoithms fo constaint satisfaction poblems: A suvey. AI MAGAZINE 13, 1, L EMPITSKY, V. S., ROTHER, C., ROTH, S., AND B LAKE, A. 21. Fusion moves fo makov andom field optimization. IEEE Tans. Patten Anal. Mach. Intell. 32, 8, M ACKWORTH, A. K Consistency in netwoks of elations. Atificial Intelligence 8, 1, M ERRELL, P., S CHKUFZA, E., L I, Z., AGRAWALA, M., AND KOLTUN, V Inteactive funitue layout using inteio design guidelines. In SIGGRAPH 211. M ORSE, B., T HORNTON, D., X IA, Q., AND U IBEL, J. 27. Image-based colo schemes. In ICIP 27. P EARL, J Reveend bayes on infeence engines: A distibuted hieachical appoach. In AAAI, P ERAZZI, F., K R A HENB U HL, P., P RITCH, Y., AND H ORNUNG, A Saliency filtes: Contast based filteing fo salient egion detection. In CVPR 212. S ZELISKI, R., Z ABIH, R., S CHARSTEIN, D., V EKSLER, O., KOLMOGOROV, V., AGARWALA, A., TAPPEN, M. F., AND ROTHER, C. 26. A compaative study of enegy minimization methods fo makov andom fields. In ECCV 26. V EKSLER, O. 27. Gaph cut based optimization fo mfs with tuncated convex pios. In CVPR 27. WAINWRIGHT, M. J., JAAKKOLA, T., AND W ILLSKY, A. S. 25. Map estimation via ageement on tees: message-passing and linea pogamming. IEEE Tansactions on Infomation Theoy 51, 11, W OODFORD, O. J., T ORR, P. H. S., R EID, I. D., AND F ITZGIB BON, A. W. 28. Global steeo econstuction unde second ode smoothness pios. In CVPR, IEEE Compute Society. Y EH, Y.-T., YANG, L., WATSON, M., G OODMAN, N. D., AND H ANRAHAN, P Synthesizing open wolds with constaints using locally annealed evesible jump mcmc. ACM Tans. Gaph. 31, 4 (July), 56:1 56:11. Y U, L.-F., Y EUNG, S.-K., TANG, C.-K., T ERZOPOULOS, D., C HAN, T. F., AND O SHER, S. J Make it home: automatic optimization of funitue aangement. In SIGGRAPH 211.