Workshop In Advanced Data Structures Push-Relabel Heuristics. Maya Orshizer Sagi Hed

Transcription

1 Workshop In Advanced Data Structures Push-Relabel Heuristics Maya Orshizer Sagi Hed 1

2 Contents 1 Overview And Goals 3 2 Our Implementation Of Push Relabel Definitions Maintained Data Initialization Termination Methods Phase II Active Node Selection Strategy Heuristics Global Relabeling Heuristic Overview The Heauristic Correctness Layout Code Variation Level-Cut Heuristic Overview Intention Gap Relabeling Heuristic Code Correctness Layout Maintaining Minimum Cuts A Maximum Flow Bound For Global Relabeling Timestamp Heuristic Overview Intention Correctness Layout Code Implementation Details 15 5 Graphs And Work Plan 15 6 Testing Plan 16 7 Appendix Correctness Of Gap Relabeling References 16 2

3 1 Overview And Goals In our workshop we implement the push-relabel algorithm of A.V. Goldberg and R.E. Tarjan for the maximum flow problem (see [1]). We add to the current implementation of A.V. Goldberg by 2 variations, and 2 more original heuristics of our own. Our primary goal is to show some factor of improvement compared to A.V. Goldberg s h prf code (it was renamed to hi pr in its newest version - The h prf implementation of pushrelabel uses highest level active node selection - slightly different than our own suggestion (see 2.7), global relabeling heuristic - slightly different then our own suggestion (see 3.1), and gap relabeling heuristic - which is generalized by our level-cut heuristic (see 3.2). We also add the timestamp heuristic (see 3.3). It is possible that our code will not be directly comparable to h prf (i.e. will not perform better than h prf even without the suggested heuristics). However if our code will show an improvement factor compared to itself without the heuristics, then it might be deduced that the heuristics are worthwhile (and perhaps then attempt to integrate them into h prf). 2 Our Implementation Of Push Relabel The push-relabel algorithm has many variations, namely in the form of different active node selection strategies. In this section we will outline the steps of the algorithm, providing pseudo code for the main methods and specifications of the data that is maintained throughout the algorithm. We will do so in light of our chosen variation so that pseudo codes here should correspond to the actual layout of our later implemented code. The push-relabel algorithm is fully described in [1]. We will not provide a full description here, only a recap in light of our own implementation. Our heuristic additions to the algorithm are described in section Definitions G = (V, E) - The input graph. V = n, E = m. G f = (V, E f ) - The residual graph of G with the pre-flow f, consisting of edges and anti-edges. c(e) is a capacity function on the edges of G f. For a residual anti-edge e, c(e ) = 0. f(e) is a pre-flow function on the edges of G f. For a residual anti-edge e, f(e ) = f(e ). c f (e) is a residual capacity on the edges of G f. c f (e) = c(e) f(e) for any edge e. e(v) is the amount of excess flow on node v. e(s) =. Active node is a node v s with e(v) > 0. d(v) is a label value on node v. s is the source node for the maximum flow problem. t is the sink node for the maximum flow problem. 3

4 2.2 Maintained Data We will provide a generic adjacency list graph implementation. This will be used to maintain G f. In this implementation each edge e will hold c(e) and f(e). It will also hold whether e is an edge or an anti-edge. For the final output, the flow on anti-edges will be meaningless. Each node will hold d(v) and e(v). This will provide O(1) access to all the definitions in 2.1. Each node v also holds current edge(v) which maintains one of v s edges as the current one, and first edge(v) which is the first edge on v s adjacency list. The active node data structures are described in Initialization Initialization() for all edges e: f(e) = 0 for all nodes v: current edge(v) = f irst edge(v) d(s) = n, d(t) = 0 for all v s, t: d(v) = 1 and for all edges e coming from s, Push(s,e) treating e(s) as (see 2.5) 2.4 Termination The algorithm terminates when there are no more active nodes. Correctness is proven in [1]. For details of how termination is determined see Methods Not surprisingly, there are two simple basic methods in push-relabel: push and relabel. Push(v, e = (v, u)) Applicable: if v is active ( e(v) > 0 ) and d(v) = d(u) + 1. amount to push = min {e(v), c f (e)} Add a flow of amount push to edge e. Update e(u), flow on edges (v, u) and (u, v) and consequently the residual graph G f. Relabel(v) Applicable: if v is an active node and no push can be applied to it (the relabel can be run if a push is applicable but it will simply do nothing) d(v) = min (v,u) Ef {d(u) + 1} 4

5 These methods are combined using the Discharge method Discharge(v) Applicable: if v is active (e(v) > 0) next min label(v) = For each edge e = (v, u) of v starting from current edge(v): if d(v) = d(u) + 1 then Push(v,e) if e(v) = 0 then break (exit loop) if d(v) < d(u) + 1 then: if d(u) < next min label(v) then do: next min label(v) = d(u) if e(v) > 0 then: // otherwise next sweep will continue from current edge(v) current edge(v) = f irst edge(v) Relabel(v) using next min label(v) Remembering the current edge(v) after the sweep, in case we did not finish going through all edges, is critical to keeping the time complexity of the algorithm (in fact it keeps the amount of unproductive edge checks at O(nm) during the entire run, see [1] Theorem 4.2). 2.6 Phase II Although this is only remotely described in [3], A.V. Goldberg uses a two phase variant of push-relabel in his h prf implementation. Phase I operates normally as described above (and in 2.7) on all active nodes v in G f with d(v) < n. All active nodes v with d(v) n are accumulated and ignored until phase II when there are no more active nodes v with d(v) < n. Note that nodes v with d(v) n have no path to t, since such a path would be of at most n 1 edges contradictory to the label which is a lower bound on the distance to t. So phase II simply returns all excess flow back to s. Note that phase II is unnecessary if you are interested only in the maximum flow value (it will be e(t) at the end of phase I), or in the minimum cut that creates it. Phase2() Applicable: when all active nodes have d(v) n Run backwards DFS from s: if encountered a black node (already marked by DFS): reduce the minimal flow along the circuit from the entire circuit thus breaking it (removed the minimal edge) when done with edge (v, u): reduce e(u) flow on edge (v, u) thus adding to e(v) 5

6 2.7 Active Node Selection Strategy The general run of the algorithm is to use some active node selection strategy and apply the Discharge method to the active nodes until there are no more active nodes v with d(v) < n. We intend to use the HL - Highest Level active node selection strategy. With this strategy, the algorithm has a proven time bound of O(n 2 m) according to [3]. To implement the HL selection efficiently, a few modifications and some extra data structures are needed. We will now describe these. The algorithm will maintain an array actives of size n (there are n possible label values during phase I of the algorithm). actives[i] will be a doubly linked list holding all the active nodes with label i. highest level will hold the current highest level (highest active node label). next levels will be an array of size n. next levels[i] will be next highest level after level i (the index in actives we should turn to when we are done with the active node list in actives[i]). If there is not a second highest level, next levels[i] = i. Initialize() actives = allemptylists highest level = 1 for every i: next levels[i] = i Note that after Initialize() all active nodes are in level 1 so highest level, actives and next levels are initially correct. Selection() While actives[highest level] is not empty: For some node v in actives[highest level] do: Discharge(v) When there are no more active nodes, highest level will remain the last highest level, and actives[highest level] will be an empty list which will make the loop indeed terminate. We will show correctness by induction on the algorithm steps - we will assume by induction that highest level indeed holds the current highest level of any active node, and that actives and next levels are also correct by the definition above. We can see with that assumption, the Discharge (and therefore Push and Relabel) will always operate on some highest level active node. It remains only to see that Push and Relabel correctly update highest level, actives and next levels (as to prove the induction step). We will add to the Relabel method - 6

7 Relabel(v) remove v from actives[highest level] if d(v) < n: add v to actives[d(v)] if actives[highest level] is now empty: next levels[d(v)] = next levels[highest level] else: next levels[d(v)] = highest level highest level = d(v) else: if actives[highest level] is now empty: highest level = next levels[highest level] Obviously if v was active before Relabel, then it remains active afterwards (only the label changes). So v is updated into the correct list in actives. If we assume highest level used to be the highest level of active nodes and since d(v) > highest level (Relabel strictly increases labels, see [1]), then there are no other active nodes with label between highest level and d(v). Therefore it is easy to see that next levels is updated correctly. It also entails that d(v) is the new highest level. We will add to the Push method - Push(v, e = (v, u)) if e(u) < 0 then u previously not active = true if u previously not active = true and e(u) > 0: add u to actives[highest level 1] next levels[highest level] = highest level 1 if e(v) = 0: remove v from actives[highest level] if actives[highest level] is now empty: highest level = next levels[highest level] From the definition of when Push is applicable, we see that d(v) = highest level and d(u) = highest level 1. If u becomes active during a push into u (this is the only way a node can become active) then we update actives[d(u)]. Obviously the second highest level with active nodes will now be highest level 1 (there are no other levels between highest level and highest level 1) and so we update next levels correctly. highest level did not change, as no higher active nodes were created. If v becomes inactive during a push from v (this is the only way a node can become inactive) then we update actives[d(v)]. We 7

8 rely on the inductive correctness of next levels and update highest level to be the next highest level that still has some active nodes. The primary result of this effort, is that we obtain O(1) access to some highest level active node for every discharge, with O(1) update cost at every Push or Relabel operation. This enables us to implement the HL active node selection with minimum cost (and no asymptotic theoretical cost). Notice that h prf code works differently - highest level is updated, but next highest level is not maintained but rather manually searched for by iterating through the levels. This has no asymptotic importance as this costs O(n 2 ) compared to the O(n 2 m) total cost of the algorithm (since we are not implementing this anyhow, no proof is attached). 3 Heuristics 3.1 Global Relabeling Heuristic Overview This heuristic is suggested in [1], [2] and [3], and implemented in h prf (see 1) The Heauristic Every some c m steps of the algorithm run a backwards BFS from t (i.e. find shortest paths from nodes to t rather than vice-versa), and update node labels according to the BFS label. If t is reachable from node v and d is the level BFS assigned to v, then update d(v) = d. For all nodes v from which t is not reachable, assign d(v) = n (so they will be ignored until phase II, see 2.6) Correctness Layout It can be shown that the heuristic does not harm the correctness of the algorithm, since a global relabeling is the same as running a normal relabel operation on all nodes in a certain order (in fact a single relabel is the exact basic step of a BFS) Code We intend to implement this heuristic in our push-relabel implementation. Notice that highest level, actives and next levels will have to be updated during each global relabeling, whenever we encounter a BFS level with active nodes Variation We will run a global relabeling if either of the following applies - 1. c m work (push / relabel) has been done since the last global relabel. 8

9 2. c max flow bound m/n 2 units of flow or more have been added to t since the last global relabel (this is inspired from [2]). max flow bound is a parameter obtainable through the level-cut heuristic (see 3.2). (1) is suggested already in [1]. Since it is performed at most once every O(m) work of the algorithm and performs O(m) work itself, it obviously does not change the asymptotic time bound of the algorithm. (2) is our own variation. In general, max flow bound will always be an n upper bound on maximum flow of the network. Since after at most c m 2 times that (2) is performed, maximum flow has reached t, t is no longer reachable by any active node (otherwise excess could still reach the sink), thus one more global relabeling will cause all active nodes to have label > n and end phase I. Therefore (2) is performed at most O(n 2 / m) times and since every global relabeling takes O(m) time (BFS), we reach a total maximum cost of O(n 2 m) which keeps the algorithm within its asymptotic time bound. 3.2 Level-Cut Heuristic Overview This heuristic generalizes the gap relabeling heuristic suggested in [3], in the sense that it is expected to improve in at least any scenario in which gap relabeling is expected to improve. It is a simple heuristic in concept. Generally described, we maintain a number of certain cut values in G f. We attempt to never push more flow towards t then some minimum cut which blocks us allows, since this flow will never reach t anyhow Intention Hopefully, if we push less extra flow towards t, we will have less flow to push back and try through other paths. This is the heuristic intention. There are easy obvious examples where this heuristic works, such as the one below, but it is not clear how it performs in practice. In the example in Figure 1, a is the only active node. We can see clearly that normally we would push 2 units of flow towards t, then push back 1 unit of flow from c back to a. We will see that with the level-cut heuristic, the cut of {a, b, c} {d, f} is recognized as a cut with cut value = 1, and therefore only 1 unit of flow will be pushed all the way to t, and no flow will be pushed back from c to a, thus saving unnecessary pushes Gap Relabeling Heuristic In gap relabeling heuristic we maintain an array A of size n, holding in A[i] the number of nodes for each label (up to n). If a label d is found, such that A[d] = 0, then all nodes with label > d are relabeled to label n. This is due to 9

10 Figure 1: Level Cut Heuristic Example the observation that t is unreachable from all such nodes (see proof in 7.1). Let S + d be the set of nodes with label d (which includes the source), and S d the set of nodes with label < d (which includes the sink), for any 0 < d < n. To see how our heuristic generalizes the gap relabeling heuristic, we need only to note that whenever the gap relabeling heuristic applies to level d (i.e. A[d] = 0), (S + d, S d ) is a cut in G f with cut value = 0 (in fact, in this case, (S + d+1, S d+1 ) will be the same cut). The cut value is 0, since no residual edge exists between S + d and S d with orientation towards S d (a consequent of the proof in 7.1) Code Maintain an array cuts of size n. cuts[i] will be the cut value induced by the cut (S + i, S i ) in G f. Also maintain an array min cuts of size n. min cuts[i] will be min j i {cuts[j]}. For any node v with d(v) = i, v S + j for any j i. Therefore the amount of flow that v further pushes that can actually reach t is limited by any of the cuts (S + j, S j ) such that j i. Therefore it is limited by min cuts[i]. We use this information to make the following modification to Push. Push(v, e = (v, u)) if min cuts[d(v)] < c f (e): c(e) = min cuts[d(v)] + f(e) update c f (e) by definition // = min cuts[d(v)] 10

11 To deal with nodes that have been blocked out by cuts from reaching t, but are stuck with excess that will return to s during phase II, we add the following adjustments. Relabel(v) if v has no edges: d(v) = n Phase2() restore all original capacities of residual edges In addition, to imitate the effects of gap relabeling (and more), we can add Discharge(v) if min cuts[d(v)] = 0: d(v) = n For a general understanding, it suffices to say that changing the capacities alters the attempts to push flow towards t, preventing us from pushing unnecessary excess flow towards t, that would have otherwise never reached it because of the minimum cut. However, to enable the excess flow to later return to s, we must keep active nodes until phase II before which we will restore all original capacities to enable the excess to return to s. Note that even without the addition to Discharge, a node v with min cuts[d(v)] = 0 would not have pushed any flow at any rate. However, this check saves the iteration through its edges Correctness Layout For an attempt at a full proof of correctness, we follow the proof of push-relabel in [1] lemma The sections that invalidate are Theorem 3.4 and lemma 3.5 (and consequently 3.7). Lemma 3.5 is irrelevant when we use the two phase variant of push-relabel as A.V. Goldberg does in h prf code, and lemma 3.7 (all labels < n) becomes immediate and does not require lemma 3.5. We will show the correctness of Theorem 3.4 (the terminating flow is maximal). Like the correctness of the algorithm without the level-cut heuristic, when phase I terminates all nodes that can reach t have 0 excess and therefore the flow is maximal. However, when we use the heuristic, the flow is maximal for the terminating graph which has different capacities than the original graph. We will show that this terminating graph has the same maximal flow, by showing 11

12 that each change of capacity did not change the maximum flow possible in the graph. To see this, let s look at the situation just before the capacity change of edge e at time t 1. Let s define the current flow as the current pre-flow deducted by some flow from s to active nodes that turns it into a legal flow (such a deduction always exists according to the flow decomposition theory of Ford and Falkerson, which is explained in [4] p.752). The maximum flow function for G, must be a sum of the current flow f 1 through G and some extra flow f 2 through G f. This is true in fact for any flow (so all the more for f 1 at time t 1 ) and it is the grounds for using the residual network with augmenting paths for the maximum flow problem. Obviously by definition f 1 (e) < f(e). f 2 (e) is bounded anyhow by the cut in G f that produced min cuts[i], so its bounded f 2 (e) < min cuts[i]. We receive that the capacity is reduced to min cuts[i] + f(e) > f 2 (e) + f 1 (e), so f 1 + f 2 is still a possible flow through G. The residual capacity is reduced to min cuts[i] > f 2 (e), so f 2 is still a possible additive flow through G f at time t Maintaining Minimum Cuts To maintain cuts and min cuts we add the following - Push(v, e = (v, u)) cuts[d(v)] = cuts[d(v)] amount push Discharge(v) if min cuts[d(v)] = 0: d(v) = n next min label(v) = add to cut(v) = 0 For each edge e = (v, u) of v starting from current edge(v): if d(v) = d(u) + 1 then Push(v,e) if e(v) = 0 then break (exit loop) if d(v) < d(u) + 1 then: if d(u) < next min label(v) then do: next min label(v) = d(u) add to cut(v) = 0 else if d(u) = next min label(v) then do: add to cut(v)+ = c f (v, u) if e(v) > 0 then: // otherwise next sweep will continue from current edge(v) current edge(v) = f irst edge(v) Relabel(v) using using next min label(v), add to cut(v) Relabel(v) if min cuts[d(v)] = 0: 12

13 cuts[new label]+ = add to cut for each i in range [old label, new label]: min cuts[i] = min {cuts[i], min cuts[i 1]} Obviously the push operation maintains cuts[i] correctly. The Relabel operation can add several edges to the cut in the new level it is moved to. However the total residual capacity of these edges can be pre-calculated when going through all the edges of v in the Discharge method. So cuts[i] is updated here correctly as well. min cuts[i] is only updated whenever a Relabel operation elevates a node to a level higher than i. Since we always work on the highest level active node, we will not access min cuts[i] before some active node elevated above i, an operation which would update min cuts. More so, while the highest level > i, min cuts[i] will remain correct since changes to cuts[j] with j > i do not effect min cuts[i]. A simpler update to cuts and min cuts should be inserted into GlobalRelabel() as well A Maximum Flow Bound For Global Relabeling If we maintain also a lowest level (like in the h prf implementation), then an easy estimate for max flow bound used in the global relabeling heuristic can be - max flow bound = min {min cuts[lowest level] + e(t), {e(v) v is active} + e(t)} (the maximum flow is the excess that t already contains + the maximum flow from all active nodes which is bounded by the minimum cut that contains all of them or their own total excess). Notice the second argument to the minimum remains a constant during the algorithm (in fact it is the sum of s s edge capacities). 3.3 Timestamp Heuristic Overview This heuristic is rather simple. Keep a timestamp during the algorithm - it will be a counter, which will advance by one in every push. Every edge will have the timestamp of its last push (an edge and anti-edge will share the same timestamp). Keep the adjacency list of nodes sorted by the timestamp, from smallest (oldest) to biggest (newest), and so Discharge will go through the edges of a node in that order. Note that to maintain correctness we will change the order of a node s edges only once a full sweep on its edges has been completed Intention When flow pushed towards t is blocked by a saturated edge, it is naturally pushed back to search for other routes. However, this push back process can be ineffective resulting in many back and forth flow pushes on the same edges before the flow reaches another path. For example look at Figure 2-13

14 Figure 2: Timestamp Heuristic Example d is the only active node with e(d) = 4. Let s assume a worst case order of each node s edges in its adjacency list - b s edges are ordered ((b, c), (b, a)) c s edges are ordered ((c, d), (c, b)) d s edges are ordered ((d, t), (d, c)) Now if we follow the Discharge operations and always make sure to go through edges in the above order. Flow will pushed to c, then back to d, then back to c, then back to b, back to c, back to d, back to c, back to b, back to a. We receive a total of 9 push operations before the excess of 4 reaches node a (and it becomes active). If we use a timestamp and always turn first to the edge least recently used, we receive only 3 push operations before the excess reaches node a Correctness Layout The order in which we perform each sweep on a node s edges in the Discharge method is a degree of freedom in the generic push-relabel algorithm. It therefore does not harm its correctness Code We will keep 2 adjacency lists for each node v, which will keep references to the same edges only in a different order. One will be v s current edge list. The other will be v s next edge list. After completion of a sweep, we will swap between the lists. Discharge(v) if e(v) > 0 then: 14

15 swap v s edge lists current edge(v) = first edge(v) in the new list Push(v, e = (v, u)) move (v, u) to end of v s next edge list move (u, v) to end of u s next edge list Since the global timestamp always strictly increases, it is in fact (a little surprisingly) not necessary at all to maintain an actual timestamp. Moving the edge and anti-edge to the end of the list, keeps the edge list in timestamp order without actually knowing the timestamp itself. 4 Implementation Details We will implement the code in C++, using Visual Studio IDE. We will require two general structures/classes - PushRelabel - implementing the push-relabel algorithm with all the methods shown in sections 2 and 3. ResidualGraph - a structure to hold the residual graph and read it from input. 5 Graphs And Work Plan We intend to run on precisely the same graph families as in [3], so that our results will be as comparable as possible. These are - 1. Genrmf-Long graphs, generated by the genrmf.c program. 2. Genrmf-Wide graphs. generated by the genrmf.c program. 3. Washington-RLG-Long graphs generated by the washington.c program. 4. Washington-RLG-Wide graphs, generated by the washington.c program. 5. Washington-Line-Moderate graphs, generated by the washington.c program. 6. Acyclic-Dense graphs generated by the ac.c program. 7. AK graphs, generated by the ak.c program. All programs are described in synthetic.htm and available in The programs take a seed as a parameter, so we will save the seeds with the results to make them traceable. For exact running parameters, see [3]

16 We intend to run on all graphs in 9 modes, measuring running time for each. Each mode will be one of the Cartesian product of {globalrelabeling, globalrelabelingwithvariation} {cut levelheuristic, nocut levelheuristic} {timestampheuristic, notimestampheuristic}. 6 Testing Plan We intend to test our code against the h prf code. We will verify legal flow in the output and also compare the maximal flow value with the h prf output. This way we will ensure correct maximal flow - the flow will be legal due to the first check, and maximal because it will be the same as h prf flow value. We will run this test on all input graphs. 7 Appendix 7.1 Correctness Of Gap Relabeling Lemma 7.1 If a gap exists in the node labels between label L 1 and label L 2 such that L 1 L 2 2 (i.e. any node v has d(v) L 1 or d(v) L 2 ), then there cannot be any edge (u, v) such that d(u) L 2 and d(v) L 1. Proof: Lets assume in contradiction there is such an edge (u, v). Lets look at the time t 2 in which this edge (u, v) exists and crosses the gap. Labels strictly increase, so we will note that (*) d(v) L 1 during all the run of the algorithm until and including t 2. At the point t 1 < t 2 when u was first relabeled to a label L 2, (u, v) could not have existed since otherwise d(u) would have been at most d(v) + 1 L L 2 1 < L 2. So (u, v) did not exist at the point t 1 < t 2. But since at point t 1 (u, v) exists again, it must have been re-entered into G f by adding flow to the anti-edge (v, u). Therefore flow was pushed on (v, u) sometime before t 1 and therefore d(v) = d(u) + 1 since we only push flow one level down. But this concludes that d(v) = d(u) + 1 L > L 1 which is a contradiction to (*). References [1] A. V. Goldberg and R. E. Tarjan. A New Approach to the Maximum Flow Problem. In Proc. 18th Annual ACM Symposium on Theory of Computing, pages , [2] A. V. Goldberg and R. Kennedy, Global Price Updates Help. SIAM Journal of Discrete Mathematics, Vol. 10, pages , [3] B. V. Cherkassky and A. V. Goldberg, On Implementing Push-Relabel Method for the Maximum Flow Problem. Algorithmica, Vol. 19, pages ,

17 [4] R. K. Ahuja; James B. Orlin, A Fast and Simple Algorithm for the Maximum Flow Problem. Operations Research, Vol. 37, No. 5. (Sep. - Oct., 1989), pp