network flow (7)

capacity-scaling algorithm

analysis ford-fulkerson alg (with integral capacities)

Assumption: Every edge capacity \(c(e)\) is an integer in \([1,C]\)

Integrality invariant: Throughout Ford-Fulkerson, every edge flow \(f(e)\) and residual capacity \(c_f(e)\) is an integer

Pf: By induction on the number of augmenting paths \(\quad\qed\)

Theorem: Ford-Fulkerson terminates after at most \(\mathit{val}(f^*) \leq nC\) augmenting paths, where \(f^*\) is a max flow (consider cut \(A = \{s\}\); assumes no parallel edges)

Pf: Each augmentation increases value of flow by at least \(1\) \(\qed\)

Corollary: The running time of Ford-Fulkerson is \(O(mnC)\)

Pf: Can use either BFS or DFS to find an augmenting path in \(O(m)\) time \(\qed\)

analysis ford-fulkerson alg (with integral capacities)

Assumption: Every edge capacity \(c(e)\) is an integer in \([1,C]\)

• Integrality invariant: Throughout Ford-Fulkerson, every edge flow \(f(e)\) and residual capacity \(c_f(e)\) is an integer
• Theorem: Ford-Fulkerson terminates after at most \(\mathit{val}(f^*) \leq nC\) augmenting paths, where \(f^*\) is a max flow
• Corollary: The running time of Ford-Fulkerson is \(O(mnC)\)

Integrality theorem: There exists an integral max flow \(f^*\) (\(f(e)\) is an integer for every \(e\))

Pf: Since Ford-Fulkerson terminates, theorem follows from integrality invariant (and augmenting path theorem) \(\quad\qed\)

Ford-Fulkerson: exponential example

Q: Is generic Ford-Fulkerson algorithm poly-time in input size (\(m\), \(n\), and \(\log C\))?

A: No. If max capacity is \(C\), then algorithm can take \(\geq C\) iterations

\(s \rightarrow a \rightarrow b \rightarrow t\) \(s \rightarrow b \rightarrow a \rightarrow t\) \(s \rightarrow a \rightarrow b \rightarrow t\) \(s \rightarrow b \rightarrow a \rightarrow t\) \(s \rightarrow a \rightarrow b \rightarrow t\) \(s \rightarrow b \rightarrow a \rightarrow t\) ...

each augmenting path sends only 1 unit of flow (number of augmenting paths is \(2C\))

fork-fulkerson algorithm: exponential-time example

Bad news: Number of augmenting paths can be exponential in input size

‹ □ 0 play: 1fps play: 2fps play: 4fps play: 8fps ›

quiz: network flow

The Ford-Fulkerson algorithm is guaranteed to terminate if the edge capacities are ...

1. Rational numbers
2. Real numbers
3. Both A and B
4. Neither A nor B

choosing good augmenting paths

Use care when selecting augmenting paths

• Some choices lead to exponential algorithms
• Clever choices lead to polynomial algorithms

Pathology: When edge capacities can be irrational, no guarantee that Ford-Fulkerson terminates (or converges to a maximum flow)!

ford-fulkerson algorithm: pathological example

Intuition: Let \(r > 0\) satisfy \(r^2 = 1 - r\)

• Initially, some residual capacities are \(1\) and \(r\)
• After two augmentating paths, some residual capacities are \(r\) and \(r^2\) (\(1-r\))
• After two more augmenting paths, some residual capacities are \(r^2\) and \(r^3\) (\(r-r^2\))
• After two more, some residual capacities are \(r^3\) and \(r^4\)
• By carefully choreographing the augmenting paths, infinitely many residual capacities arise!

\[\begin{eqnarray} r = \frac{\sqrt{5} - 1}{2} & \quad\Rightarrow\quad & r^2 = 1 - r \\ r \approx 0.618 & \quad\Rightarrow\quad & r^4 < r^3 < r^2 < r < 1 \end{eqnarray}\]

ford-fulkerson algorithm: pathological example

• \(r^2 = 1 - r\)
• \(C\) is sufficiently large that it won't ever be a bottleneck (suppressing labels for clarity)

‹ □ 0 play: 1fps play: 2fps play: 4fps play: 8fps ›

ford-fulkerson algorithm: pathological example

flow after augmenting path:

• 1: \(\{ r-r^1, 1, 1-r^0 \}\), \(\mathit{val}(f) = 1\)
• 5: \(\{ r-r^3, 1, 1-r^2 \}\), \(\mathit{val}(f) = 1+2r+2r^2\)
• 9: \(\{ r-r^5, 1, 1-r^4 \}\), \(\mathit{val}(f) = 1+2r+2r^2+2r^3+2r^4\)

ford-fulkerson algorithm: pathological example

Theorem: the Ford-Fulkerson algorithm may not terminate; moreover, it may converge to a value not equal to the value of the maximum flow

Pf:

• With augmenting paths of form just descripbed and \(r = \frac{\sqrt{5} - 1}{2}\)
• After \((1+4k)\) augmenting paths, the value of the flow \[ \mathit{val}(f) = 1 + 2 \sum_{i=1}^{2k} r^i \leq 1 + 2\sum_{i=1}^\infty r^i = 1 + \frac{2r}{1-r} < 5 \]
• Value of maximum flow = \(2C+1 \quad\qed\)

choosing good augmenting paths

Use care when selecting augmenting paths

• Some choices lead to exponential algorithms
• Clever choices lead to polynomial algorithms

Pathology: When edge capacities can be irrational, no guarantee that Ford-Fulkerson terminates (or converges to a maximum flow)!

Goal: Choose augmenting paths so that

• Can find augmenting paths efficiently
• Few iterations

choosing good augmenting paths

Choosing augmenting paths with

• Max bottleneck capacity ("fattest") (how to find?)
• Sufficiently large bottleneck capacity (next)
• Fewest edges (ahead)

Dinitz invented algorithm in response to a pre-class exercise by Adelson-Velsky

capacity-scaling algorithm

Overview: Choosing augmenting paths with "large" (though not necessarily largest) bottleneck capacity

• Maintain scaling parameter \(\Delta\)
• Let \(G_f(\Delta)\) be the part of the residual network containing only the edges with capacity \(\geq \Delta\)
• Any augmenting path in \(G_f(\Delta)\) has bottleneck capacity \(\geq \Delta\)

capacity-scaling algorithm

Capacity-Scaling(G)
    Foreach edge e in E: f(e) <- 0
    Delta <- largest power of 2 <= C
    While(Delta >= 1)
        // body of this loop is Delta-scaling phase
        Gf(Delta) <- Delta-residual network of G with respect to flow f
        While(there exists an s-t path P in Gf(Delta))
            f <- Augment(f, c, P)
            Update Gf(Delta)
        Delta <- Delta / 2
    Return f

capacity-scaling alg: proof of correctness

Assumption: All edge capacities are integers between \(1\) and \(C\)

Invariant: The scaling parameter \(\Delta\) is a power of 2

Pf: Initially a power of 2; each phase divides \(\Delta\) by exactly 2 \(\qed\)

Integrality invariant: Throughout the algorithm, every edge flow \(f(e)\) and residual capacity \(c_f(e)\) is an integer.

Pf: Same as for generic Ford-Fulkerson \(\qed\)

capacity-scaling alg: proof of correctness

Assumption: All edge capacities are integers between \(1\) and \(C\)

• Invariant: The scaling parameter \(\Delta\) is a power of 2
• Integrality invariant: Throughout the algorithm, every edge flow \(f(e)\) and residual capacity \(c_f(e)\) is an integer.

Theorem: If capacity-scaling algorithm terminates, then \(f\) is a max flow

Pf:

• By integrality invariant, when \(\Delta = 1 \Rightarrow G_f(\Delta) = G_f\)
• Upon termination of \(\Delta = 1\) phase, there are no augmenting paths
• Results follow augmenting path theorem \(\qed\)

capacity-scaling alg: analysis of running time

Lemma 1: There are \(1 + \lfloor\log_2 C\rfloor\) scaling phases

Lemma 2: Let \(f\) be the flow at the end of a \(\Delta\)-scaling phase. Then, the max-flow value \(\leq \mathit{val}(f) + m\Delta\)

Lemma 3: There are \(\leq 2m\) augmentations per scaling phase

capacity-scaling alg: analysis of running time

Lemma 1: There are \(1 + \lfloor\log_2 C\rfloor\) scaling phases

Pf: Initially \(C / 2 < \Delta \leq C\); \(\Delta\) decreases by a factor of \(2\) in each iteration \(\qed\)

capacity-scaling alg: analysis of running time

Lemma 2: Let \(f\) be the flow at the end of a \(\Delta\)-scaling phase. Then, the max-flow value \(\leq \mathit{val}(f) + m\Delta\)

Pf:

• Show \(\exists\) cut \((A,B)\) such that \(\mathit{cap}(A,B) \leq \mathit{val}(f) + m\Delta\)
• Choose \(A\) to be the set of nodes reachable from \(s\) in \(G_f(\Delta)\)
• By definition of \(A\): \(s \in A\). By definition of flow \(f\): \(t \notin A\)

\[ \begin{eqnarray} \mathit{val}(f) & =^\dagger & \sum_{e\text{ out of }A} f(e) - \sum_{e\text{ in to }A} f(e) \\ & \geq & \sum_{e\text{ out of }A} (c(e) - \Delta) - \sum_{e\text{ in to }A}\Delta \\ & \geq & \sum_{e\text{ out of }A} c(e) - \sum_{e\text{ out of }A}\Delta - \sum_{e\text{ in to }A}\Delta \\ & \geq & \mathit{cap}(A,B) - m\Delta \quad\qed \end{eqnarray} \] \(\dagger\): flow value lemma

capacity-scaling alg: analysis of running time

Lemma 3: There are \(\leq 2m\) augmentations per scaling phase

Pf:

• Let \(f\) be the flow at the beginning of a \(\Delta\)-scaling phase (or equivalently, at the end of a \(2\Delta\)-scaling phase)
• Lemma 2 \(\Rightarrow\) max-flow value \(\leq \mathit{val}(f) + m(2\Delta)\)
• Each augmentation in a \(\Delta\)-phase increases \(\mathit{val}(f)\) by at
  least \(\Delta\) \(\quad\qed\)

capacity-scaling alg: analysis of running time

Lemma 1: There are \(1 + \lfloor\log_2 C\rfloor\) scaling phases

Lemma 2: Let \(f\) be the flow at the end of a \(\Delta\)-scaling phase. Then, the max-flow value \(\leq \mathit{val}(f) + m\Delta\)

Lemma 3: There are \(\leq 2m\) augmentations per scaling phase

Theorem: the capacity-scaling algorithm takes \(O(m^2 \log C)\) time

Pf:

• Lemma 1 + Lemma 3 \(\Rightarrow O(m \log C)\) augmentations
• Finding an augmenting path takes \(O(m)\) time \(\quad\qed\)

network flow (7)

shortest augmenting paths

shortest augmenting path

Q: How to choose next augmenting path in Ford-Fulkerson?

A: Pick one that uses the fewest edges (can find via BFS)

Shortest-Augmenting-Path(G)
    Foreach e in E: f(e) <- 0
    Gf <- residual network of G with respect to flow f
    While(there exists an s-t path P in Gf)
        P <- Breadth-First-Search(Gf)
        f <- Augment(f, c, P)
        Update Gf
    Return f

shortest augmenting path: overview of analysis

Lemma 1: The length (number of edges) of a shortest augmenting path never decreases. (Pf later)

Lemma 2: After at most \(m\) shortest-path augmentations, the length of a shortest augmenting path strictly increases (Pf later)

Theorem: The shortest-augmenting-path algorithm take \(O(m^2n)\) time

Pf:

• \(O(m)\) time to find a shortest augmenting path via BFS
• There are \(\leq mn\) augmentations
  • at most \(m\) augmenting paths of length \(k\) (Lemmas 1 + 2)
  • at most \(n-1\) different lengths (augmenting paths are simple paths) \(\qed\)

shortest augmenting path: analysis

Defn: Given a digraph \(G = (V,E)\) with source \(s\), its level graph is defined by:

• \(\ell(v) =\) number of edges in shortest \(s \leadsto v\) path
• \(L_G = (V,E_G)\) is the subgraph of \(G\) that contains only those edges \((v,w) \in E\) with \(\ell(w) = \ell(v)+1\)

quiz: network flow

Which edges are in the level graph of the following digraph?

1. \(c \rightarrow t\)
2. \(d \rightarrow t\)
3. Both A and B
4. Neither A nor B

shortest augmenting path: analysis

Defn: Given a digraph \(G = (V,E)\) with source \(s\), its level graph is defined by:

• \(\ell(v) =\) number of edges in shortest \(s \leadsto v\) path
• \(L_G = (V,E_G)\) is the subgraph of \(G\) that contains only those edges \((v,w) \in E\) with \(\ell(w) = \ell(v)+1\)

Key property: \(P\) is a shortest \(s \leadsto v\) path in \(G\) iff \(P\) is an \(s \leadsto v\) path in \(L_G\)

shortest augmenting path: analysis

Lemma 1: The length (number of edges) of a shortest augmenting path never decreases

Pf:

• Let \(f\) and \(f'\) be flow before and after a shortest-path augmentation
• Let \(L_G\) and \(L_{G'}\) be level graphs of \(G_f\) and \(G_{f'}\)
• Only back edges added to \(G_{f'}\) (any \(s \leadsto t\) path that uses a back edge is longer than previous length) \(\qed\)

shortest augmenting path: analysis

Lemma 2: After at most \(m\) shortest-path augmentations, the length of a shortest augmenting path strictly increases

Pf:

• At least one (bottleneck) edge is deleted from \(L_G\) per augmentation
• No new edge added to \(L_G\) until shortest path length strictly increases \(\qed\)

shortest augmenting path: analysis review

Lemma 1: The length (number of edges) of a shortest augmenting path never decreases. (Pf later)

Lemma 2: After at most \(m\) shortest-path augmentations, the length of a shortest augmenting path strictly increases (Pf later)

Theorem: The shortest-augmenting-path algorithm take \(O(m^2n)\) time

shortest augmenting path: improving the running time

Note: \(\Theta(mn)\) augmentations necessary for some flow networks

• Try to decrease time per augmentation instead
• Simple idea: \(O(mn^2)\), Dinitz 1970 (ahead)
• Dynamic trees: \(O(mn\log n)\), Sleator-Tarjan 1983

network flow (7)

dinitz' algorithm

Dinitz' algorithm

Start by constructing level graph \(L_G\) of original graph \(G\)

Two phases of augmentations

• Normal: within phase, length of shortest path does not change
• Special: length of shortest path strictly increases

Normal Augmentation Phase:

• Start at \(s\)
• Advance along edges in \(L_G\) until reach \(t\) or get stuck
  • Reach \(t\): augment flow; update \(L_G\); restart from \(s\)
  • Stuck at \(v\): retreat to prev node and delete \(v\) from \(L_G\)

Special Augmentation Phase:

• Reconstruct new \(L_G\) based on residual capacities
  • If sink never reached, found maximum flow!
  • Otherwise, restart Normal Augmentation Phase

Dinitz' algorithm

Phase of normal augmentations (within a phase, length of shortest augmenting path does not change)

• Start at \(s\)
• Advance along edges in \(L_G\) until reach \(t\) or get stuck
  • Reach \(t\): augment flow; update \(L_G\); restart from \(s\)
  • Stuck: delete node from \(L_G\) and retreat to previous node

‹ □ 0 play: 1fps play: 2fps play: 4fps play: 8fps ›

dinitz' algorithm (as refined by even and itai)

Dinitz(G)
    Foreach e in E: f(e) <- 0
    Gf <- residual network of G with respect to flow f
    Initialize(Gf)

Initialize(Gf)
    LG <- level-graph of Gf  // residual network, not G!
    If(t not reached)
        // found max-flow
        Stop
    P <- empty set
    Goto Advance(s)

// assuming LG,P,s,t,Gf,f are accessible in following functions

Advance(v)
    If(v=t)
        Augment(P)
        Update Gf
        Remove saturated edges from LG
        P <- empty set
        Goto Advance(s)
    If(there exists edge (v,w) in LG)
        Add edge (v,w) to P
        Goto Advance(w)
    Else
        Goto Retreat(v)

Retreat(v)
    If(v=s)
        // switch to special phase
        Goto Initialize(Gf)
    Else
        Delete v (and all incident edges) from LG
        Remove last edge (u,v) from P
        Goto Advance(u)

quiz: network flow

How to compute the level graph \(L_G\) efficiently?

1. Depth-first search
2. Breadth-first search
3. Both A and B
4. Neither A nor B

dinitz' algorithm: analysis

Lemma: A phase can be implemented to run in \(O(mn)\) time

Pf:

• Initialization happens once per phase (\(O(m)\) using BFS)
• At most \(m\) augmentations per phase because an augmentation deletes at least one edge from \(L_G\) (\(O(mn)\) per phase)
• At most \(n\) retreats per phase because a retreat deletes one node from \(L_G\) (\(O(m+n)\) per phase)
• At most \(mn\) advances per phase because at most \(n\) advances before retreat or augmentation (\(O(mn)\) per phase) \(\quad\qed\)

dinitz' algorithm: analysis

Lemma: A phase can be implemented to run in \(O(mn)\) time

Theorem: Dinitz' algorithm runs in \(O(mn^2)\) time

Pf:

• By Lemma, \(O(mn)\) time per phase
• At most \(n-1\) phases as in shortest-augmenting path analysis \(\qed\)

augmenting-path algorithms: summary

\(m\) edges, \(n\) nodes, integer capacites in \([1,C]\)

\(*\): fat paths; \(\dagger\): shortest paths

maximum-flow algorithms: theory highlights

\(m\) edges, \(n\) nodes, integer capacities in \([1,C]\)
\(a=3/2\), \(b=1/2\), \(c=10/7\), \(d=1/7\)

maximum-flow algorithms: practice

Push-relabel algorithm (section 7.4) increases flow one edge at a time instead of one augmenting path at a time.

maximum-flow algorithms: practice

Caveat: Worst-case running time is generally not useful for predicting or comparing max-flow algorithm performance in practice

Best in practice: Push-relabel method with gap relabeling: \(O(m^{3/2})\) in practice

maximum-flow algorithms: practice

Computer vision: different algorithms work better for some dense problems that arise in applications to computer vision

maximum-flow algorithms: matlab

maximum-flow algorithms: google

network flow (7)

simple unit-capacity networks