Linear Programming in Subexponential Time Linear Programming Algebraic View

for x, c ∈ ℝ^{d}; b ∈ ℝ^{n}; A ∈ ℝ^{(n×d)} min c^{T} x subject to Ax ≤ b

Linear Programming Geometric View

2

Goals

The Holy Grail polynomial time algorithm in unit-cost model

Previous Work polynomial time algorithm in bit-cost model O(d!n) time algorithm in unit-cost model (Seidel)

Today subexponential time algorithm in unit-cost model

Outline

SolveLP1 — Clarkson (1988) SolveLP2 — Clarkson (1988) BasisLP — Matousek, Sharir and Welzl (1992)

also Kalai (1992)

Combined Result: O(d^{2}n + e^{O(√d log d)})

Definitions

H . . . . . . . . . . . . . set of n linear constraints. v(H). . . . . . . . . . . optimum point subject to H. a basis . . . . . . . . . minimal set B with v(B) > −∞. a basis for H … basis B ⊆ H s.t. v(B) = v(H).

Note: the size of a basis is d.

Example

6

Random Sampling

Randomly reduce large problem to small subproblem. Solve small subproblem. If solution solves large problem, you win. Otherwise, get hints from wrong solution and try again.

Algorithm: SolveLP1

SolveLP1(H) G := ∅ repeat

R := random subset of H of size r B := SolveLP1(G ∪ R) V := set of constraints violated by v(B) if |V | is small enough, then G := G ∪ V

until V = ∅

Example

r = 2, “small enough” = 1 key: R = green V = red G = blue

Analysis

Bounding the number of iterations until a successful one

number of violated constraints is nd/r

E[|V |] = n Pr(constraint is violated) ≤ n(d/(r + 1))

“small enough” := 2nd/r

number of iterations until a successful one is 2 Pr(|V | ≥ 2E[|V |]) < 1/2 SolveLP1(H)

G := ∅ repeat

R := random subset of H of size r B := SolveLP1(G ∪ R) V := set of constraints violated by v(B) if |V | ≤ 2nd/r, then G := G ∪ V

until V = ∅

Analysis

Bounding the number of successful iterations

If no basis constraint is violated,

then the solution must be the optimal solution!

In every successful iteration, a basis constraint is added to G!

number of successful iterations is d

Analysis

Running time

number of constraints in recursive call is at most r + 2nd^{2}/r

r + |G| ≤ r + d|V | ≤ r + 2nd^{2}/r

running time is

T_{1}(n) ≤ 2dT_{1}(r + 2nd^{2}/r) + O(d^{2}n)

r:=d√2n

base case is n = O(d^{2}) n ≤ d√2n ⟹ n ≤ 2d^{2} SolveLP1(H)

if n ≤ 2d^{2}, then return solution G := ∅ repeat

R := random subset of H of size d√2n B := SolveLP1(G ∪ R) V := set of constraints violated by v(B) if |V | ≤ √2n, then G := G ∪ V

until V = ∅

Analysis with Seidel

SolveLP1(H) if n ≤ 2d^{2}, then return Seidel(H) G := ∅ repeat

R := random subset of H of size d√2n B := Seidel(G ∪ R) V := set of constraints violated by v(B) if |V | ≤ √2n, then G := G ∪ V

until V = ∅

Analysis with Seidel

Plug in Seidel S(n, d) = O(d!n)

running time is O(d^{2}n + d^{2}d!√n)

T (n) = 2dS(d√n, d) + O(d^{2}n)
= O(d^{2}n + d^{2}d!√n)

Improving the Algorithm

Reduce sample size

Instead of forcing the violated constraints, include them in random sample with higher probability!

H is a multiset

SolveLP2(H)

repeat R := random subset of H of size r B := SolveLP2(R) V := multiset of constraints violated by v(B) if |V | is small enough, then H := H ∪ V

until V = ∅

Analysis

Bounding the number of iterations until a successful one number of violated constraints is |H|d/r

“small enough” := 2|H|d/r

number of iterations until a successful one is 2

H is a multiset

SolveLP2(H)

repeat R := random subset of H of size r B := SolveLP2(R) V := multiset of constraints violated by v(B) if |V | ≤ 2|H|d/r, then H := H ∪ V

until V = ∅

Analysis

Bounding the number of successful iterations Fix a basis B of H.

in each iteration, multiplicity of a constraint in B is doubled after kd iterations, some constraint in B has multiplicity

at least 2^{k}

in each iteration, size of H increases by 2|H|d/r after kd iterations, |H| is at most n exp (2d^{2}/r)^{k}

|H| ≤ n(1 + 2d/r)^{kd} < n exp (2kd^{2}/r)

2^{k} < n exp (2d^{2}/r)^{k}

Analysis

Bounding the number of successful iterations

2^{k} < n exp (2d^{2}/r)^{k}

algorithm will terminate when inequality is violated minimize r subject to 2 ≥ exp (2d^{2}/r)

r = O(d^{2})

minimize k subject to 2^{k} ≥ ne^{k/2} k = O(log n)

H is a multiset

SolveLP2(H)

repeat R := random subset of H of size 4d^{2} B := SolveLP2(R) V := multiset of constraints violated by v(B) if |V | ≤ |H|/2d, then H := H ∪ V

until V = ∅

Analysis

Running Time

running time is

T_{2}(n) ≤ O(d log nT′(d^{2}) + d^{2}n log n)

base case is n = O(d^{2})

H is a multiset

SolveLP2(H)

repeat R := random subset of H of size 4d^{2} B := solution for R V := multiset of constraints violated by v(B) if |V | ≤ |H|/2d, then H := H ∪ V

until V = ∅

Analysis with SolveLP1 and Seidel

SolveLP1(H) if n ≤ 2d^{2}, then return SolveLP2(H) G := ∅ repeat

R := random subset of H of size d√2n B := SolveLP2(G ∪ R) V := set of constraints violated by v(B) if |V | ≤ √2n, then G := G ∪ V

until V = ∅

Analysis with SolveLP1 and Seidel

H is a multiset

SolveLP2(H)

repeat R := random subset of H of size 4d^{2} B := Seidel(R) V := multiset of constraints violated by v(B) if |V | ≤ |H|/2d, then H := H ∪ V

until V = ∅

Analysis with SolveLP1 and Seidel

Plug in Seidel and plug into SolveLP1

S(n, d) = O(d!n)

running time is O(d!d^{3} log n + d^{3}√n log n + d^{2}n)

T (n) = O(dT_{2}(d√n) + d^{2}n)
= O(d(d log nS(d^{2}, d) + d^{2}√n log n) + d^{2}n)
= O(d!d^{3} log n + d^{3}√n log n + d^{2}n)

Solving Small LPs

Seidel(H) if |H| = d, then return H else

Pick a random constraint c ∈ H B := Seidel(H − {c}) if v(B) does not violate c, then return B else project constraints onto c and solve

running time is O(d!n)

S(n, d) = S(n − 1, d) + (d/n)S(n − 1, d − 1) + O(d^{2})

Algorithm: BasisLP

Modify Seidel!

BasisLP(H,B) Pick a random constraint in c ∈ H − B B := BasisLP(H − {c},B) if v(B) does not violate c, then return B else return BasisLP(H,Basis(B ∪ {c}))

Analysis Idea

Define a parameter that captures the quality of the hint! hint quality k is (roughly) d − |B_{hint} ∩ B_{real}|

T (n, k) ≤ T (n − 1, k) + \frac{1}{n − d} k ; ∑ ; i=1(1 + T (n, k − i))

running time is O (nd exp (√d ln(n + 1) ))

Conclusion

Use BasisLP as base of recursion!

running time is O(d^{2}n + e^{O(√d log d)})