【MIP】Benders Decomposition & MIP

Benders decomposition

Consider the following general LP, where $x$ is a real valued variable and $y$ is a variable whose domain is defined by polyhedron $\mathbb{Y}$. $y$ is a complicating variable:
$$\begin{array}{rlr}& \min & c^{T}x+f^{T}y\\ & s.t.& Ax+By\ge b\\ & & x\ge 0,y\in \mathbb{Y}\end{array}$$
The Benders decomposition method partitions the problem in two: a master problem containing the $y$ variables and a sub-problem containing the $x$ variables. With $q(y)$ as the incumbent value for the $x$ part, thus, we can define a LP using only variable $y$:
$$\begin{array}{rlr}& \min & f^{T}y+q(y)\\ & s.t.& y\in \mathbb{Y}\end{array}$$
Then, we have the sub-problem in terms of $x$. Note that if the sub-problem is unbounded, the original problem is unbounded as well. Under the assumption of bounded, we calculate the value of $q(y)$ by solving the following LP:
$$\begin{array}{rlr}& \min & c^{T}x\\ & s.t.& Ax\ge b-By\\ & & x\ge 0\end{array}\text{\hspace{1em}(Sub)}$$
Considering the dual variable, $\alpha$ associated with the sub-problem:
$$\begin{array}{rlr}& \max & {\alpha }^T(b-By)\\ & s.t.& A^T\alpha \le c\\ & & \alpha \ge 0\end{array}\text{\hspace{1em}(Dual)}$$
When the solution space is not empty, we can enumerate all extreme rays ${\rho }_i,i\in [1,I]$ and extreme points ${\pi }_j,j\in [1,J]$ of the feasible region. Given a solution vector $y^{\ast }$, we can solve the dual problem by checking if we can find:

• ${\rho }_i$ such that: ${\rho }_i^T(b-B{y}^{\ast })>0$, in which case the dual is unbounded;
• ${\pi }_i$ maximizing: ${\pi }_j^T(b-B{y}^{\ast })$, in which case both the primal and dual have infinite solutions.

Rewriting the sub-problem in terms of a single variable $q$. We have the master problem in terms of $y$ and $q$ only, and the constraints are called cuts in Benders:
$$\begin{array}{rlrlr}& \min & f^{T}y+q& & \\ & s.t.& {\rho }_i^T(b-By)\le 0& \forall i\in [1,I]& (1)\\ & & {\pi }_j^T(b-By)\le q& \forall j\in [1,J]& (2)\\ & & q\in \mathbb{R},y\in \mathbb{Y}& & \end{array}$$
Since there is an exponential number of extreme points and extreme rays, and because their enumeration is $NP$-hard, Benders starts without any cut and solves a relaxed master problem which gives a candidate solution $(y^{\ast },q^{\ast })$. Using this solution, it solves the sub-problem to obtain a value $q(y^{\ast })$. If $q^{\ast }=q(y^{\ast })$ then the candidate solution is optimal for the original problem. Otherwise we have two cases:

• the dual is unbounded, then we select an extreme ray to generate a constraint of type $(4)$, which is called a feasibility cut.
• we have $q(y^{\ast })>q^{\ast }$, then we select and exteren point to add a constraint of type $(5)$, which is called an optimality cut as it gives a lower bound for the incumbent.

The algorithm then repeats the previous steps: it solves the master problem, now with an additional constraint, and checks wheter the solution is optimal or if a new cut is needed.

Xpress demo

Problem:
$$\begin{array}{rlr}& \min & c_1\ast x+c_2\ast y\\ & s.t.& A_1\ast x+A_2\ast y\le b\end{array}$$
$x$ binary, $n_1$-dimensional vector
$y\ge 0$ continuous, $n_2$ dimensional vector

import xpress as xp

def demo_1():
    x1 = xp.var(vartype=xp.integer, name='x1', lb=-10, ub=10)
    x2 = xp.var(name='x2')
    p = xp.problem(x1, x2,       # variables of the problem 
                   x1**2+2*x2,   # objective function
                   x1+3*x2>=4,   # constraints
                   name='myexample')

    p.solve()
    print('solution: {0}={1}; {2}={3}'.format(x1.name, p.getSolution(x1), x2.name, p.getSolution(x2)))


import sys
def demo_2():
    c1 = [1, 6, 5, 7]       # n1 * 1
    c2 = [9, 3, 0, 2, 3]    # n2 * 1
    b = [-3, -4, 1, 4, 5]   # m * 1
    A1 = [[0, -2, 3, 2],
          [-5, 0, -3, 1],
          [1, 0, 4, -2],
          [0, -3, 4, -1],
          [-5, -4, 3, 0]]

    A2 = [[3, 4, 2, 0, -5],
          [0, 2, 3, -2, 1],
          [2, 0, 1, -3, -5],
          [-5, 3, -2, -3, 0],
          [-2, 3, -1, 2, -4]]

    m, n1, n2 = len(b), len(c1), len(c2)

    ## Define variables
    p = xp.problem()
    x = [xp.var(vartype=xp.binary) for _ in range(n1)]
    y = [xp.var() for _ in range(n2)] # positive real variable
    p.addVariable(x, y)

    ## Define constraints
    cons = [xp.Sum(A1[i][j]*x[j] for j in range(n1)) + xp.Sum(A2[i][j]*y[j] for j in range(n2))<=b[i] for i in range(m)]
    p.addConstraint(cons)

    ## Define objective
    p.setObjective(xp.Sum(c1[j]*x[j] for j in range(n1))+xp.Sum(c2[j]*y[j] for j in range(n2)),sense=xp.minimize)

    # solve & retrieve solution
    p.solve()
    if p.getProbStatus() != xp.mip_optimal:
        raise RuntimeError('Problem could not be solved to MIP optimality')
    
    y_opt = p.getSolution(y)
    print('The optimal value is: ', p.getObjVal())
    print('The first stage variables are: ', p.getSolution(x))
    print('The second stage variables are: ', y_opt)
    print('The objective of the second stage is:', sum(c2[j]*y_opt[j] for j in range(n2)))
def main():
    # demo_1()
    demo_2()


if __name__ == '__main__':
    main()

Reference

Benders blog
Gurobi列生成和Benders分解
Sparse portfolio selection with uncertain probability distribution