[Formal method] PartB: Linear Programming (linear programming)

table of Contents

Generally speaking, a linear programming problem (hereinafter referred to as LP) consists of two parts: constraint C and objective function F. Our goal is to find the minimum or maximum value of F under the constraint of C. Both constraints and objective functions are in linear form. In this part, we will study some LP problems, also with the help of Z3.

For example, given constraints: we want to calculate:

To solve the LP problem with Z3, we will use the optimization module, specifically, the maximize() API will also be used. For more details about modules and APIs, you can refer to the official documentation .

For the above example, we can have the following Z3 code:

 x, y = Ints("x y")

    solver = Optimize()
    cons = [x < 2, (y - x) < 1]
    solver.add(cons)
    solver.maximize(x + 2*y)

    if solver.check() == sat:
        print(solver.model())

The output of Z3 is: [y = 1, x = 1]

Exercise 14: Read the code in the linear_programming.py Python file. Run the example and make sure you understand the code. Complete the exercise by filling in the missing code to add constraints

def lp_exercise():
    opt = Optimize()
    # exercise 14: Given the constraints:
    #  x - y > 2.1
    #  x + z < 5.5
    #  y - z < 1.1
    #
    # try to calculate the maximal value of
    #   x + y + z
    # by using LP in Z3
    # Your code here:
    # raise Todo("exercise 14: please fill in the missing code.")
    # add the constraints is same with LA
    x, y , z = Reals("x y z")
    cons = [(x - y) > 2.1, (x + z) < 5.5, (y - z) < 1.1]
    opt.add(cons)

    # use maximize() and minimize() method to add the target function
    opt.maximize(x + y + z)

    # check the Optimizer as we do with Solver
    if opt.check() == sat:
        print(opt.model())

if __name__ == '__main__':
    lp_examples()

    # should output: [z = 9/10, y = 3/2, x = 41/10]
    lp_exercise()

Output result:

Backpack problem

1. 0-1 knapsack problem

The knapsack problem is a typical optimization problem with a history of research for hundreds of years. The problem is: Given a set of items, each item has a weight and a value. Determine the number of items so that the total weight is less than the given limit and the total value is as large as possible. The knapsack problem has several sub-problems: 0-1 knapsack problem, complete knapsack problem, multiple knapsack problem, multi-dimensional knapsack problem and so on. Please refer to the backpack question for more information.

Let's discuss the 0-1 knapsack problem first, assuming that each item has only one quantity. Below, we use the following symbols:

We use the flag F to represent each item: 0 means item i is not selected, 1 means it is selected

In order to ensure that the total weight does not exceed the bag capacity C, we have: where N is the total number of items.

Our optimization goal is to maximize the value of the selected items:

Exercise 15: Read the code in the knapsack.py Python file, complete the zero_one_knapsack_lp() method code, and use 0-1 ILP to solve the 0-1 backpack problem. Don't forget to test your code.

#DP算法解决背包问题
def zero_one_knapsack_dp(weights, values, cap):
    def knapsack_dp_do(rest_cap, index):
        if rest_cap <= 0 or index <= 0:
            return 0

        if weights[index - 1] > rest_cap:
            return knapsack_dp_do(rest_cap, index - 1)

        value_1 = knapsack_dp_do(rest_cap, index - 1)
        value_2 = knapsack_dp_do(rest_cap - weights[index - 1], index - 1)

        if value_1 >= (value_2 + values[index - 1]):
            return value_1

        return value_2 + values[index-1]

    start = time.time()
    result = knapsack_dp_do(cap, len(weights))
    print(f"zero_one_knapsack_dp solve {len(weights)} items by time {(time.time() - start):.6f}s")
    return result

#0-1线性规划解决背包问题
def zero_one_knapsack_lp(weights, values, cap, verbose=False):
    # create a new solver, but
    solver = Optimize()

    # the decision variables
    flags = [Int(f"x_{i}") for i in range(len(weights))]
    # print(flags)

    # flags are 0-1
    for flag in flags:
        solver.add(Or(flag == 0, flag == 1))

    # @exercise 15: solve the 0-1 knapsack problem by using 0-1 ILP
    # construct the constraint
    #   \sum_i weights[i] * flags[i] <= cap
    # and the the target
    #   \sum_i values[i] * flags[i]
    # Your code here:
    # @begin
    # raise Todo("exercise 15: please fill in the missing code.")
    wf_exp = []
    i = 0
    for w in weights:
        wf_exp.append(w*flags[i])
        i = i + 1
    solver.add(sum(wf_exp) <= cap) # 约束条件：\sum_i weights[i] * flags[i] <= cap
    j = 0
    vf_exp = []
    for v in values:
        vf_exp.append(v*flags[j])
        j = j + 1
    solver.maximize(sum(vf_exp))
    # @end

    start = time.time()
    result = solver.check()
    print(f"zero_one_knapsack_lp solve {len(weights)} items by time {(time.time() - start):.6f}s")

    if result == sat:
        model = solver.model()
        verbose = True
        # print the chosen items
        if verbose:
            print("\n".join([f"Index: {index}, Weight: {weights[index]}, Value: {values[index]}"
                             for index, flag in enumerate(flags) if model[flag] == 1]))

        return True, sum([values[index] for index, flag in enumerate(flags) if model[flag] == 1])

    return False, result

# 测试用例
if __name__ == '__main__':
    # a small test case
    W = [4, 6, 2, 2, 5, 1]
    V = [8, 10, 6, 3, 7, 2]
    C = 12
    print(zero_one_knapsack_dp(W, V, C))
    print(zero_one_knapsack_lp(W, V, C))

    # another test case
    W = [23, 26, 20, 18, 32, 27, 29, 26, 30, 27]
    V = [505, 352, 458, 220, 354, 414, 498, 545, 473, 543]
    C = 67
    print(zero_one_knapsack_dp(W, V, C))
    print(zero_one_knapsack_lp(W, V, C))

Output result:

2. Complete knapsack problem

Another sub-problem of the knapsack problem is the complete knapsack problem. It assumes that the number of various items is unlimited, and you can choose any item.

So we need to declare a variable, set a quantity A for each selected item F[i]: 0 ≤ i <N, 0≤ A

So the constraint condition of the total weight is:

Where N is the total number of item types. The maximum value of the selected item is:

Exercise 16: Read the code in the knapsack_py Python file and fill in the code in the complete_knapsack_lp() method to solve the complete knapsack_lp problem. Don't forget to test your code.

def complete_knapsack_lp(weights, values, cap, verbose=False):
    solver = Optimize()

    # @exercise 16: solve the complete knapsack problem by using LP
    # note that flags[i] will be a integer and flags[i] >= 0
    # construct the constraint
    #   \sum_i weights[i] * flags[i] <= cap
    # and the the target
    #   \sum_i values[i] * flags[i]
    # Your code here:
    # @begin
    # raise Todo("exercise 16: please fill in the missing code.")
    flags = [Int(f"x_{i}") for i in range(len(weights))]

    for flag in flags:
        solver.add(flag>=0)

    wf_exp = []
    i = 0
    for w in weights:
        wf_exp.append(w * flags[i])
        i = i + 1
    solver.add(sum(wf_exp) <= cap)  # 约束条件：\sum_i weights[i] * flags[i] <= cap
    j = 0
    vf_exp = []
    for v in values:
        vf_exp.append(v * flags[j])
        j = j + 1
    solver.maximize(sum(vf_exp))
    start = time.time()
    result = solver.check()
    print(f"zero_one_knapsack_lp solve {len(weights)} items by time {(time.time() - start):.6f}s")

    if result == sat:
        model = solver.model()
        verbose = True
        # print the chosen items
        if verbose:
            print("\n".join([f"Index: {index}, Weight: {weights[index]}, Value: {values[index]},Amount:{model[flag].as_long()}"
                             for index, flag in enumerate(flags) if model[flag].as_long() > 0]))

        return True, sum([values[index]*model[flag].as_long() for index, flag in enumerate(flags) if model[flag].as_long() > 0])

    return False, result
    # @end

test:

if __name__ == '__main__':
   
    W = [23, 26, 20, 18, 32, 27, 29, 26, 30, 27]
    V = [505, 352, 458, 220, 354, 414, 498, 545, 473, 543]
    C = 133
    print("Maximal Value: ", complete_knapsack_lp(W, V, C, verbose=True))

Output result:

The main difficulty: find the function as_long() that converts z3 integers into python large numbers

After judging that model[flag] is greater than 0, it has been stuck for a long time, writing model[flag]> 0 keeps reporting errors. It turns out that model[flag] is an integer of z3. When converted into py semantics, you need to write: model[flag].as_long()> 0

z3.py库中对as_long()函数的解释    
def as_long(self):
        """Return a Z3 integer numeral as a Python long (bignum) numeral.

        >>> v = IntVal(1)
        >>> v + 1
        1 + 1
        >>> v.as_long() + 1
        2
        """
        if z3_debug():
            _z3_assert(self.is_int(), "Integer value expected")
        return int(self.as_string())

The knapsack problem can also be solved by dynamic programming.

Exercise 17: Read the code in the knapsack_py Python file. The zero_one_knapsack_dp() method has provided a solution to the 0-1 knapsack problem based on dynamic programming (DP). Using the data generated by the get_large_test() method, try to compare the efficiency of the DP algorithm and the LP algorithm. What are your observations? What conclusions can you draw from the results?

# 读取较大数值文件的函数
def get_large_test():
    # this test data is fetched from:
    # https://people.sc.fsu.edu/~jburkardt/datasets/knapsack_01/knapsack_01.html
    # the expect maximum value should be: 13549094
    def read_numbers_from_file(file_path):
        with Path(file_path).open(mode="r") as fp:
            content = fp.readlines()
            return [int(x.strip()) for x in content]

    file_folder = Path(__file__).parent.resolve()
    return (read_numbers_from_file(file_folder / "p08_w.txt"),
            read_numbers_from_file(file_folder / "p08_p.txt"))

Test case:

if __name__ == '__main__':
    
    # a large test case
    W, V = get_large_test()
    C = 6404180

    # @exercise 17: compare the efficiency of the DP and the
    # LP algorithm, by running your program on a larger data.
    # what's your observation? What conclusion you can draw from these data?
    # Your code here:
    # @begin
    # raise Todo("exercise 17: please fill in the missing code.")
    print(zero_one_knapsack_dp(W, V, C))
    print(zero_one_knapsack_lp(W, V, C))
    # @end

Output result:

(Ps: The LP algorithm has been running for too long without waiting for it to run out of results, and then map the results)

Conclusion: Dynamic programming is much more efficient than LP algorithm in dealing with large-number knapsack problems

#中科大软院-hbj formalized course notes-Welcome to leave a message and exchange private messages

#随手点赞, I will be happier~~^_^