动态规划算法——背包问题(Dynamic Programming Algorithm – Knapsack Problem)

背包问题(Knapsack Problem)

(picture is from https://en.wikipedia.org/wiki/Knapsack_problem)
The knapsack problem is a problem in combinatorial optimisation: given n items, each with a weight and a value. Find the most valuable selection of items that will fit in the knapsack.

As shown in the picture,
knapsack capacity: 15 kg
weights: 1, 12, 2, 1, 4 kg
values: 2, 4, 2, 1, 10 dollars

If we choose the yellow, grey, blue and orange items, it will create the value of 2+2+1+10=15 dollars and has a weight of 8kg less than knapsack capacity. This is the most valuable selection.

How can we find the most valuable selection?

方法一 暴力破解(Brute force)

Here is an example. We find all combinations of items and then compute its sum of weights and values.

PS: “NF” means “Not feasible”. The sum of weights is greater than knapsack capacity.

From this figure, we can find {3, 4} is the best choice.

方法二 动态规划算法(Dynamic Programming Algorithm)
Dynamic programming is an algorithm design technique that is sometimes suitable when we want to solve a recurrence relation and the recursion involves overlapping instances.

What is the recurrence relation in the knapsack problem?

Let K(i, w) be the value of the best choice of items amongst first i items fitting in knapsack capacity w.

Amongst first i items we either pick item i or we don’t.

For a choice which includes item i. We need find the optimal choice of first i-1 items to fit in w-wi. Hence, the value is K(i-1, w-wi) + wi. Notes the prerequisite that wi ≤ w.

For a choice which excludes item i. This choice is the same with the choice that chooses optimal solution amongst i-1 items to fit in w. The value is K(i-1, w).

Now, we can express the recurrence relation:
K(i, w) = 0 if i=0 or w=0
otherwise
K(i, w) = max{K(i-1, w), K(i-1,w-wi)+wi} if wi ≤ w
K(i, w) = K(i-1, w)

伪代码(Pseudocode)

function Knapsack()
    for i ⟵ 0 to n do
        K[i][0] ⟵ 0
    for j ⟵ 1 to W do
        K[0][j] ⟵ 0
    for i ⟵ 1 to n do
        for j ⟵ 1 to W do
            if j < wi then
                K[i][j] ⟵ K[i-1][j]
            else
                K[i][j] ⟵ max(K[i-1][j], K[i-1][j-wi]+vi)
    return K[n][W]

时间复杂度(Time Complexity)
The time complexity is Θ(nW).
Cause the basic operation is comparison “j < wi”. It was executed n*W times.

示例(Example)

Java Code

public class DynamicProgramming{
    //Knapsack method
    public static int Knapsack(int[] w, int[] v, int n, int W){
        int[][] K = new int[n+1][W+1];
        int i, j;
        for(i=0; i< n+1; i++)
            K[i][0] = 0;
        for(j=1; j<W+1; j++)
            K[0][j] = 0;
        for(i=1; i<n+1; i++){
            for(j=1; j<W+1; j++){
                if(j < w[i])
                    K[i][j] = K[i-1][j];
                else
                    K[i][j] = Math.max(K[i-1][j], K[i-1][j-w[i]]+v[i]);
            }
        }
        return K[n][W];
    }

    //Test
    public static void main(String[] args){
        int[] value={0, 42, 12, 40, 25};
        int[] weight={0, 7, 3, 4, 5};
        int n = 4;
        int W = 8;
        int optimal = DynamicProgramming.Knapsack(weight, value, n, W);
        System.out.println("The maximum value is "+optimal);
    }
}

运行结果(Result)

The maximum value is 52

写在最后的话(PS)
Dynamic programming is an important algorithm design technique. Be serious.
Welcome questions always and forever.