Linear Programming

Linear programming is an optimization technique for a system of linear constraints and a linear objective function. An objective function defines the quantity to be optimized, and the goal of linear programming is to find the values of the variables that maximize or minimize the objective function.

Linear programming is useful for many problems that require an optimization of resources. It could be applied to manufacturing, to calculate how to assign labor and machinery to minimize cost of operations. It could be applied in high-level business operations, to decide which products to sell and in what quantity in order to maximize profit. It could also be applied in logistics, to decide how to apply resources to get a job done in the minimum amount of time.

Contents

When to use Linear Programming

Linear programming can be used to solve a problem when the goal of the problem is to maximize some value and there is a linear system of inequalities that defines the constraints on the problem.

A constraint is an inequality that defines how the values of the variables in a problem are limited. In order for linear programming techniques to work, all constraints should be linear inequalities.

Returning to the example in the introduction:

Note that there is a cost associated with producing each part. Each doodad costs $2 to make and each whirligig costs $4 to make. The factory only has $220 available to spend on these costs, so the production is limited by cost. Let \(x\) be the number of doodads produced, and let \(y\) be the number of whirligigs produced. Then this constraint can be written as an inequality:

\[2x+4y \le 220.\]

There is also the limitation on how much time the factory has to produce these parts. Each doodad requires 3 hours to make and each whirligig requires 2 hours to make. The factory only has 150 hours available this week, so production is also limited by time. This constraint can be written as an inequality:

\[3x+2y \le 150.\]

In addition to these constraints, there is also a couple of "common sense" constraints. It's not possible to produce less than 0 of any part, so these constraints are also written:

\[\begin x &\ge 0 \\ y &\ge 0. \end\]

These are called non-negative constraints. Altogether, the constraints form a system of inequalities:

\[\begin\begin 2x+4y &\le 220 \\ 3x+2y &\le 150 \\ x &\ge 0 \\ y &\ge 0. \end\end\]

Graphing these inequalities in the coordinate plane creates a polygon shape.

Graph the system of constraints

\[\begin\begin 2x+4y &\le 220 \\ 3x+2y &\le 150 \\ x &\ge 0 \\ y &\ge 0. \end\end\]

The shaded region above is the feasible region of this problem.

The region that is bound by the system of constraints is called the feasible region. It represents the possible values of the variables that satisfy all of the constraints. In order for linear programming techniques to work, it should be a convex polytope (in 2 dimensions, a convex polygon; in 3 dimensions, a convex polyhedron; and so on).

Finding the feasible region is only sufficient to give the possible solutions of a problem. The goal of linear programming is to find the best solution to a problem. This is done by maximizing or minimizing the objective function.

The objective function is a function that defines some quantity that should be minimized or maximized. The arguments of the objective function are the same variables that are used in the constraints. In order for linear programming techniques to work, the objective function should be linear.

Each doodad costs $2 to make and sells for $6. This gives a profit of $4 per doodad. Each whirligig costs $4 to make and sells for $7. This gives a profit of $3 per whirligig. The profit function can be defined as

\[p(x,y)=4x+3y.\]

This is the objective function of this problem, and the goal is to maximize it.

It seems like the strategy now would be to test ordered pairs in the feasible region until a maximum profit is found. However, a more efficient method is available.

Let \(P\) be the maximum profit in the feasible region:

\[P=4x+3y.\]

Solve for y:

\[y=-\fracx+\frac.\]

This maximum profit gives an equation of a line, and whatever point in the feasible region passes through this line is the optimal solution. The \(y\)-intercept of this line is \(\frac.\) Since \(P\) is maximized, this \(y\)-intercept should be maximized as well.

Graph several lines with the same slope of \(-\frac.\)

The line that maximizes the \(y\)-intercept is the one that passes through the vertex at \((20,45),\) the intersection of the first two constraints. All other higher lines do not pass through the feasible region. All other lower lines pass through more than one point in the feasible region, and do not maximize the \(y\)-intercept of the line.

Therefore, the factory should produce 20 doodads and 45 whirligigs. This will give a profit of \($215.\) \(_\square\)

Linear Programming in Two Variables

In the previous example, it was shown that the optimal solution was on a vertex of the feasible region. This is true for all linear programming problems.

Given a convex polygonal feasible region and a linear objective function, the solution that maximizes or minimizes the objective function will be located on one of the vertices of the feasible region.

Let the objective function be \(f(x,y)=ax+by.\) Let the maximum value of this function be \(P,\) and let the minimum value of this function be \(Q.\) There exist lines which intersect each of the optimal solutions, \((x,y)\):

\[\begin ax+by &= P &&\qquad (1) \\ ax+by &= Q &&\qquad (2) \\\\ \Rightarrow y &= -\fracx + \frac&&\qquad (1) \\ y &= -\fracx + \frac. &&\qquad (2) \end\]

Since \(P\) is the maximum value of the objective function, \((1)\) has the maximum \(y\)-intercept of a line with slope \(-\frac\) that passes through the feasible region. Likewise, \((2)\) has the minimum \(y\)-intercept of a line with slope \(-\frac\) that passes through the feasible region.

Suppose that \((1)\) or \((2)\) passes through a point that is not one of the vertices of the feasible region.

Hence, \((1)\) and \((2)\) must intersect a vertex of the feasible region. Each optimal solution is located at a vertex of the feasible region. \(_\square\)

This theorem gives a simple method for finding the optimal solution to a linear programming problem in two variables.

A farmer feeds his cows a feed mix to supplement their foraging. The farmer uses two types of feed for the mix. Corn feed contains 100 g protein per kg and 750 g starch per kg. Wheat feed contains 150 g protein per kg and 700 g starch per kg. Each cow should be fed at most 7 kg of feed per day. The farmer would like each cow to receive at least 650 g protein and 4000 g starch per day. If corn feed costs $0.40/kg and wheat costs $0.45/kg, then what is the optimal feed mix that minimizes cost? Round your answers to the nearest gram.

Let \(c\) be the kilograms of corn feed per cow per day, and let \(w\) be the kilograms of wheat feed per cow per day. The system of constraints can be written:

\[\begin \begin 0.1c+0.15w &\ge 0.65 \\ 0.75c+0.7w &\ge 4 \\ c+w &\le 7 \\ c &\ge 0 \\ w &\ge 0. \end \end\]

The objective function is

\[\text \ f(c,w)=0.40c+0.45w.\]

Graphing the system of constraints gives an idea of where the vertices of the feasible region are, and which lines intersect to form them:

Solve for each vertex of the feasible region by solving each pair of intersecting lines as a system of equations. For example, to solve for the vertex within the \(1^\text\) quadrant, solve the system of equations

\[\begin \begin 0.1c+0.15w &= 0.65 \\ 0.75c +0.7w &= 4. \end \end\]

Solving this system gives \(c \approx 3.411\) and \(w \approx 2.059.\) These values can be substituted into the objective function to obtain the cost of this mix:

\[f(3.411,2.059) = \$2.29.\]

Note that it is not necessary to solve for every vertex. Since the problem requires a minimum and the objective function line has a negative slope, the optimal solution must be on the underside of the feasible region. Solve for these vertices:

\[\begin \begin 0.75c +0.7w &= 4 \\ c &= 0 \end \end \implies c=0, w \approx 5.714 \implies f(0,5.714)=\$2.57 \]

\[\begin \begin 0.1c+0.15w &= 0.65 \\ w &= 0 \end \end \implies c=6.5, w=0 \implies f(6.5,0)=\$2.60. \]

The feed mix that minimizes cost contains 3411 g corn and 2059 g wheat. It costs $2.29 per cow. \(_\square\)

Submit your answer

A manufacturer has 750 meters of cotton and 1000 meters of polyester. Production of a sweatshirt requires 1 meter of cotton and 2 meters of polyester, while production of a shirt requires 1.5 meters of cotton and 1 meter of polyester. The sale prices of a sweatshirt and a shirt are 30 € and 24 €, respectively. What are the number of sweatshirts \((S)\) and the number of shirts \((C)\) that maximize total sales?

Jordan has $100 to buy some comic books. He really likes the Star Wars books which cost $12 each, but he could also buy the Marvels books which cost $5 each. If he has to buy at least 12 books, what is the maximum number of the Star Wars books that he can buy?

Submit your answer

An amateur bodybuilder is looking for supplement protein bars to build his muscle fast, and there are 2 available products: protein bar A and protein bar B.

Each protein bar A contains 15 g of protein and 30 g of carbohydrates and has total 200 calories. On the other hand, each protein bar B contains 30 g of protein and 20 g of carbohydrates and has total 240 calories.

According to his nutritional plan, this bodybuilder needs at least 20,000 calories from these supplements over the month, which must comprise of at least 1,800 g of protein and at least 2,200 g of carbohydrates.

If each protein bar A costs $3 and each protein bar B costs $4, what is the least possible amount of money (in $) he can spend to meet all his one-month requirements?

This kind of method would also work for linear optimization problems in more than two variables. However, these kinds of problems are more challenging to visualize with a coordinate graph, and there can be many more vertices to check for the optimal solution. The simplex algorithm was developed as an efficient method to solve these kinds of problems.

Simplex Algorithm

The simplex algorithm is a method to obtain the optimal solution of a linear system of constraints, given a linear objective function. It works by beginning at a basic vertex of the feasible region, and then iteratively moving to adjacent vertices, improving upon the solution each time until the optimal solution is found.

The simplex algorithm has many steps and rules, so it is helpful to understand the logic behind these steps and rules with a simple example before proceeding with the formal algorithm.

Given the system of constraints

\[\begin \begin 2x+3y &\le 90 \\ 3x+2y &\le 120 \\ x & \ge 0 \\ y & \ge 0, \end \end\]

maximize the objective function

\[f(x,y)=7x+5y.\]

The simplex algorithm begins by converting the constraints and objective functions into a system of equations. This is done by introducing new variables called slack variables. Slack variables represent the positive difference, or slack, between the left hand side of an inequality and the right hand side of that inequality.

The inequality

\[2x+3y \le 90\]

becomes

\[2x+3y+s_1=90.\]

Likewise, the inequality

\[3x+2y \le 120\]

becomes

\[3x+2y+s_2 = 120.\]

In addition to the slack variables, a variable \(z\) is introduced to represent the value of the objective function. This gives the equation

\[z-7x-5y=0.\]

These equations give the system of equations

\[\begin \begin z & - & 7x & - & 5y & & & & & = & 0 && (0) \\ & & 2x & + & 3y & + & s_1 & & & = & 90 && (1) \\ & & 3x & + & 2y & & & + & s_2 & = & 120. && (2) \end \end\]

In augmented matrix form, this is

\[\left[\begin 1 & -7 & -5 & 0 & 0 & 0 \\ 0 & 2 & 3 & 1 & 0 & 90 \\ 0 & 3 & 2 & 0 & 1 & 120 \end\right]. \qquad \begin (0) \\ (1) \\ (2) \end\]

It is implied that all variables in this system (including \(s_1,\) \(s_2,\) and \(z\)) are greater than or equal to 0. The variables \(s_1\) and \(s_2\) have zero coefficients in row \((0)\) and are called basic variables. The variables \(x\) and \(y\) have non-zero coefficients in row \((0)\) and are called non-basic variables. At any point in this process, the basic solution is given by setting the non-basic variables to 0. Currently, the basic solution is

\[x=0, \quad y=0, \quad s_1=90, \quad s_2=120, \quad z=0.\]

Consider what effect increasing the values of the non-basic variables would have on the value of \(z.\) Increasing either \(x\) or \(y\) would cause \(z\) to also increase, because \(x\) and \(y\) have negative coefficients in row \((0).\) Thus, this is not the optimal solution.

The iterations of the simplex algorithm involve exchanging basic variables and non-basic variables by using matrix row operations. At each step of the process, a non-basic variable in row \((0)\) is eliminated, leading another basic variable to take its place as a non-basic variable. This is called a pivot.

Suppose \(x\) were to be eliminated in row \((0).\) This can be done with either row \((1)\) or row \((2).\)

Case 1. Eliminating \(x\) in row \((0)\) with row \((1)\),

\[\left[\begin 1 & 0 & \frac & \frac & 0 & 315 \\ 0 & 2 & 3 & 1 & 0 & 90 \\ 0 & 3 & 2 & 0 & 1 & 120 \end\right]. \qquad \begin (0)\vphantom> \\ (1) \\ (2) \end\]

During this pivot, the variable \(x\) entered as a basic variable, and the variable \(s_1\) left to become a non-basic variable. Now eliminate \(x\) in row \((2)\):

\[\left[\begin 1 & 0 & \frac & \frac & 0 & 315 \\ 0 & 2 & 3 & 1 & 0 & 90 \\ 0 & 0 & -\frac & -\frac & 1 & -15 \end\right]. \qquad \begin (0)\vphantom> \\ (1) \\ (2)\vphantom> \end\]

This gives the basic solution

\[x=45, \quad y=0, \quad s_1=0, \quad s_2=-15, \quad z=315.\]

This solution is impossible, because it leads to one of the variables being negative.

Case 2. Eliminating \(x\) in row \((0)\) with row \((2)\),

\[\left[\begin 1 & 0 & -\frac & 0 & \frac & 280 \\ 0 & 2 & 3 & 1 & 0 & 90 \\ 0 & 3 & 2 & 0 & 1 & 120 \end\right]. \qquad \begin (0)\vphantom> \\ (1) \\ (2) \end\]

During this pivot, the variable \(x\) entered as a basic variable, and the variable \(s_2\) left to become a non-basic variable. Now eliminate \(x\) in row \((1)\):

\[\left[\begin 1 & 0 & -\frac & 0 & \frac & 280 \\ 0 & 0 & \frac & 1 & -\frac & 10 \\ 0 & 3 & 2 & 0 & 1 & 120 \end\right]. \qquad \begin (0)\vphantom> \\ (1)\vphantom> \\ (2) \end\]

This gives the basic solution

\[x=40, \quad y=0, \quad s_1=10, \quad s_2=0, \quad z=280.\]

This solution is possible, but it is not optimal, because there is a negative coefficient in row \((0).\) This implies that \(z\) can be increased further by increasing \(y.\) Another pivot will be needed to find the optimal solution.

It is a fair amount of work to perform a pivot, only to find that it gives an infeasible solution. Fortunately, one can anticipate which pivot will result in a feasible solution by observing the ratio of the element in the right part of the augmented matrix to the coefficient of the entering variable. Consider \(y\) as the entering variable, and calculate these ratios:

\[\left[\begin 1 & 0 & -\frac & 0 & \frac & 280 \\ 0 & 0 & \frac> & 1 & -\frac & 10> \\ 0 & 3 & 2> & 0 & 1 & 120> \end\right]. \qquad \begin (0)\vphantom> \\ (1)\vphantom> \\ (2) \end\]

For entering variable \(y,\) this ratio is \(10\div \frac=6\) for row \((1)\) and \(\frac=60\) for row \((2).\) Selecting the row that minimizes this ratio will ensure that the pivot results in a feasible solution. Thus, row \((1)\) should be selected as the pivot row.

Eliminating \(y\) in row \((0)\) with row \((1)\),

\[\left[\begin 1 & 0 & 0 & \frac & \frac & 282 \\ 0 & 0 & \frac & 1 & -\frac & 10 \\ 0 & 3 & 2 & 0 & 1 & 120 \end\right]. \qquad \begin (0)\vphantom> \\ (1)\vphantom> \\ (2) \end\]

Then eliminating \(y\) in row \((2)\),

\[\left[\begin 1 & 0 & 0 & \frac & \frac & 282 \\ 0 & 0 & \frac & 1 & -\frac & 10 \\ 0 & 3 & 0 & -\frac & \frac & 108 \end\right]. \qquad \begin (0)\vphantom> \\ (1)\vphantom> \\ (2)\vphantom> \end\]

This gives the basic solution

\[x=36, \quad y=6, \quad s_1=0, \quad s_2=0, \quad z=282.\]

This solution must be optimal, because any increase in the non-basic variables \(s_1\) and \(s_2\) will cause a decrease in \(z.\) Thus, the maximum value of the objective function is

\[f(36,6)=282.\ _\square\]

Submit your answer

A toy factory manufactures three kinds of toys: cars, motorcycles, and boats. One toy car makes $20 profit, one toy motorcycle makes $15 profit, and one toy boat makes $25 profit. There are three departments of labour: casting, which has 8 workers; assembly, which has 12 workers; quality control, which has 10 workers.

Every day, the labour is allocated as follows: a toy car requires 2 casting, 2 assembly, 2 quality control; a toy motorcycles requires 1 casting, 2 assembly, 1 quality control; a toy boat requires 2 casting, 3 assembly, 3 quality control.

What is the maximum profit per day (in dollars) the toy company can achieve?

The simplex algorithm for minimization problems works by converting the problem to a maximization problem. This concept that every maximization problem has a corresponding minimization problem is formalized with the von Neumann duality principle.

Given the system of constraints

\[\begin\begin 4x+3y+5z &\ge 65 \\ x+3y+2z &\ge 38 \\ 2x+3y+4z &\ge 52 \\ x,y,z &\ge 0, \end\end\]

minimize the function

\[f(x,y,z)=12x+3y+10z.\]

This problem could be put into the form shown in the maximization examples above, but an issue would occur with finding the first basic solution: setting the \(x,\) \(y,\) and \(z,\) variables to \(0\) would give an infeasible solution with the slack variables taking on negative values. The simplex algorithm needs to start with a feasible solution, so this would not work. The Big-M method gives a workaround to this problem, but there is a much simpler method for this problem.

A "dual" of this problem can be written by transposing the coefficients. Place the coefficients of the constraints into an augmented matrix. Place the coefficients of the objective function into the bottom row, with a 0 in the right part:

\[\left[\begin \color4 & \color3 & \color5 & \color65 \\ \color1 & \color3 & \color2 & \color38 \\ \color2 & \color3 & \color4 & \color52 \\ \hline \color12 & \color3 & \color10 & \color0 \end\right].\]

Transpose the elements of the matrix:

\[\left[\begin \color4 & \color1 & \color2 & \color12 \\ \color3 & \color3 & \color3 & \color3 \\ \color5 & \color2 & \color4 & \color10 \\ \hline \color65 & \color38 & \color52 & \color0 \end\right].\]

Note: It's tempting to divide out the 3 in the second row of this matrix, but this will break the symmetry that is required to return to the original problem.

This gives a new system of constraints and an objective function to be maximized: Given the system of constraints

\[\begin\begin 4u+v+2w &\le 12 \\ 3u+3v+3w &\le 3 \\ 2u+3v+4w &\le 52 \\ u,v,w &\ge 0, \end\end\]

maximize the function

\[g(u,v,w)=65u+38v+52w.\]

Now the simplex algorithm can be applied to find the optimal solution

\[\left[\begin 1 & -65 & -38 & -52 & 0 & 0 & 0 & 0 \\ 0 & 4 & 1 & 2 & 1 & 0 & 0 & 12 \\ 0 & 3 & 3 & 3 & 0 & 1 & 0 & 3 \\ 0 & 2 & 3 & 4 & 0 & 0 & 1 & 52 \end\right]. \qquad \begin (0) \\ (1) \\ (2) \\ (3) \end\]

Enter \(u\) with row \((2)\):

\[\left[\begin 1 & 0 & 27 & 13 & 0 & \frac & 0 & 65 \\ 0 & 0 & -3 & -2 & 1 & -4 & 0 & 8 \\ 0 & 1 & 1 & 1 & 0 & \frac & 0 & 1 \\ 0 & 0 & 1 & 2 & 0 & -2 & 1 & 50 \end\right]. \qquad \begin (0)\vphantom<\frac> \\ (1) \\ (2)\vphantom<\frac> \\ (3) \end\]

All coefficients in row \((0)\) are positive, so this is the optimal solution. The maximum value in the top right of the matrix, \(65,\) is the same as the minimum value for the original problem. However, the variables \(u,\) \(v,\) and \(w\) are not the same as the variables in the original problem. Fortunately, the values of the variables that minimize the original problem correspond to the coefficients of the slack variables in row \((0).\)

\[\left[\begin 1 & 0 & 27 & 13 & \color0 & \color\frac & \color0 & 65 \\ 0 & 0 & -3 & -2 & 1 & -4 & 0 & 8 \\ 0 & 1 & 1 & 1 & 0 & \frac & 0 & 1 \\ 0 & 0 & 1 & 2 & 0 & -2 & 1 & 50 \end\right]. \qquad \begin (0)\vphantom<\frac> \\ (1) \\ (2)\vphantom<\frac> \\ (3) \end\]

Thus, the values of the original problem that minimize the objective function are

\[x=0, \quad y=\frac, \quad z=0.\ _\square\]

Submit your answer

An amateur bodybuilder is looking for supplement protein bars to build his muscle fast, and there are 2 available products: protein bar A and protein bar B.

Each protein bar A contains 15 g of protein and 30 g of carbohydrates and has total 200 calories. On the other hand, each protein bar B contains 30 g of protein and 20 g of carbohydrates and has total 240 calories.

According to his nutritional plan, this bodybuilder needs at least 20,000 calories from these supplements over the month, which must comprise of at least 1,800 g of protein and at least 2,200 g of carbohydrates.

If each protein bar A costs $3 and each protein bar B costs $4, what is the least possible amount of money (in $) he can spend to meet all his one-month requirements?