Math 407 Summer 2005 Class Web Page

Math 407 Summer 2005 Class Web Page LINEAR OPTIMIZATION
a: Instructor

Name	Brad Bell (http://www.seanet.com/~bradbell/)
Office Location	Room 471 of Henderson Hall (http://www.washington.edu/home/maps/northwest.html?HND)
Office Phone	206-543-6855
Paging	206-543-1300
E-mail	brad at apl dot washington dot edu

b: Other Versions of this Site
If the text

π

appears as \pi instead of the corresponding Greek letter, you are viewing home.htm . If your browser supports XHTML + MathML, you will get a better view of these and other pages in this site by starting with home.xml . Printable versions of this web site can be found in the files _printable.htm and _printable.xml .

c: Contents

Table of Contents: 1
The Simple Method: 2
Linear Programming Duality Theory: 3
Zero Sum Matrix Games: 4
Part II: Selected Applications: 5
Neos Interface to the Clp Linear Program Solver: 6
Math 407 Summer 05 Tests: 7
Bibliography: 8
Alphabetic Listing of Cross Reference Tags: 9
Keyword Index: 10

d: Class Time and Location
Class meets from 9:40 to 11:20, in Room 108 of Fishery Science Bldg (FSH) (http://www.washington.edu/home/maps/southwest.html?FSH) on the following days:

Tuesday	06/21	06/28	07/05	07/12	07/19	07/26	08/02	08/09	08/16
Thursday	06/23	06/30	07/07	07/14	07/21	07/28	08/04	08/11	08/18

e: Text Book

Author	Chvatal, Vasek
Title	Linear programming
Pub info	New York : W.H. Freeman, c1983

f: Home Work
There is no graded home work for this class. The instructor will suggest that the students read certain sections of the text book and work some of the corresponding problems.

g: Computer Solution
You can determine the solution of many of the problems in the text using the Neos interface to Clp: 6 (Clp is an open source linear program solver).

h: Testing

h.a: Quizzes
There will be 7 quizzes each 30 minutes long at the end of class on

06/23

06/30

07/07

07/19

07/26

08/02

08/09

Only the 6 highest quizzes will count for a total possible 6*40 = 240 points (total time working on quizzes = 6*30 = 180 minutes).

h.b: Final
The final will be during the entire last day of class and will be worth 160 points (total time working on final = 100 minutes).

h.c: Rules
You may use the text book, a printed version of the class web pages, a calculator or computer, and any notes you have written, during any of the quizzes and final exam, but that is all.

Makeups for tests must be arranged with the instructor ahead of time. If a student misses a test with out arranging a makeup ahead of time, a score of zero will be assigned for that test. If this only happens for one quiz, it becomes the quiz that does not count. If this happens for the final, and the student had an emergency that, in the instructors opinion, prevented the student from taking the final, she or he will receive an incomplete (http://www.washington.edu/students/reg/incomplete.html) (instead of a zero for the final).

h.d: Grading
Tests will be returned during class with in one week of the time they are taken. The graded final exam will be available at the Math department office. If a student thinks that the grading of a particular question is not correct, she or he may submit, in writing, an explanation for why they think the grading is not correct and what they think would be correct. The class grade will be determined by the total number of points out of a possible 480 (7 * 40 possible for quizzes and 200 possible for final).

1: Table of Contents

Math 407 Summer 2005 Class Web Page: home: 
    Table of Contents: _contents: 1
    The Simple Method: SimplexMethod: 2
        Dictionary Method Solution of Equation 2.1 in Chvatal: Dictionary2.1: 2.1
        Tableau Method Solution of Equation 2.1 in Chvatal: Tableau2.1: 2.2
        Neos Input and Output File for Equation 2.1 in Chvatal: Equation2.1: 2.3
        Preform a Pivot Operation: Pivot: 2.4
            Using Pivot Function to solve Equation 2.1: Pivot2_1: 2.4.1
        Multiple Solutions: Multiple: 2.5
            Neos Input and Output File for Problem 2.2 in Chvatal: Problem2.2: 2.5.1
        Two Phase Simplex Method: TwoPhase: 2.6
            Neos Input and Output File for Problem 3.9a in Chvatal: Problem3.9a: 2.6.1
        Degeneracy: Degeneracy: 2.7
        The Cycling of Simplex Method: Basis: 2.8
        Bland's Pivot Rule: Bland: 2.9
            Example Use of Bland's Method: BlandExample: 2.9.1
    Linear Programming Duality Theory: Duality: 3
        A Linear Programming Duality Example: DualExample: 3.1
        Duality With Inequality Constraints: DualIneq: 3.2
        The Duality Theorem: DualTheorem: 3.3
        The Complementary Slackness Condition: CompSlack: 3.4
        An Example Computing the Dual Variables: DualComp: 3.5
        Relation Between Dual and Perturbed Upper Bound: PerturbUpper: 3.6
        Duality with Equality and Inequality Constraints: DualEqIneq: 3.7
    Zero Sum Matrix Games: MatrixGame: 4
        Problem 15.1 of the Text: Problem15.1: 4.1
        Rock Paper Scissor as a Zero Sum Matrix Game: RockPaperScissor: 4.2
    Part II: Selected Applications: Applications: 5
        Product Manufacturing: Manufacture: 5.1
        Toll Booth Scheduling: Schedule: 5.2
        Electronics Company: Electronic: 5.3
        A Forestry Example: Forestry: 5.4
    Neos Interface to the Clp Linear Program Solver: NeosClp: 6
        The MPS Input Format: MpsInputFile: 6.1
        Some of The Clp Executable Commands: ClpCommand: 6.2
    Math 407 Summer 05 Tests: Test: 7
        Math 407 Summer 05 Quiz 06-23: Quiz0623: 7.1
        Math 407 Summer 05 Quiz 06-30: Quiz0630: 7.2
        Math 407 Summer 05 Quiz 07-05: Quiz0705: 7.3
        Math 407 Summer 05 Quiz 07-19: Quiz0719: 7.4
        Math 407 Summer 05 Quiz 07-26: Quiz0726: 7.5
        Math 407 Summer 05 Quiz 08-02: Quiz0802: 7.6
        Math 407 Summer 05 Quiz 08-09: Quiz0809: 7.7
        Math 407 Summer 05 Final Exam: Final: 7.8
    Bibliography: Bib: 8
    Alphabetic Listing of Cross Reference Tags: _reference: 9
    Keyword Index: _index: 10

2: The Simple Method
2.a: Contents

Dictionary Method Solution of Equation 2.1 in Chvatal: 2.1
Tableau Method Solution of Equation 2.1 in Chvatal: 2.2
Neos Input and Output File for Equation 2.1 in Chvatal: 2.3
Preform a Pivot Operation: 2.4
Multiple Solutions: 2.5
Two Phase Simplex Method: 2.6
Degeneracy: 2.7
The Cycling of Simplex Method: 2.8
Bland's Pivot Rule: 2.9

2.1: Dictionary Method Solution of Equation 2.1 in Chvatal
2.1.a: Equation 2.1 of Chvatal




maximize    5*x1 + 4*x2 + 3*x3


subject to  2*x1 + 3*x2 +   x3 <= 5


            4*x1 +   x2 + 2*x3 <= 11


            3*x1 + 4*x2 + 2*x3 <= 8


              x1 ,   x2 ,   x3 >= 0

2.1.b: Add Slack And Objective Variables




maximize    z  =      5*x1 + 4*x2 + 3*x3


subject to  s1 = 5  - 2*x1 - 3*x2 -   x3 


            s2 = 11 - 4*x1 -   x2 - 2*x3


            s3 = 8  - 3*x1 - 4*x2 - 2*x3 


            x1, x2, x3, s1, s2, s3 >= 0

2.1.c: Basic Feasible Solution
The basic feasible solution sets the variables on the right side of the equations to zero and then solves for the ones on the left side; i.e., the basic feasible solution corresponding to the case above is




     x1=0, x2=0, x3=0, s1=5, s2=11, s3=8, z=0

2.1.d: Choose First Pivot
We notice that z increases as we increase x1 and keep x2 = x3 = 0 (because the x1 coefficient in the equation for z is positive). If the resulting point is feasible,




     0 <= s1 = 5  - 2*x1; i.e.,      x1 <=  5/2


     0 <= s2 = 11 - 4*x1; i.e.,      x1 <= 11/4


     0 <= s3 = 8  - 3*x1; i.e.,      x1 <=  8/3

We notice that if we choose x1 so that s1 is zero, all the constraints will be satisfied. This corresponds exchanging the roles of x1 and s1 in the set of equations.

2.1.e: First Variables Exchange
We solve for the variable x1 in the row corresponding to s1; i.e,




     x1 = 2.5 - .5*s1 - 1.5*x2 - .5*x3

We replace the row corresponding to s1 with this equation and we use this equation to replace all other occurrences of x1 in the problem; i.e.,




maximize    z  =       5*(2.5 - .5*s1 - 1.5*x2 - .5*x3) +  4*x2 +  3*x3


subject to  x1 =          2.5 - .5*s1 - 1.5*x2 - .5*x3 


            s2 = 11  - 4*(2.5 - .5*s1 - 1.5*x2 - .5*x3) -    x2 -  2*x3


            s3 = 8   - 3*(2.5 - .5*s1 - 1.5*x2 - .5*x3) -  4*x2 -  2*x3


            x1, x2, x3, s1, s2, s3 >= 0

Regrouping terms we have the equivalent problem




maximize    z  = 12.5 - 2.5*s1 - 3.5*x2 + .5*x3


subject to  x1 = 2.5  -  .5*s1 - 1.5*x2 - .5*x3 


            s2 = 1    +   2*s1 +   5*x2 


            s3 = .5   + 1.5*s1 +  .5*x2 - .5*x3 


            x1, x2, x3, s1, s2, s3 >= 0

The basic feasible solution corresponding to the representation above is




     x1=2.5, x2=0, x3=0, s1=0, s2=1, s3=.5, z=12.5

We note that the vale of z has increased from the previous problem representation.

2.1.f: Choose Second Pivot
We notice that z increases as we increase x3 and keep s1 = x2 = 0 (because the x3 coefficient in the equation for z is positive). If the resulting point is feasible,




     0 <= x1 = 2.5 - .5*x3; i.e.,      x3 <= 5


     0 <= s2 = 1;           i.e.,       0 <= 1


     0 <= s3 = .5 -  .5*x3; i.e.,      x3 <= 1

We notice that if we choose x3 so that s3 is zero, all the constraints will be satisfied. This corresponds exchanging the roles of x3 and s3

2.1.g: Second Variables Exchange
We solve for the variable x3 in the row corresponding to s3; i.e,




maximize    z  = 12.5 - 2.5*s1 - 3.5*x2 + .5*(1 + 3*s1 + x2 - 2*s3)


subject to  x1 = 2.5  -  .5*s1 - 1.5*x2 - .5*(1 + 3*s1 + x2 - 2*s3) 


            s2 = 1    +   2*s1 +   5*x2 


            x3 =                              1 + 3*s1 + x2 - 2*s3 





            x1, x2, x3, s1, s2, s3 >= 0

Regrouping terms we have the equivalent problem




maximize    z  = 13 -   s1 - 3*x2 -   s3


subject to  x1 = 2  - 2*s1 - 2*x2 +   s3 


            s2 = 1  + 2*s1 + 5*x2 


            x3 = 1  + 3*s1 +   x2 - 2*s3 





            x1, x2, x3, s1, s2, s3 >= 0

The basic feasible solution corresponding to the representation above is




     x1=2, x2=0, x3=1, s1=0, s2=1, s3=0, z=13

We note that the vale of z has increased from the previous problem representation. This is the optimal solution (because there is no feasible direction in which the objective function increases).

2.2: Tableau Method Solution of Equation 2.1 in Chvatal
2.2.a: Equation 2.1 of Chvatal




maximize    5*x1 + 4*x2 + 3*x3


subject to  2*x1 + 3*x2 +   x3 <= 5


            4*x1 +   x2 + 2*x3 <= 11


            3*x1 + 4*x2 + 2*x3 <= 8


              x1 ,   x2 ,   x3 >= 0

2.2.b: Add Slack And Objective Variables




maximize    5*x1 + 4*x2 + 3*x3                - z  = 0


subject to  2*x1 + 3*x2 +   x3 + s1                = 5


            4*x1 +   x2 + 2*x3      + s2           = 11


            3*x1 + 4*x2 + 2*x3           + s3      = 8


              x1 ,   x2 ,   x3 >= 0

2.2.c: Tableau Format
The tableau representation of the equations above is

\begin{matrix} x_{1} & x_{2} & x_{3} & s_{1} & s_{2} & s_{3} & z & b \\ 2 & 3 & 1 & 1 & 0 & 0 & 0 & 5 \\ 4 & 1 & 2 & 0 & 1 & 0 & 0 & 11 \\ 3 & 4 & 2 & 0 & 0 & 1 & 0 & 8 \\ 5 & 4 & 3 & 0 & 0 & 0 & -1 & 0 \end{matrix}

(Note that we have moved the equation for

z

to the last row.) The corresponding basic feasible solution is

x_{1} = x_{2} = x_{3} = 0

s_{1} = 5

s_{2} = 11

s_{3} = 8

z = 0

.

2.2.d: Choose First Pivot
We search the row corresponding to the objective

z

(last row) for a positive entry and find the value 5 in the

x_{1}

column. Thus was can increase the objective by increasing the value of

x_{1}

and holding

x_{2}

x_{3}

at zero. We now solve for the value of

x_{1}

that results in

\begin{matrix} s_{1} = 0 & \Leftrightarrow & x_{1} = 5 / 2 & = 2.5 \\ s_{2} = 0 & \Leftrightarrow & x_{1} = 11 / 4 & = 2.75 \\ s_{3} = 0 & \Leftrightarrow & x_{1} = 8 / 3 & = 2.66 \dots \end{matrix}

Thus we can pivot on the pair

(x_{1}, s_{1})

and obtain a new basic feasible solution with an improved value for the objective.

2.2.e: Divide by First Pivot
Divide the row corresponding to pivot element by the value of the pivot element:

\begin{matrix} x_{1} & x_{2} & x_{3} & s_{1} & s_{2} & s_{3} & z & b \\ 1 & 1.5 & .5 & .5 & 0 & 0 & 0 & 2.5 \\ 4 & 1 & 2 & 0 & 1 & 0 & 0 & 11 \\ 3 & 4 & 2 & 0 & 0 & 1 & 0 & 8 \\ 5 & 4 & 3 & 0 & 0 & 0 & -1 & 0 \end{matrix}

2.2.f: First Row Reduction
For each of the rows, except the one corresponding to the pivot element, subtract the value in the column corresponding to

x_{1}

times the row corresponding to the pivot element:

\begin{matrix} x_{1} & x_{2} & x_{3} & s_{1} & s_{2} & s_{3} & z & b \\ 1 & 1.5 & .5 & .5 & 0 & 0 & 0 & 2.5 \\ 0 & -5 & 0 & -2 & 1 & 0 & 0 & 1 \\ 0 & -.5 & .5 & -1.5 & 0 & 1 & 0 & .5 \\ 0 & -3.5 & .5 & -2.5 & 0 & 0 & -1 & -12.5 \end{matrix}

The corresponding basic feasible solution is

s_{1} = x_{2} = x_{3} = 0

x_{1} = 2.5

s_{2} = 1

s_{3} = .5

z = 12.5

.

2.2.g: Choose Second Pivot
We search the row corresponding to the objective

z

(last row) for a positive entry and find .5 in the

x_{3}

column. Thus we can increase the objective by increasing

x_{3}

and holding

x_{2}

s_{1}

at zero. We now solve for the value of

x_{3}

that results in

\begin{matrix} x_{1} = 0 & \Leftrightarrow & x_{3} = 5 \\ s_{2} = 1 & for all & x_{3} \\ s_{3} = 0 & \Leftrightarrow & x_{3} = 1 \end{matrix}

Thus we can pivot on the pair

(x_{3}, s_{3})

and obtain a new basic feasible solution with an improved value for the objective.

2.2.h: Divide by Second Pivot
Divide the row corresponding to pivot element by the value of the pivot element:

\begin{matrix} x_{1} & x_{2} & x_{3} & s_{1} & s_{2} & s_{3} & z & b \\ 1 & 1.5 & .5 & .5 & 0 & 0 & 0 & 2.5 \\ 0 & -5 & 0 & -2 & 1 & 0 & 0 & 1 \\ 0 & -1 & 1 & -3 & 0 & 2 & 0 & 1 \\ 0 & -3.5 & .5 & -2.5 & 0 & 0 & -1 & -12.5 \end{matrix}

2.2.i: Second Row Reduction

\begin{matrix} x_{1} & x_{2} & x_{3} & s_{1} & s_{2} & s_{3} & z & b \\ 1 & 2 & 0 & 2 & 0 & -1 & 0 & 2 \\ 0 & -5 & 0 & -2 & 1 & 0 & 0 & 1 \\ 0 & -1 & 1 & -3 & 0 & 2 & 0 & 1 \\ 0 & -3 & 0 & -1 & 0 & -1 & -1 & -13 \end{matrix}

s_{1} = x_{2} = s_{3} = 0

x_{1} = 2

s_{2} = 1

x_{3} = 1

z = 13

. This is an optimal solution because all of the coefficients in the row corresponding to the objective

z

are less than zero.

2.3: Neos Input and Output File for Equation 2.1 in Chvatal
2.3.a: Input File




<document>





<category>lp</category>


<solver>Clp</solver>


<inputMethod>MPS</inputMethod>





<comments><![CDATA[


Equation 2.1 in Vasek Chvatal's book: Linear Programming


maximize    5*x1 + 4*x2 + 3*x3


subject to  2*x1 + 3*x2 +   x3 <= 5


            4*x1 +   x2 + 2*x3 <= 11


            3*x1 + 4*x2 + 2*x3 <= 8


              x1 ,   x2 ,   x3 >= 0


The solution is x1 = 2, x2 = 0, x3 = 1


The residuals are 0=s1=5-r1, 1=s2=11-r2, 0=s3=8-r3


]]></comments>





<MPS><![CDATA[*


*Op Name0---  Name1---  Value1------   Name2---  Value2------


*23 56789012  56789012  567890123456   01234567  012345678901


NAME          CE-2.1


ROWS


 N  z


 L  r1


 L  r2


 L  r3


COLUMNS


    x1        z         5              r1        2


    x1        r2        4              r3        3


*


    x2        z         4              r1        3


    x2        r2        1              r3        4


*


    x3        z         3              r1        1


    x3        r2        2              r3        2


RHS


    b         r1        5


    b         r2        11


    b         r3        8


ENDATA


*]]></MPS>





<param><![CDATA[


maximize


primalSimplex


solution -


]]></param>





</document>

2.3.b: Output File




%%%%%%%%%%%%%%%%%%%% CLP Results %%%%%%%%%%%%%%%%%%%%





Load Avg: ( 2.0 , 2.0 , 2.0 )


Coin LP version 1.02.02, build Aug  3 2005


At line 4 NAME          CE-2.1


At line 5 ROWS


At line 10 COLUMNS


At line 19 RHS


At line 23 ENDATA


Problem CE-2.1 has 3 rows, 3 columns and 9 elements


Model was imported from ./clp.mps in 0 seconds


Switching to line mode


Clp:Clp:Clp:Presolve 3 (0) rows, 3 (0) columns and 9 (0) elements


0  Obj -0 Dual inf 7.47056 (3)


4  Obj 13


Optimal - objective value 13


Optimal objective 13 - 4 iterations time 0.002


Clp:


      0 r1                    5                      1


      1 r2                   10                     -0


      2 r3                    8                      1


      0 x1                    2         -2.8223853e-16


      1 x2                    0                     -3


      2 x3                    1           5.647295e-16


Clp:





%%%%%%%%%%%%%%%%%%%% CLP Results %%%%%%%%%%%%%%%%%%%%

2.3.c: Solution Correspondence
We note that




     s1 = 5  - r1 = 0


     s2 = 11 - r2 = 1


     s3 = 8  - r3 = 0


     x1 = 2


     x2 = 0


     x3 = 1

2.4: Preform a Pivot Operation

Syntax B = Pivot(A, r, c)

2.4.a: Description
Performs a pivot operation on the Tableau A about row r and column c. Let

m

(

n

) be the number of rows (columns) in the matrix A. The output matrix B and the same number of rows and columns and for

i = 1, \dots, m

i \neq r

j = 1, \dots, n

\begin{matrix} B_{r, j} & = & A_{r, j} \frac{1}{A_{r, c}} \\ B_{i, j} & = & A_{i, j} - A_{r, j} \frac{A_{i, c}}{A_{r, c}} \end{matrix}

It follows that

B_{i, c} = {\begin{matrix} 1 & if i = r \\ 0 & otherwise \end{matrix}

2.4.b: Example
The section Pivot2_1: 2.4.1 contains an example use of the Pivot function.

2.4.c: Matlab Source Code



function B = Pivot(A, r, c)




[m, n]     = size(A);


B          = zeros(m, n);


B(r, :)    = A(r, :) / A(r, c);


for i = 1 : m


     if i ~= r


          B(i, :) = A(i, :) - A(i, c) * B(r, :);


     end


end

2.4.1: Using Pivot Function to solve Equation 2.1
2.4.1.a: Description
In this section we apply the Matlab function Pivot: 2.4 to the problem in Equation 2.1.

2.4.1.b: Initial Tableau
The initial tableau format: 2.2.c for this problem is specified by entering




     A = [ 2 3 1 1 0 0 0  5


           4 1 2 0 1 0 0  11


           3 4 2 0 0 1 0  8


           5 4 3 0 0 0 -1 0 ]

The result of this operation is:




     A = {


     [ 2 , 3 , 1 , 1 , 0 , 0 , 0 , 5 ]


     [ 4 , 1 , 2 , 0 , 1 , 0 , 0 , 11 ]


     [ 3 , 4 , 2 , 0 , 0 , 1 , 0 , 8 ]


     [ 5 , 4 , 3 , 0 , 0 , 0 , -1 , 0 ]


     }

2.4.1.c: First Pivot
As in Tableau2.1: 2.2.d we choose row index 1 and column index 1 in the tableau for our first pivot:




     r = 1;


     c = 1;


     B = Pivot(A, r, c)

The result of this operation is:




     B = {


     [ 1 , 1.5 , 0.5 , 0.5 , 0 , 0 , 0 , 2.5 ]


     [ 0 , -5 , 0 , -2 , 1 , 0 , 0 , 1 ]


     [ 0 , -0.5 , 0.5 , -1.5 , 0 , 1 , 0 , 0.5 ]


     [ 0 , -3.5 , 0.5 , -2.5 , 0 , 0 , -1 , -12.5 ]


     }

2.4.1.d: Second Pivot
As in Tableau2.1: 2.2.g we choose row index 3 and column index 3 in the tableau for our second pivot:




     r = 3;


     c = 3;


     C = Pivot(B, r, c);

The result of this operation is:




     C = {


     [ 1 , 2 , 0 , 2 , 0 , -1 , 0 , 2 ]


     [ 0 , -5 , 0 , -2 , 1 , 0 , 0 , 1 ]


     [ 0 , -1 , 1 , -3 , 0 , 2 , 0 , 1 ]


     [ 0 , -3 , 0 , -1 , 0 , -1 , -1 , -13 ]


     }

2.5: Multiple Solutions
2.5.a: Problem 2.2
The tableau corresponding to Problem 2.2 of the text is:

 


     x1    x2    x3    x4    s1    s2      z     b


      1     2     3     1     1     0      0     5


      1     1     2     3     0     1      0     3


      2     3     5     4     0     0     -1     0

2.5.b: First Pivot
Suppose that we choose the pair (x3, s2) for our first pivot operation. The result of the pivot will be:

 


     x1    x2    x3    x4    s1    s2      z     b


    -.5    .5     0  -3.5     1  -1.5      0    .5


     .5    .5     1   1.5     0    .5      0   1.5


    -.5    .5     0  -3.5     0  -2.5     -1  -7.5

(we used Pivot: 2.4 to do the arithmetic in this operation.)

2.5.c: Second Pivot
Suppose that we choose the pair (x2, s1) for our second pivot operation. The result of the pivot will be:




     x1    x2    x3    x4    s1    s2      z     b


     -1     1     0    -7     2    -3      0     1


      1     0     1     5    -1     2      0     1


      0     0     0     0    -1    -1     -1    -8

(we used Pivot: 2.4 to do the arithmetic in this operation.)

2.5.d: Basic Feasible Solution
The corresponding basic feasible solution is




     x1 = 0, x2 = 1, x3 = 1, x4 = 0, s1 = 0, s2 = 0, z = 8

It follows that this is an optimal solution because all the variable coefficients in the last row are less than or equal zero (not counting the b coefficient which is also <= 0).

2.5.e: Other Optimal Solutions
The tableau above represents the problem:




maximize      z = 8             -   s1 -   s2 


subject to   x2 = 1 + x1 + 7*x4 - 2*s1 + 3*s2


             x3 = 1 - x1 - 5*x4 +   s1 - 2*s2

Thus, for any choice of a >= 0 and b >= 0, the solution




s1 = 0, s2 = 0, z = 8, x1 = a, x4 = b,  x2 = 1 + a + 7*b, x3 = 1 - a - 5*b

is also an optimal for this problem.

2.5.f: Neos Solution
The corresponding Neos input file is problem 2.2 input file: 2.5.1.a and the resulting output is problem 2.2 output file: 2.5.1.b .

2.5.1: Neos Input and Output File for Problem 2.2 in Chvatal
2.5.1.a: Input File




<document>





<category>lp</category>


<solver>Clp</solver>


<inputMethod>MPS</inputMethod>





<comments><![CDATA[


Problem 2.2 in Vasek Chvatal's book: Linear Programming


maximize    2*x1 + 3*x2 + 5*x3 + 4*x4


subject to    x1 + 2*x2 + 3*x3 +   x4 <= 5


              x1 +   x2 + 2*x3 + 3*x4 <= 3


              x1 ,   x2 ,   x3 ,   x4 >= 0


For any choice of a >= 0 and b >= 0, the values


     x1 = a, x4 = b,  x2 = 1 + a + 7*b, x3 = 1 - a - 5*b


yield an optimal for this problem.


The corresponding slack variable values are given by


     0 = s1 = 5 - r1, 0 = s2 = 3 - r2


]]></comments>





<MPS><![CDATA[*


*Op Name0---  Name1---  Value1------   Name2---  Value2------


*23 56789012  56789012  567890123456   01234567  012345678901


NAME          CP-2.2


ROWS


 N  z


 L  r1


 L  r2


COLUMNS


    x1        z         2              r1        1


    x1        r2        1


*


    x2        z         3              r1        2


    x2        r2        1


*


    x3        z         5              r1        3


    x3        r2        2


*


    x4        z         4              r1        1


    x4        r2        3


RHS


    b         r1        5


    b         r2        3


ENDATA


*]]></MPS>





<param><![CDATA[


maximize


primalSimplex


solution -


]]></param>





</document>

2.5.1.b: Output File




%%%%%%%%%%%%%%%%%%%% CLP Results %%%%%%%%%%%%%%%%%%%%





Load Avg: ( 2.09 , 2.05 , 2.01 )


Coin LP version 1.02.02, build Aug  3 2005


At line 4 NAME          CP-2.2


At line 5 ROWS


At line 9 COLUMNS


At line 21 RHS


At line 24 ENDATA


Problem CP-2.2 has 2 rows, 4 columns and 8 elements


Model was imported from ./clp.mps in 0 seconds


Switching to line mode


Clp:Clp:Clp:Presolve 2 (0) rows, 4 (0) columns and 8 (0) elements


0  Obj -0 Dual inf 10.642 (4)


2  Obj 8


Optimal - objective value 8


Optimal objective 8 - 2 iterations time 0.002


Clp:


      0 r1                    5                      1


      1 r2                    3                      1


      0 x1                    1         -7.9526954e-17


      1 x2                    2           1.282961e-16


      2 x3                    0          3.3210123e-16


      3 x4                    0         -1.2220844e-15


Clp:





%%%%%%%%%%%%%%%%%%%% CLP Results %%%%%%%%%%%%%%%%%%%%

2.5.1.c: Solution Correspondence
The values of the objective z, r1, and r2 are as expected. If we chose a = 1 and b = 0, the corresponding optimal solution is




     x1 = 1, x4 = 0,  x2 = 2, x3 = 0

which agrees with the Clp output.

2.6: Two Phase Simplex Method
2.6.a: Original Problem
Consider Problem 3.9.a of the text:




maximize    z  =   3*x1 + x2


subject to  s1 = -   x1 + x2 - 1


            s2 =     x1 + x2 - 3


            s3 = - 2*x1 - x2 + 4


            x1, x2, s1, s2, s3 >= 0

Note that the basic solution corresponding to this dictionary




     x1 = 0, x2 = 0, s1 = -1, s2 = -3, s3 = 4, z = 0

is not feasible because s1 and s2 are less than zero.

2.6.b: Neos Solution
You can select the following links to view the Neos problem 3.9a input file: 2.6.1.a and output 3.9a input file: 2.6.1.b for this problem.

2.6.c: Phase I: Obtaining a Feasible Solution

2.6.c.a: Auxiliary Problem




maximize    y  = - x0


subject to  s1 =   x0 -   x1 + x2  - 1


            s2 =   x0 +   x1 + x2  - 3


            s3 =   x0 - 2*x1 - x2  + 4


            x0, x1, x2, s1, s2, s3 >= 0

Note that in the basic solution, s2 has the smallest (most negative) value. Thus, if we pivot on the pair of variables (x0, s2), the resulting basic solution will be feasible.

2.6.c.b: Auxiliary Pivot
The equation for the new basic variable is




     x0 = -x1 - x2 + s2 + 3

We replace all occurrences of x0 with the right hand side and obtain:




maximize    y  = - (-x1 - x2 + s2 + 3)


subject to  s1 =   (-x1 - x2 + s2 + 3) -   x1 + x2  - 1


            x0 =    -x1 - x2 + s2 + 3


            s3 =   (-x1 - x2 + s2 + 3) - 2*x1 - x2  + 4


            x0, x1, x2, s1, s2, s3 >= 0




maximize    y  =     x1 +   x2 - s2 - 3


subject to  s1 = - 2*x1        + s2 + 2


            x0 = -   x1 -   x2 + s2 + 3


            s3 = - 3*x1 - 2*x2 + s2 + 7


            x0, x1, x2, s1, s2, s3 >= 0

The corresponding basic feasible solution is




     x0 = 3, x1 = 0, x2 = 0, s1 = 2, s2 = 0, s3 = 7, y = -3

2.6.c.c: Next Pivot: Auxiliary Problem
We note that the coefficient for x2 in the auxiliary objective y is positive. The corresponding feasibility constraint is




     x2 = min( 3/1, 7/2 ) = 3

The corresponding pivot pair is (x2, x0). The corresponding equation for x2 is




     x2 = - x1 - x0 + s2 + 3

The corresponding dictionary is




maximize    y  =     x1 -   (x1 + x0 - s2 - 3) - s2 - 3


subject to  s1 = - 2*x1        + s2 + 2


            x0 = -   x1 +   (x1 + x0 - s2 - 3) + s2 + 3


            s3 = - 3*x1 + 2*(x1 + x0 - s2 - 3) + s2 + 7


            x0, x1, x2, s1, s2, s3 >= 0




maximize    y  = - x0


subject to  s1 =       - 2*x1 + s2 + 2


            x2 = - x0  -   x1 + s2 + 3


            s3 = 2*x0  -   x1 - s2 + 1 


            x0, x1, x2, s1, s2, s3 >= 0

The corresponding basic feasible solution is




     x0 = 0, x1 = 0, x2 = 3, s1 = 2, s2 = 0, s3 = 1, y = 0

We note that x0 is zero. Hence this must be an optimal value for the auxiliary problem and a feasible point for the original problem.

2.6.d: Phase II: Solving the Original Problem

2.6.d.a: Original Problem Feasible Dictionary
The constraints in the auxiliary problem, plus the condition x0 = 0, are equivalent to the constraints in the original problem. Thus the following problem is equivalent to the original problem.




maximize    z  =   3*x1 + x2


subject to  s1 = - 2*x1 + s2 + 2


            x2 = -   x1 + s2 + 3


            s3 = -   x1 - s2 + 1 


            x1, x2, s1, s2, s3 >= 0

Now we use the constraint equations to replace the basic variables in the objective; i.e.,




     z  =  3*x1 + (- x1 + s2 + 3)

The resulting feasible dictionary representation is




maximize    z  =   2*x1 + s2 + 3


subject to  s1 = - 2*x1 + s2 + 2


            x2 = -   x1 + s2 + 3


            s3 = -   x1 - s2 + 1 


            x1, x2, s1, s2, s3 >= 0

The corresponding basic feasible solution is




     x1 = 0, x2 = 3, s1 = 2, s2 = 0, s3 = 1, z = 3

2.6.d.b: Next Pivot: Original Problem
We now choose s2 for our next pivot column. The corresponding feasibility constraint is




     s2 = 1

The corresponding pivot pair is (s2, s3). The corresponding equation for s2 is




     s2 = - x1 - s3 + 1

The corresponding dictionary is




maximize    z  =   2*x1 + (- x1 - s3 + 1) + 3


subject to  s1 = - 2*x1 + (- x1 - s3 + 1) + 2


            x2 = -   x1 + (- x1 - s3 + 1) + 3


            s2 =           - x1 - s3 + 1 


            x1, x2, s1, s2, s3 >= 0




maximize    z  =     x1 - s3 + 4


subject to  s1 = - 3*x1 - s3 + 3


            x2 = - 2*x1 - s3 + 4


            s2 = -   x1 - s3 + 1 


            x1, x2, s1, s2, s3 >= 0

The corresponding basic feasible solution is




     x1 = 0, x2 = 4, s1 = 3, s2 = 1, s3 = 0, z = 4

2.6.d.c: Last Pivot: Original Problem
We now choose x1 for our next pivot column. The corresponding feasibility constraint is




     x1 = min(3/3, 4/2, 1/1) = 1

We choose (x1, s2) for the corresponding pivot pair (we could have chosen (x1, s1)). The corresponding equation for x1 is




     x1 = - s2 - s3 + 1

The corresponding dictionary is




maximize    z  =     (- s2 - s3 + 1) - s3 + 4


subject to  s1 = - 3*(- s2 - s3 + 1) - s3 + 3


            x2 = - 2*(- s2 - s3 + 1) - s3 + 4


            x1 =      - s2 - s3 + 1 


            x1, x2, s1, s2, s3 >= 0




maximize    z  = -   s2 - 2*s3 + 5


subject to  s1 =   3*s2 + 2*s3 + 0


            x2 =   2*s2 +   s3 + 2


            x1 =   - s2 -   s3 + 1 


            x1, x2, s1, s2, s3 >= 0

The corresponding basic feasible solution is




     x1 = 1, x2 = 2, s1 = 0, s2 = 0, s3 = 0, z = 5

2.6.1: Neos Input and Output File for Problem 3.9a in Chvatal
2.6.1.a: Input File




<document>





<category>lp</category>


<solver>Clp</solver>


<inputMethod>MPS</inputMethod>





<comments><![CDATA[


maximize    3*x1 + x2


subject to    x1 - x2 <= - 1


            - x1 - x2 <= - 3


            2*x1 + x2 <=   4


              x1 , x2 >= 0


The corresponding optimal solution is


     x1 = 1, x2 = 2, r1 = -1, r2 = -3, r3 = 4


]]></comments>





<MPS><![CDATA[*


*Op Name0---  Name1---  Value1------   Name2---  Value2------


*23 56789012  56789012  567890123456   01234567  012345678901


NAME          CP-2.2


ROWS


 N  z


 L  r1


 L  r2


 L  r3


COLUMNS


    x1        z         3              r1         1


    x1        r2        -1             r3         2


*


    x2        z         1              r1         -1


    x2        r2        -1             r3         1


RHS


    b         r1        -1


    b         r2        -3


    b         r3        4


ENDATA


*]]></MPS>





<param><![CDATA[


maximize


primalSimplex


solution -


]]></param>





</document>

2.6.1.b: Output File




%%%%%%%%%%%%%%%%%%%% CLP Results %%%%%%%%%%%%%%%%%%%%





Load Avg: ( 4.01 , 4.0 , 3.91 )


Coin LP version 1.02.02, build Aug  3 2005


At line 4 NAME          CP-2.2


At line 5 ROWS


At line 10 COLUMNS


At line 16 RHS


At line 20 ENDATA


Problem CP-2.2 has 3 rows, 2 columns and 6 elements


Model was imported from ./clp.mps in 0 seconds


Switching to line mode


Clp:Clp:Clp:Presolve 0 (-3) rows, 0 (-2) columns and 0 (-6) elements


Empty problem - 0 rows, 0 columns and 0 elements


Optimal - objective value 5


After Postsolve, objective 5, infeasibilities - dual 0 (0), primal 0 (0)


Optimal objective 5 - 0 iterations time 0.002, Presolve 0.00


Clp:


      0 r1                   -1             0.33333333


      1 r2                   -3                     -0


      2 r3                    4              1.3333333


      0 x1                    1          1.6653345e-16


      1 x2                    2          5.5511151e-17


Clp:





%%%%%%%%%%%%%%%%%%%% CLP Results %%%%%%%%%%%%%%%%%%%%

2.7: Degeneracy
2.7.a: Definition
When the basic feasible solution has some of the basic variables equal to zero, the solution is called degenerate.

2.7.b: Example
We consider the Tableau corresponding to the problem at the bottom of page 29 of the text:




      x1   x2   x3   x4   x5   x6    z    b


       0    0    2    1    0    0    0    1 


       2   -4    6    0    1    0    0    3 


      -1    3    4    0    0    1    0    2 


       2   -1    8    0    0    0   -1    0

The corresponding basic feasible solution is




     x1 = 0, x2 = 0, x3 = 0, x4 = 1, x5 = 1, x6 = 1, z = 0

Note that if we choose x3 for the new basic variable, its value after the pivot will be




     x3 = min (1/2, 3/6, 2/4)  = 1/2

The important point here is that if we increase x3 to 1/2, the basic variables x4, x5, x6 will all be zero and hence we could choose any one of them for the new non-basic variable.

2.7.c: First Pivot
As in the text, we choose the pivot pair (x3, x4). The resulting tableau is:




      x1   x2   x3   x4   x5   x6    z    b


      0,   0,    1,  .5,   0,   0,   0,  .5


      2,  -4,    0,  -3,   1,   0,   0,   0


     -1,   3,    0,  -2,   0,   1,   0,   0


      2,  -1,    0,  -4,   0,   0,  -1,  -4

The corresponding basic feasible solution is




     x1 = 0, x2 = 0, x3 = .5, x4 = 0, x5 = 0, x6 = 0, z = 4

Note that x1 is the only possible choice for the new basic variable, its value after the next pivot will be




     x1 = 0/2  = 0

The important point here is that x1 will not increase when we change the basis. Hence the objective function value corresponding to the basic solution for the new basis will be the same as the objective function value for this basis.

2.7.d: Second Pivot
As in the text, this pivot pair is (x1, x5). The resulting tableau is:




      x1   x2   x3    x4   x5   x6    z    b


       0    0    1    .5    0    0    0   .5


       1   -2    0  -1.5   .5    0    0    0


       0    1    0  -3.5   .5    1    0    0


       0    3    0    -1   -1    0   -1   -4

The corresponding basic feasible solution is




     x1 = 0, x2 = 0, x3 = .5, x4 = 0, x5 = 0, x6 = 0, z = 4

Note that x2 is the only possible choice for the new basic variable, its value after the next pivot will be




     x2 = 0/1  = 0

2.7.e: Third Pivot
This pivot pair is (x2, x6). The resulting tableau is:




      x1   x2   x3    x4    x5   x6    z    b


       0    0    1   0.5     0    0    0   .5


       1    0    0  -8.5   1.5    2    0    0


       0    1    0  -3.5    .5    1    0    0


       0    0    0   9.5  -2.5   -3   -1   -4

The corresponding basic feasible solution is




     x1 = 0, x2 = 0, x3 = .5, x4 = 0, x5 = 0, x6 = 0, z = 4

Note that x4 is the only possible choice for the new basic variable, its value after the next pivot will be




     x2 = .5/.5  = 1

2.7.f: Fourth Pivot
This pivot pair is (x4, x3). The resulting tableau is:




      x1   x2   x3   x4    x5   x6    z      b


       0   0     2    1     0    0    0      1


       1   0    17    0   1.5    2    0    8.5


       0   1     7    0    .5    1    0    3.5


       0   0   -19    0  -2.5   -3   -1  -13.5

The corresponding basic feasible solution is




     x1 = 8.5, x2 = 3.5, x3 = 0, x4 = 1, x5 = 0, x6 = 0, z = 13.5

Note that this is the optimal solution because none of the coefficients in the last row are greater than zero.

2.8: The Cycling of Simplex Method
2.8.a: Definition
A basis is a one to one mapping of the form

B : {1, \dots, m} \to {1, \dots, n}

where

m

is the number of constraints in the tableau and

n

is the number of variables. The basis corresponding to an iteration of the simplex method is the mapping

B

such that column

B (k)

of the Tableau contains the k-th column of the

m \times m

identity matrix.

2.8.b: Basis Lemma
If two iterations of the simplex method correspond to the same basis, all the other terms in the corresponding tableau are also the same.

2.8.b.a: Proof
Suppose that we have two iterations with the same basis. Let

B

be that basis and

S \in R^{(m + 1) \times (n + 2)}

T \in R^{(m + 1) \times (n + 2)}

, be the tableaus corresponding to the two iterations. It follows that there is an invertible matrix

A \in R^{m \times m}

such that

A S = T

For

k = 1, \dots, m

, we use

e^{k}

to denote the k-th column of the

m \times m

identity matrix. It follows that

e^{k}

is equal to column

B (k)

of both

S

and

T

. Thus, for

i = 1, \dots, m

\begin{matrix} \sum_{j = 1}^{m} A_{i, j} S_{j, B (k)} & = & T_{i, B (k)} \\ \sum_{j = 1}^{m} A_{i, j} e_{i}^{k} & = & e_{i}^{k} \\ A_{i, k} & = & {\begin{matrix} 1 & if i = k \\ 0 & otherwise \end{matrix} \end{matrix}

It follows that

A

is the identity matrix and hence

S = T

.

2.8.c: Cycling Corollary
If the simplex algorithm does not terminate, then it must cycle; i.e., the tableau must completely repeat itself.

2.8.c.a: Proof
There is a basis corresponding to each iteration of the simplex method and there are only a finite number of such mappings,

n! / (n - m)!

to be exact. Thus the basis must repeat itself and hence the entire tableau must repeat itself.

2.9: Bland's Pivot Rule
2.9.a: Notation
We use

B^{k} : {1, \dots, m} \to {1, \dots, n}

to denote the basis corresponding to the k-th iteration of the Simplex Algorithm. We use

j \in B^{k}

to denote the fact that

j

is in the range of

B^{k}

; i.e., there exists an

i

such that

j = B^{k} (i)

. We use

B^{- k}

to denote the inverse of

B^{k}

(defined on the range of

B^{k}

); i.e., for

i = 1, \dots, m

B^{- k} [B^{k} (i)] = i

.

2.9.a.a: Dictionary
We use the following notation for the dictionary at the k-th iteration

\begin{matrix} x_{B^{k} (i)} & = & b_{i}^{k} - \sum_{j \notin B^{k}} A_{i, j}^{k} x_{j} (i = 1, \dots, m) \\ z & = & v^{k} + \sum_{j \notin B^{k}} c_{j}^{k} x_{j} \end{matrix}

where

A^{k} \in R^{m \times n}

b^{k} \in R^{m}

c^{k} \in R^{n}

v^{k} \in R

.

2.9.b: The Pivot Rule
Choose the pivot column

J (k)

as the minimum column index in the set

{j : c_{j}^{k} > 0}

Choose the pivot row index

I (k)

such that

B^{k} [I (k)]

is the minimum column index in the set of indices

{B^{k} (i) : i \in {1, \dots, m} and A_{i, J (k)}^{k} > 0}

2.9.c: Example
Example Use of Bland's Method: 2.9.1

2.9.d: Bland's Theorem
If you use Bland's pivot rule, the Simplex method cannot cycle; i.e., it is not possible for the basis

B^{k}

to be the same as

B^{k + ℓ}

for some positive integer

ℓ

.

2.9.e: Proof
This proof is essentially the same as on pages 37-38 of Chvatal: 8.a . The main difference in this version of the proof is that the notation makes it easier to remember what the different terms mean. With out loss of generality, suppose that the cycle starts at the zero iteration; i.e.,

B^{0}

is the same as

B^{ℓ}

. We will prove the theorem by showing that this leads to a contradiction.

Define the iteration

p

J (p) = \max {J (k) : k = 1, \dots, ℓ}

It follows that

J (p) \notin B^{p}

and

J (p) \in B^{p + 1}

. Thus there is an iteration

q

such that

J (p) \in B^{q}

and

J (p) \notin B^{q + 1}

. We now define the function

X : R \to R^{n}

X_{j} (λ) = {\begin{matrix} 0 & if j \notin B^{q} and j \neq J (q) \\ λ & if j = J (q) \\ b_{B^{- q} (j)}^{q} - λ A_{B^{- q} (j), J (q)}^{q} & if j \in B^{q} \end{matrix}

(note

i = B^{- q} (j)

means that

B^{q} (i) = j

). It follows that for

i = 1, \dots m

X_{B^{q} (i)} (λ) = b_{i}^{q} - \sum_{j \notin B^{q}} A_{i, j}^{q} X_{j} (λ)

Thus the objective function evaluated at

X_{j} (λ)

must have the same value in the dictionary at iteration

p

and the dictionary at iteration

q

; i.e.,

v^{p} + \sum_{j \notin B^{p}} c_{j}^{p} X_{j} (λ) = v^{q} + \sum_{j \notin B^{q}} c_{j}^{q} X_{j} (λ)

The objective function value corresponding to the basic feasible solution at iteration

k

v^{k}

. In addition, the Simplex algorithm ensures that

v^{k + 1} \geq v^{k}

. On the other hand, the fact that

B^{1} = B^{ℓ}

ensures that

v^{1} = v^{ℓ}

. It follows that

v^{1} = v^{k}

for all

k

and

\sum_{j \notin B^{q}} c_{j}^{q} X_{j} (λ) = \sum_{j \notin B^{p}} c_{j}^{p} X_{j} (λ)

X_{j} (λ)

is a non-zero component of

X (λ)

, either

j \in B^{q}

j = J (q)

. Thus we have

\begin{matrix} c_{J (q)}^{q} X_{J (q)} (λ) & = & c_{J (q)}^{p} X_{J (q)} (λ) + \sum_{j \in B^{q}} c_{j}^{p} X_{j} (λ) \\ λ [c_{J (q)}^{q} - c_{J (q)}^{p} + \sum_{j \in B^{q}} c_{j}^{p} A_{B^{- q} (j), J (p)}^{q}] & = & \sum_{j \in B^{q}} c_{j}^{p} b_{B^{- q} (j)}^{q} \end{matrix}

This equation holds for all

λ \in R

, hence

\begin{matrix} c_{J (q)}^{q} - c_{J (q)}^{p} + \sum_{j \in B^{q}} c_{j}^{p} A_{B^{- q} (j), J (p)}^{q} = 0 \end{matrix}

We know that

c_{J (q)}^{q} > 0

because

J (q)

is chosen as the pivot column at iteration

q

. We also know that

c_{J (q)}^{p} \leq 0

because

J (p)

is chosen as the pivot column at iteration

p

and

J (p) > J (q)

. Thus, it follows that there is a column index

N

such that

c_{N}^{p} A_{B^{- q} (N), J (p)}^{q} < 0

We know that

J (p) \in B^{q}

and

J (p) \notin B^{q + 1}

It follows that

A_{B^{- q} [J (p)], J (p)}^{q} > 0

We also know that

c_{J (p)}^{p} > 0

thus,

N \neq J (p)

. We note also have that

c_{j}^{p}

is zero for all

j \in B^{p}

whence,

N \notin B^{p}

. We also have that

N \in B^{q}

(because

c_{N}^{q} \neq 0

), whence

N = J (k)

for some

k

and

N < J (p)

. It now follows that

c_{N}^{p} \leq 0

(otherwise it would have been chosen at iteration

p

J (p)

) and thus

A_{B^{- q} (N), J (p)}^{q} > 0

Thus,

N < J (p)

and

N

is in the set

{B^{q} (i) : i \in {1, \dots, m} and A_{i, J (p)}^{q} > 0}

This contradicts the fact that

J (p)

is the minimal element of the set above (which follows form

J (p) \in B^{q}

and

J (p) \notin B^{q + 1}

). This contradiction completes the proof of the theorem.

2.9.1: Example Use of Bland's Method
2.9.1.a: Initial Dictionary
We begin with the problem on Page 31 of Chvatal: 8.a :




maximize z subject to:


     x5 =    -.5*x1 + 5.5*x2 + 2.5*x3 -  9*x4


     x6 =    -.5*x1 + 1.5*x2 +  .5*x3 -    x4


     x7 = 1  -   x1


     z  =     10*x1 - 57*x2  -   9*x3 - 24*x4


(x1, x2, x3, x4, x5, x6, x7) >= 0

2.9.1.b: Example Pivot Rule
The pivot rule used to choose rows (variable to leave basis) in this example is to choose the candidate with the smallest index. This is the same as the row choice in Bland's rule. The pivot rule to choose columns (nonbasic variable to enter basis) in this example is to choose the candidate that has the largest coefficient in the z equation. Thus, so long at the column candidate with the largest coefficient is also the column candidate with the smallest index, the result will be the same as for Bland's rule.

2.9.1.c: First Pivot
In the first dictionary

 


     z  = 10*x1 - 57*x2 - 9*x3 - 24*x4

Hence is only one column with z coefficient greater than zero. Hence the pivot with Bland's rule would be the same as for the example.

2.9.1.d: Second Pivot
In the second dictionary

 


     z = 53*x2 + 41*x3 - 204*x4 - 20*x5

Hence there are two candidate columns, the one corresponding to x2 and the one corresponding to x3. The one with the smaller index is also the one with the larger coefficient. Hence the pivot with Bland's rule would be the same as for the example.

2.9.1.e: Third Pivot
In the third dictionary

 


     z  = 14.5*x3 - 98*x4 - 6.75*x5 - 13.25*x6

Hence is only one column with z coefficient greater than zero. Hence the pivot with Bland's rule would be the same as for the example.

2.9.1.f: Fourth Pivot
In the fourth dictionary

 


     z = - 29*x1 + 18*x4 + 15*x5 - 93*x6

Hence there are two candidate columns, the one corresponding to x4 and the one corresponding to x5. The one with the smaller index is also the one with the larger coefficient. Hence the pivot with Bland's rule would be the same as for the example.

2.9.1.g: Fifth Pivot
In the fifth dictionary

 


     z  = -20*x1 - 9*x2 + 10.5*x5 - 70.5*x6

Hence is only one column with z coefficient greater than zero. Hence the pivot with Bland's rule would be the same as for the example.

2.9.1.h: Tableau A
The sixth dictionary (dictionary after the fifth pivot) is




x5 =      4*x1 -   8*x2 -  2*x3 +  9*x6


x4 =   - .5*x1 + 1.5*x2 + .5*x3 -    x6


x7 = 1 -    x1


z  =     22*x1 -  93*x2 - 21*x3 + 24*x6

The corresponding tableau is




     x1      x2      x3      x4      x5      x6      x7       z       b


  -4.00    8.00    2.00    0.00    1.00   -9.00    0.00    0.00    0.00


   0.50   -1.50   -0.50    1.00    0.00    1.00    0.00    0.00    0.00


   1.00    0.00    0.00    0.00    0.00    0.00    1.00    0.00    1.00


  22.00  -93.00  -21.00    0.00    0.00   24.00    0.00   -1.00    0.00

2.9.1.i: Sixth Pivot (Example Method)
The sixth example pivot chooses the pivot pair (x6, x4). The result of this pivot on Tableau A is:




     x1      x2      x3      x4      x5      x6      x7       z       b


   0.50   -5.50   -2.50    9.00    1.00    0.00    0.00    0.00    0.00


   0.50   -1.50   -0.50    1.00    0.00    1.00    0.00    0.00    0.00


   1.00    0.00    0.00    0.00    0.00    0.00    1.00    0.00    1.00


  10.00  -57.00   -9.00  -24.00    0.00    0.00    0.00   -1.00    0.00

2.9.1.j: Sixth Pivot (Bland Method)
Bland's rule chooses the pivot pair (x1, x4). The result of this pivot on Tableau A we call Tableau B:




     x1      x2      x3      x4      x5      x6      x7       z       b


   0.00   -4.00   -2.00    8.00    1.00   -1.00    0.00    0.00    0.00


   1.00   -3.00   -1.00    2.00    0.00    2.00    0.00    0.00    0.00


   0.00    3.00    1.00   -2.00    0.00   -2.00    1.00    0.00    1.00


   0.00  -27.00    1.00  -44.00    0.00  -20.00    0.00   -1.00    0.00

2.9.1.k: Seventh Pivot (Bland Method)
Bland's rule chooses the pivot pair (x3, x7). The result of this pivot on Tableau B we call Tableau C:




     x1      x2      x3      x4      x5      x6      x7       z       b


   0.00    2.00    0.00    4.00    1.00   -5.00    2.00    0.00    2.00


   1.00    0.00    0.00    0.00    0.00    0.00    1.00    0.00    1.00


   0.00    3.00    1.00   -2.00    0.00   -2.00    1.00    0.00    1.00


   0.00  -30.00    0.00  -42.00    0.00  -18.00   -1.00   -1.00   -1.00

We see that this is the basic feasible solution




     x1 = 1, x2 = 0, x3 = 1, x4 = 0, x5 = 2, x6 = 0, x7 = 0

is optimal and the corresponding objective value is one.

2.9.1.l: Neos Solution




%%%%%%%%%%%%%%%%%%%% CLP Results %%%%%%%%%%%%%%%%%%%%





Load Avg: ( 4.0 , 4.02 , 3.95 )


Coin LP version 1.02.02, build Aug  3 2005


At line 4 NAME          Cycle


At line 5 ROWS


At line 10 COLUMNS


At line 22 RHS


At line 26 ENDATA


Problem Cycle has 3 rows, 4 columns and 9 elements


Model was imported from ./clp.mps in 0 seconds


Switching to line mode


Clp:Clp:Clp:Clp:Presolve 2 (-1) rows, 3 (-1) columns and 6 (-3) elements


0  Obj -0 Dual inf 16.0282 (1)


2  Obj 1


Optimal - objective value 1


After Postsolve, objective 1, infeasibilities - dual 0 (0), primal 0 (0)


Optimal objective 1 - 2 iterations time 0.002, Presolve 0.00


Clp:


      0 r1                   -2                     -0


      1 r2        8.0135898e-13                     18


      2 r3                    1                      1


      0 x1                    1          -1.110223e-15


      1 x2                    0                    -30


      2 x3                    1                      0


      3 x4                    0                    -42


Clp:





%%%%%%%%%%%%%%%%%%%% CLP Results %%%%%%%%%%%%%%%%%%%%

2.9.1.m: Neos Input File




<document>





<category>lp</category>


<solver>Clp</solver>


<inputMethod>MPS</inputMethod>


<comments><![CDATA[


Problem at top of page 31 in Chvatal


maximize     10*x1 -  57*x2 -   9*x3 - 24*x4 


subject to   .5*x1 - 5.5*x2 - 2.5*x3 +  9*x4 <= 0


             .5*x1 - 1.5*x2 -  .5*x3 +    x4 <= 0


                x1                           <= 1


                x1 ,     x2 ,     x3 ,    x4 >= 0


The solution is x1 = 1, x2 = 0, x3 = 1, x4 = 0


The row values are r1 = -2, r2 = 0, r3 = 1


]]></comments>


<MPS><![CDATA[*


*Op Name0---  Name1---  Value1------   Name2---  Value2------


*23 56789012  56789012  567890123456   01234567  012345678901


NAME          Cycle 


ROWS


 N  z


 L  r1


 L  r2


 L  r3


COLUMNS


    x1        z         10             r1        .5


    x1        r2        .5             r3        1


*


    x2        z         -57            r1        -5.5


    x2        r2        -1.5


*


    x3        z         -9             r1        -2.5


    x3        r2        -.5 


*


    x4        z         -24            r1        9


    x4        r2        1


RHS


    b         r1        0


    b         r2        0 


    b         r3        1


ENDATA


*]]></MPS>





<param><![CDATA[


maximize


primalPivot dantzig


primalSimplex


solution -


]]></param>





</document>

3: Linear Programming Duality Theory
3.a: Contents

A Linear Programming Duality Example: 3.1
Duality With Inequality Constraints: 3.2
The Duality Theorem: 3.3
The Complementary Slackness Condition: 3.4
An Example Computing the Dual Variables: 3.5
Relation Between Dual and Perturbed Upper Bound: 3.6
Duality with Equality and Inequality Constraints: 3.7

3.1: A Linear Programming Duality Example
3.1.a: Notation
We use

R_{+}^{n}

to denote the set of

x \in R^{n}

such that

x \geq 0

; i.e.,

x_{j} \geq 0

for

j = 1, \dots, n

.

3.1.b: Primal Problem
We start with the primal problem on page 54 of Chvatal: 8.a :

\begin{matrix} maximize & 4 x_{1} & + & x_{2} & + & 5 x_{3} & + & 3 x_{4} \\ subject to & x_{1} & - & x_{2} & - & x_{3} & + & 3 x_{4} & \leq & 1 \\ 5 x_{1} & + & x_{2} & + & 3 x_{3} & + & 8 x_{4} & \leq & 55 \\ - x_{1} & + & 2 x_{2} & + & 3 x_{3} & - & 5 x_{4} & \leq & 3 \\ x_{1} & , & x_{2} & , & x_{3} & , & x_{4} & \geq & 0 \end{matrix}

The feasible set for this problem is defined by

X = {x \in R_{+}^{4} : \begin{matrix} x_{1} & - & x_{2} & - & x_{3} & + & 3 x_{4} & \leq & 1 \\ 5 x_{1} & + & x_{2} & + & 3 x_{3} & + & 8 x_{4} & \leq & 55 \\ - x_{1} & + & 2 x_{2} & + & 3 x_{3} & - & 5 x_{4} & \leq & 3 \end{matrix}}

The extended primal objective function for this problem

f : R_{+}^{4} \to R \cup {- \infty}

is defined by

f (x) = {\begin{matrix} 4 x_{1} + x_{2} + 5 x_{3} + 3 x_{4} & if x \in X \\ - \infty & otherwise \end{matrix}

Solving the primal problem is equivalent to finding an argument

x

that maximizes the value of

f (x)

with respect to

x \in R_{+}^{4}

.

3.1.c: The Lagrangian

3.1.d: Lagrangian, example
The Lagrangian corresponding to this problem

L : R_{+}^{4} \times R_{+}^{3} \to R

is defined by

\begin{matrix} L (x, y) & = & 4 x_{1} + x_{2} + 5 x_{3} + 3 x_{4} \\ + & y_{1} (1 - x_{1} + x_{2} + x_{3} - 3 x_{4}) \\ + & y_{2} (55 - 5 x_{1} - x_{2} - 3 x_{3} - 8 x_{4}) \\ + & y_{3} (3 + x_{1} - 2 x_{2} - 3 x_{3} + 5 x_{4}) \end{matrix}

It follows from this definition that for all

x \in R_{+}^{4}

f (x) = \inf L (x, y) with respect to y \in R_{+}^{3}

Regrouping terms in the definition of

L

, we have

\begin{matrix} L (x, y) & = & y_{1} + 55 y_{2} + 3 y_{3} \\ + & x_{1} (4 - y_{1} - 5 y_{2} + y_{3}) \\ + & x_{2} (1 + y_{1} - y_{2} - 2 y_{3}) \\ + & x_{3} (5 + y_{1} - 3 y_{2} - 3 y_{3}) \\ + & x_{4} (3 - 3 y_{1} - 8 y_{2} + 5 y_{3}) \end{matrix}

3.1.e: Dual Problem
The extended dual objective function for this problem

g : R_{+}^{3} \to R \cup {+ \infty}

is defined by

g (y) = \sup L (x, y) with respect to x \in R_{+}^{4}

The corresponding feasible set for the dual problem is

Y = {y \in R_{+}^{3} : \begin{matrix} y_{1} & + & 5 y_{2} & + & y_{3} \geq & 4 \\ - y_{1} & + & y_{2} & + & 2 y_{3} \geq & 1 \\ - y_{1} & + & 3 y_{2} & + & 3 y_{3} \geq & 5 \\ 3 y_{1} & + & 8 y_{2} & - & 5 y_{3} \geq & 3 \end{matrix}}

And the dual objective can be represented by

g (y) = {\begin{matrix} y_{1} + 55 y_{2} + 3 y_{3} & if y \in Y \\ + \infty & otherwise \end{matrix}

The dual problem is to find an argument value

y

that minimizes

g (y)

with respect to

y \in R_{+}^{3}

. This is equivalent to the following problem:

\begin{matrix} minimize & y_{1} & + & 55 y_{2} & + & 3 y_{3} \\ subject to & y_{1} & + & 5 y_{2} & + & y_{3} & \geq & 4 \\ - y_{1} & + & y_{2} & + & 2 y_{3} & \geq & 1 \\ - y_{1} & + & 3 y_{2} & + & 3 y_{3} & \geq & 5 \\ 3 y_{1} & + & 8 y_{2} & - & 5 y_{3} & \geq & 3 \\ y_{1} & , & y_{2} & , & y_{3} & \geq & 0 \end{matrix}

3.2: Duality With Inequality Constraints
3.2.a: Primal Problem
Given

A \in R^{m \times n}

b \in R^{m}

, and

c \in R^{n}

, the corresponding primal problem is

\begin{matrix} maximize & c^{T} x with respect to x \in R_{+}^{n} \\ subject to & A x \leq b \end{matrix}

The extended primal objective function

f : R_{+}^{n} \to R \cup {- \infty}

is defined by

f (x) = {\begin{matrix} c^{T} x & if b - A x \geq 0 \\ - \infty & otherwise \end{matrix}

Solving the primal problem is equivalent to finding an argument

x

that maximizes the value of

f (x)

with respect to

x \in R_{+}^{n}

.

3.2.b: Lagrangian
The Lagrangian corresponding to the problem above

L : R_{+}^{n} \times R_{+}^{m} \to R

is defined by

\begin{matrix} L (x, y) & = & c^{T} x + y^{T} (b - A x) \\ = & b^{T} y + x^{T} (c - A^{T} y) \end{matrix}

It follows that

f (x) = \inf L (x, y) with respect to y \in R_{+}^{m}

3.2.c: Dual Problem
The extended dual objective function

g : R_{+}^{m} \to R \cup {+ \infty}

is defined by

g (y) = \sup L (x, y) with respect to x \in R_{+}^{n}

It follows that

g (y) = {\begin{matrix} b^{T} y & if c - A^{T} y \leq 0 \\ + \infty & otherwise \end{matrix}

The dual problem is to find an argument value

y

that minimizes

g (y)

with respect to

y \in R_{+}^{m}

. This is equivalent to the following problem:

\begin{matrix} minimize & b^{T} y with respect to y \in R_{+}^{m} \\ subject to & A^{T} y \geq c \end{matrix}

3.2.d: Duality Gap Lemma
Given

\hat{x} \in R_{+}^{n}

and

\hat{y} \in R_{+}^{m}

, the corresponding duality gap is

g (\hat{y}) - f (\hat{x})

. The duality gap is always non-negative and if it is zero,

\hat{x}

solves the primal problem and

\hat{y}

solves the dual problem.

3.2.d.a: Proof

\begin{matrix} g (\hat{y}) & = & \sup L (x, \hat{y}) with respect to x \in R_{+}^{n} \\ \geq & L (\hat{x}, \hat{y}) \\ \geq & \inf L (\hat{x}, y) with respect to y \in R_{+}^{m} \\ = & f (\hat{x}) \end{matrix}

Thus we conclude that the duality gap is non-negative. Further if it equals zero then for all

x \in R_{+}^{n}

and

y \in R_{+}^{m}

\begin{matrix} g (y) & \geq & f (\hat{x}) & = & g (\hat{y}) \\ f (\hat{x}) & = & g (\hat{y}) & \geq & f (x) \end{matrix}

which completes the proof of the lemma.

3.3: The Duality Theorem
3.3.a: Notation
Let

A \in R^{m \times n}

b \in R^{m}

, and

c \in R^{n}

define the primal problem: 3.2.a with extended objective

f (x)

and the dual problem: 3.2.c with extended objective

g (y)

.

3.3.b: Tableau Form
We define

Q \in R^{m \times m}

to be the identity matrix and the slack vector

s \in R_{m}

are defined by

b = A x + Q s

The scalar

e \in R

and the coefficient vector

y \in R^{m}

are defined by

e = 0

y = 0

. The primal problem has the following tableau form:

\begin{matrix} maximize & z \\ subject to & b = A x & + & Q s \\ e = c^{T} x & - & y^{T} s & - & z \\ x \geq 0 & , & s \geq 0 \end{matrix}

3.3.c: Simplex Algorithm
The Simplex algorithm can be used to transform this tableau to other forms with equivalent constraints. In the theorem below, the condition

\hat{c} \leq 0

and

\hat{y} \geq 0

, means that the Simplex method has reached a point where all the objective coefficients are negative. In addition, the objective coefficients corresponding to the basic variables are all zero, hence

0 = {\hat{c}}^{T} \hat{x} - {\hat{y}}^{T} \hat{s}

were

(\hat{x}, \hat{s})

is the basic feasible solution corresponding to the tableau.

3.3.d: Theorem
Suppose that

x \in R^{n}

s \in R^{m}

, and

z \in R

satisfies the constraints in the tableau above if and only if it satisfies the constraints in the following form:

\begin{matrix} maximize & z \\ subject to & \hat{b} = \hat{A} x & + & \hat{Q} s \\ \hat{e} = {\hat{c}}^{T} x & - & {\hat{y}}^{T} s & - & z \\ x \geq 0 & , & s \geq 0 \end{matrix}

Further suppose,

\hat{c} \leq 0

\hat{y} \geq 0

, and there is a feasible pair

(\hat{x}, \hat{s})

such that

0 = {\hat{c}}^{T} \hat{x} - {\hat{y}}^{T} \hat{s}

It follows that

\hat{x}

solves the primal problem and

\hat{y}

solves the dual problem.

3.3.e: Proof
We have the equivalence of the two tableaus and

Q

is the identity matrix,

d = 0

, and

e = 0

. Thus for all

x \in R_{+}^{n}

, by setting

s = b - A x

we have

\begin{matrix} \sum_{j = 1}^{n} c_{j} x_{j} & = & - \hat{e} + \sum_{j = 1}^{n} {\hat{c}}_{j} x_{j} - \sum_{i = 1}^{m} {\hat{y}}_{i} s_{i} \\ = & - \hat{e} + \sum_{j = 1}^{n} {\hat{c}}_{j} x_{j} - \sum_{i = 1}^{m} {\hat{y}}_{i} (b_{i} - \sum_{j = 1}^{n} A_{i, j} x_{j}) \\ \sum_{i = 1}^{m} b_{i} {\hat{y}}_{i} + \hat{e} & = & \sum_{j = 1}^{n} (- c_{j} + {\hat{c}}_{j} + \sum_{i = 1}^{m} {\hat{y}}_{i} A_{i, j}) x_{j} (⋆) \end{matrix}

From equation

(⋆)

, choosing

x = 0

, we obtain

\begin{matrix} \sum_{i = 1}^{m} b_{i} {\hat{y}}_{i} + \hat{e} & = & 0 \\ \sum_{i = 1}^{m} b_{i} {\hat{y}}_{i} & = & - \hat{e} \end{matrix}

Form equation

(⋆)

, choosing

x = 0

except for the j-th component which is one, we obtain the following equation for

j = 1, \dots, n

\begin{matrix} 0 & = & - c_{j} + {\hat{c}}_{j} + \sum_{i = 1}^{m} {\hat{y}}_{i} A_{i, j} \\ c_{j} & \leq & \sum_{i = 1}^{m} {\hat{y}}_{i} A_{i, j} \end{matrix}

Note that we used

{\hat{c}}_{j} \leq 0

to obtain the inequality above. It follows that

\hat{y}

is dual feasible,

\hat{x}

is primal feasible, and

f (\hat{x}) = - \hat{e} + {\hat{c}}^{T} \hat{x} - {\hat{y}}^{T} \hat{s} = - \hat{e} = b^{T} \hat{y} = g (\hat{y})

The conclusion of the theorem now follows from the Duality Gap Lemma: 3.2.d .

3.4: The Complementary Slackness Condition
3.4.a: Notation
Let

A \in R^{m \times n}

b \in R^{m}

, and

c \in R^{n}

define the primal problem: 3.2.a with extended objective

f (x)

and the dual problem: 3.2.c with extended objective

g (y)

.

3.4.b: Definition
The vectors

\hat{x} \in R_{+}^{n}

and

\hat{y} \in R_{+}^{m}

satisfy the complementary slackness condition if the following conditions hold: for

i = 1, \dots, m

, and

j = 1, \dots, n

\begin{matrix} 0 & = & (b_{i} - \sum_{k = 1}^{n} A_{i, k} {\hat{x}}_{k}) {\hat{y}}_{i} \\ 0 & = & (\sum_{k = 1}^{m} {\hat{y}}_{k} A_{k, j} - c_{j}) {\hat{x}}_{j} \end{matrix}

3.4.c: Sufficient Condition
Suppose that

\hat{x} \in R_{+}^{n}

is feasible for the primal problem,

\hat{y} \in R_{+}^{m}

is feasible for the dual problem, and they satisfy the complementary slackness condition. It follows that the corresponding duality gap: 3.2.d is zero.

3.4.c.a: Proof
It follows that

\begin{matrix} b_{i} {\hat{y}}_{i} & = & {\hat{y}}_{i} (\sum_{k = 1}^{n} A_{i, k} {\hat{x}}_{k}) \\ c_{j} {\hat{x}}_{j} & = & (\sum_{k = 1}^{m} {\hat{y}}_{k} A_{k, j}) {\hat{x}}_{j} \end{matrix}

The vector

\hat{x} \in R_{+}^{n}

is feasible for the primal problem and

\hat{y} \in R_{+}^{m}

is feasible for the dual problem. Hence,

\begin{matrix} g (\hat{y}) & = & \sum_{i = 1}^{m} {\hat{y}}_{i} (\sum_{k = 1}^{n} A_{i, k} {\hat{x}}_{k}) \\ f (\hat{x}) & = & \sum_{j = 1}^{n} (\sum_{k = 1}^{m} {\hat{y}}_{k} A_{k, j}) {\hat{x}}_{j} \end{matrix}

Rearranging the indexing we conclude that

\begin{matrix} g (\hat{y}) = \sum_{i = 1}^{m} \sum_{j = 1}^{n} {\hat{y}}_{i} A_{i, j} {\hat{x}}_{j} = f (\hat{x}) \end{matrix}

It follows from the and

\hat{y}

solves the dual problem.

3.4.d: Necessary Condition
Suppose that

\hat{x} \in R_{+}^{n}

and

\hat{y} \in R_{+}^{m}

are such that the corresponding duality gap: 3.2.d is zero. It follows that they satisfy the complementary slackness condition.

3.4.d.a: Proof
The only infinite values for

g (y)

are plus infinity and the only infinite values for

f (x)

are minus infinity. Thus, the duality gap being zero implies that

\hat{x} \in R_{+}^{n}

is primal feasible,

\hat{y} \in R_{+}^{m} is dual feasible

, and

\sum_{j = 1}^{n} c_{j} {\hat{x}}_{j} = \sum_{i = 1}^{m} b_{i} {\hat{y}}_{i}

We now suppose that the first complementary condition is false; i.e., there is an index

I

such that

0 \neq (b_{I} - \sum_{j = 1}^{n} A_{I, j} {\hat{x}}_{j}) {\hat{y}}_{I}

It follows from the fact that

\hat{x}

is feasible and

\hat{y} \in R_{+}^{m}

that

\begin{matrix} 0 & < & (b_{I} - \sum_{j = 1}^{n} A_{I, j} {\hat{x}}_{j}) {\hat{y}}_{I} \\ b_{I} {\hat{y}}_{I} & > & \sum_{j = 1}^{n} {\hat{y}}_{I} A_{I, j} {\hat{x}}_{j} \\ \sum_{i = 1}^{m} b_{i} {\hat{y}}_{i} & > & \sum_{i = 1}^{m} \sum_{j = 1}^{n} {\hat{y}}_{i} A_{i, j} {\hat{x}}_{j} \\ > & \sum_{j = 1}^{n} {\hat{x}}_{j} \sum_{i = 1}^{m} {\hat{y}}_{i} A_{i, j} \end{matrix}

We now apply the fact that

\hat{y}

is feasible to conclude that

\sum_{i = 1}^{m} b_{i} {\hat{y}}_{i} > \sum_{j = 1}^{n} {\hat{x}}_{j} c_{j}

Which contradicts the fact that the duality gap is zero. Thus the first complementary condition must be satisfied for all indices.

We now suppose that the second complementary condition is false; i.e., there is an index

J

such that

0 \neq (\sum_{i = 1}^{n} {\hat{y}}_{i} A_{i, J} - c_{J}) {\hat{x}}_{J}

It follows from the fact that

\hat{y}

is feasible and

\hat{x} \in R_{+}^{n}

that

\begin{matrix} 0 & < & (\sum_{i = 1}^{n} {\hat{y}}_{i} A_{i, J} - c_{J}) {\hat{x}}_{J} \\ c_{J} {\hat{x}}_{J} & < & \sum_{i = 1}^{m} {\hat{y}}_{i} A_{i, J} {\hat{x}}_{J} \\ \sum_{j = 1}^{n} c_{j} {\hat{x}}_{j} & < & \sum_{j = 1}^{n} \sum_{i = 1}^{m} {\hat{y}}_{i} A_{i, j} {\hat{x}}_{j} \\ < & \sum_{i = 1}^{m} {\hat{y}}_{i} \sum_{j = 1}^{n} A_{i, j} {\hat{x}}_{j} \end{matrix}

We now apply the fact that

\hat{x}

is feasible to conclude that

\sum_{j = 1}^{n} c_{j} {\hat{x}}_{j} < \sum_{i = 1}^{m} {\hat{y}}_{i} b_{i}

Which contradicts the fact that the duality gap is zero. Thus the second complementary condition must be satisfied for all indices. This completes the proof.

3.4.e: Example
See Question II: 7.4.b of Quiz0719.

3.5: An Example Computing the Dual Variables We consider the primal problem in Equation (2.1) of the text; i.e.,

\begin{matrix} \begin{matrix} maximize & c^{T} x \\ subject to & A x \leq b & x \in R_{+}^{3} \end{matrix} \\ where A = (\begin{matrix} 2 & 3 & 1 \\ 4 & 1 & 2 \\ 3 & 4 & 2 \end{matrix}), b = (\begin{matrix} 5 \\ 11 \\ 8 \end{matrix}), c = (\begin{matrix} 5 \\ 4 \\ 3 \end{matrix}) \end{matrix}

3.5.a: Primal Solution
According to Equation (2.6) of the text

\hat{x} = (2, 0, 1)^{T}

is a solution of this problem.

3.5.b: Dual Solution
The optimal solution for the dual problem

\hat{y} = (1, 0, 1)^{T}

was computed by hand during Quiz0719: 7.4 .

3.5.c: Neos Clp Solution
A Neos Clp input file corresponding to this problem is given by Equation 2.1 input file: 2.3.a . The corresponding output file is Equation 2.1 output file: 2.3.b . The part of this output corresponding to the Clp solution: 6.2.i command is




Clp:


      0 r1                    5                      1


      1 r2                   10                     -0


      2 r3                    8                      1


      0 x1                    2         -2.8223853e-16


      1 x2                    0                     -3


      2 x3                    1           5.647295e-16


Clp:

This output has the following significance:

\begin{matrix} 5 & = & \sum_{j = 1}^{3} A_{1, j} {\hat{x}}_{j} & {\hat{y}}_{1} & = & 1 \\ 10 & = & \sum_{j = 1}^{3} A_{2, j} {\hat{x}}_{j} & {\hat{y}}_{2} & = & 0 \\ 8 & = & \sum_{j = 1}^{3} A_{3, j} {\hat{x}}_{j} & {\hat{y}}_{3} & = & 1 \\ 2 & = & {\hat{x}}_{1} & c_{1} - \sum_{i = 1}^{3} {\hat{y}}_{i} A_{i, 1} & = & 0 \\ 0 & = & {\hat{x}}_{2} & c_{2} - \sum_{i = 1}^{3} {\hat{y}}_{i} A_{i, 2} & = & -3 \\ 1 & = & {\hat{x}}_{3} & c_{3} - \sum_{i = 1}^{3} {\hat{y}}_{i} A_{i, 3} & = & 0 \end{matrix}

3.6: Relation Between Dual and Perturbed Upper Bound
3.6.a: General Notation
Given an one-to-one mapping from

β : {1, \dots, m} \to {1, \dots, n}

, a matrix

M \in R^{m \times n}

, and a column vector

v \in R^{n}

, we define the matrix

M_{β} \in R^{m \times m}

, the column vector

M_{k} \in R^{m}

, and the column vector

v_{β} \in R^{m}

, by

M_{β} = (\begin{matrix} M_{1, β (1)} & \dots & M_{1, β (m)} \\ ⋮ & ⋱ & ⋮ \\ M_{m, β (1)} & \dots & M_{m, β (m)} \end{matrix}), M_{k} = (\begin{matrix} M_{1, k} \\ ⋮ \\ M_{m, k} \end{matrix}), v_{β} = {(\begin{matrix} v_{β (1)} \\ ⋮ \\ v_{β (m)} \end{matrix})}^{T}

We use the notation

j \in β

to denote the condition that

j

is in the range of

β

; i.e., there exists an

i

such that

j = β (i)

.

3.6.b: Problem Notation
We are given a matrix

A \in R^{m \times n}

, a vector

b \in R^{m}

, a vector

Δ b \in R^{m}

, and a vector

c \in R^{n}

. The corresponding primal problem: 3.2.a is

\begin{matrix} maximize & c^{T} x with respect to x \in R_{+}^{n} \\ subject to & A x \leq (b + Δ b) \end{matrix}

We introduce the objective

z \in R

, the slack variables

s \in R^{m}

, and the combined variable vector

u \in R^{n + m}

defined by

\begin{matrix} z & = & c^{T} x \\ s & = & b + Δ b - A x \\ u & = & (\begin{matrix} x \\ s \end{matrix}) \end{matrix}

We define the tableau representation by

T \in R^{m \times (n + m)}

and

d \in R^{n + m}

T = [A | I], d^{T} = (c^{T}, 0, \dots, 0)

where

I \in R^{m \times m}

is the identity matrix. It follows that our primal problem is to maximize

z

with respect to

u \in R_{+}^{n + m}

and subject to

\begin{matrix} T u & = & b + Δ b \\ d^{T} u & - & z & = & 0 \end{matrix}

We use

B : {1, \dots, m} \to {1, \dots, n + m}

to denote the current basis and define the non-basis mapping

N : {1, \dots, n} \to {1, \dots, n + m}

N (k) = \min {j | \begin{matrix} j ∉ B \\ j \neq N (ℓ) for ℓ = 1, \dots, k - 1 \end{matrix}}

3.6.c: Simplex Representation
The tableau representation our primal problem is to maximize

z

with respect to

u \in R_{+}^{n + m}

and subject to

\begin{matrix} T_{B} u_{B} & + & T_{N} u_{N} & = & b + Δ b \\ d_{B}^{T} u_{B} & + & d_{N}^{T} u_{N} & - & z & = & 0 \end{matrix}

The simplex method solves for the basic variables

u_{B}

in terms of the other variables:

\begin{matrix} u_{B} & + & T_{B}^{-1} T_{N} u_{N} & = & T_{B}^{-1} (b + Δ b) \\ d_{B}^{T} u_{B} & + & d_{N}^{T} u_{N} & - & z & = & 0 \end{matrix}

It also zeros out the coefficient of

u_{B}

in the row in terms of the other variables. The is done by multiplying the first

m

rows by

d_{B}^{T}

and subtracting the result from the bottom row.

\begin{matrix} u_{B} & + & T_{B}^{-1} T_{N} u_{N} & = & T_{B}^{-1} (b + Δ b) \\ + & (d_{N}^{T} - d_{B}^{T} T_{B}^{-1} T_{N}) u_{N} & - & z & = & - d_{B}^{T} T_{B}^{-1} (b + Δ b) \end{matrix} (⋆)

We use

\hat{z} (Δ b)

to represent the value of

z

corresponding to the basic solution (

u_{N} = 0

) for

b + Δ b

. It follows that

\hat{z} (Δ b) - \hat{z} (0) = d_{B}^{T} T_{B}^{-1} Δ b

3.6.d: Basic Solution
We define

\hat{x} (Δ b) \in R^{n}

and

\hat{s} (Δ b) \in R^{m}

to be the basic solution corresponding to the tableau

(⋆)

; i.e.,

\begin{matrix} \hat{x} (Δ b)_{j} & = & {\begin{matrix} [T_{B}^{-1} (b + Δ b)]_{i} & if j = B (i) for some i \\ 0 & if j \in N \end{matrix} \\ \hat{s} (Δ b)_{i} & = & {\begin{matrix} [T_{B}^{-1} (b + Δ b)]_{k} & if n + i = B (k) for some k \\ 0 & if n + i \in N \end{matrix} \end{matrix}

We define

\hat{c} \in R^{n}

and

\hat{y} \in R^{m}

to be the objective coefficients of

x

and minus the objective coefficient of

s

in the tableau

(⋆)

; i.e.,

\begin{matrix} {\hat{c}}_{j} & = & {\begin{matrix} (d_{N}^{T} - d_{B}^{T} T_{B}^{-1} T_{N})_{k} & if j = N (k) for some k \\ 0 & if j \in B \end{matrix} \\ {\hat{y}}_{i} & = & {\begin{matrix} (d_{B}^{T} T_{B}^{-1} T_{N} - d_{N}^{T})_{k} & if n + i = N (k) for some k \\ 0 & if n + i \in B \end{matrix} \end{matrix}

3.6.e: Optimality Lemma
If

\hat{c} \leq 0

\hat{x} (Δ b) \geq 0

\hat{s} (Δ b) \geq 0

, and

\hat{y} \geq 0

, then

\hat{x} (Δ b)

solves the primal problem (that corresponds to

Δ b

) and

\hat{y}

solves the dual problem.

3.6.e.a: Proof
It follows from the definitions above that for

j = 1, \dots, n

{\hat{c}}_{j} \hat{x} (Δ b)_{j} = 0

. We also have for

i = 1, \dots, m

{\hat{y}}_{i} \hat{s} (Δ b)_{i} = 0

. The conclusion of the lemma now follow from the Duality Theorem: 3.3 .

3.6.f: Representation Lemma

{\hat{y}}^{T} = d_{B}^{T} T_{B}^{-1}

3.6.f.a: Proof
If

i + n = N (k)

for some

k

\begin{matrix} y_{i} & = & (d_{B}^{T} T_{B}^{-1} T_{N} - d_{N}^{T})_{k} \\ = & d_{B}^{T} T_{B}^{-1} T_{N (k)} - d_{N (k)} \\ = & d_{B}^{T} T_{B}^{-1} I_{i} \\ = & (d_{B}^{T} T_{B}^{-1})_{i} \end{matrix}

i + n = B (k)

for some

k

\begin{matrix} T_{B}^{-1} T_{B} & = & I \\ T_{B}^{-1} I_{i} & = & I_{k} \\ d_{B}^{T} T_{B}^{-1} I_{i} & = & d_{B}^{T} I_{k} \\ (d_{B}^{T} T_{B}^{-1})_{i} & = & d_{B (k)} = 0 = y_{i} \end{matrix}

3.6.g: Perturbation Theorem
We now assume that the basic solution corresponding to

(⋆)

with

Δ b = 0

is feasible, optimal, and non-degenerate. It follows that there is an

δ > 0

such that for all

| Δ b | < δ

\hat{x} (Δ b)

solves the primal problem (corresponding to

Δ b

\hat{y}

solves the dual problem, and

c^{T} \hat{x} (Δ b) - c^{T} \hat{x} (0) = {\hat{y}}^{T} Δ b

3.6.g.a: Proof
It follows form the definitions that

c^{T} \hat{x} (Δ b) - c^{T} \hat{x} (0) = \hat{z} (Δ b) - \hat{z} (0) = {\hat{y}}^{T} Δ b

Hence, it suffices to show that

\hat{x} (Δ b)

is optimal for the corresponding primal problem and

\hat{y}

is optimal for the dual. Using the Optimality Lemma: 3.6.e , it suffices to show that

\hat{c} \leq 0

\hat{x} (Δ b) \geq 0

\hat{s} (Δ b) \geq 0

, and

\hat{y} \geq 0

The solution corresponding to

Δ b = 0

is non-degenerate implies that there is a

ɛ > 0

such that

T_{B}^{-1} b \geq ɛ

; i.e., all its components are greater than or equal

ɛ

Suppose that there is an index

j

such that

{\hat{c}}_{j} > 0

or an index

i

with

{\hat{y}}_{i} < 0

. It would follow from

T_{B}^{-1} b > 0

that one iteration of the Simplex algorithm, applied to

(⋆)

with

Δ b = 0

, would obtain a basic feasible solution that had a higher objective value than the value corresponding to

\hat{x} (0)

. This contradicts the assumption of the theorem and we conclude that

\hat{c} \leq 0

and

\hat{y} \geq 0

The mapping

\hat{x} (0) \geq ɛ

and

\hat{s} (0) \geq ɛ

. Hence there is a

δ > 0

such that for

| Δ b | < δ

\hat{x} (Δ) \geq 0

and

\hat{s} (0) \geq 0

. This completes the proof of the theorem.

3.7: Duality with Equality and Inequality Constraints
3.7.a: General Notation
We are given a certain number of equality constraints

m (=)

, a number of inequality constraints

m (\leq)

, a number of free variables

n (\pm)

, and a number of non-negative variables

n (+)

. These constraints are define by

A_{=, \pm} \in R^{m (=) \times n (\pm)}

A_{=, +} \in R^{m (=) \times n (+)}

A_{\leq, \pm} \in R^{m (\leq) \times n (\pm)}

A_{\leq, +} \in R^{m (\leq) \times n (+)}

b_{=} \in R^{m (=)}

b_{\leq} \in R^{m (\leq)}

c_{\pm} \in R^{n (\pm)}

, and

c_{+} \in R^{n (+)}

.

3.7.b: Primal Problem

\begin{matrix} maximize & c_{\pm}^{T} x_{\pm} + c_{+}^{T} x_{+} with respect to x_{\pm} \in R^{n (\pm)}, x_{+} \in R_{+}^{n (+)} \\ subject to & A_{=, \pm} x_{\pm} + A_{=, +} x_{+} = b_{=} \\ A_{\leq, \pm} x_{\pm} + A_{\leq, +} x_{+} \leq b_{\leq} \end{matrix}

3.7.c: Standard Primal Form
Define

x_{\pm} = x_{\pm}^{+} - x_{\pm}^{-}

where

x_{\pm}^{+} \in R_{+}^{n (\pm)}

, and

x_{\pm}^{-} \in R_{+}^{n (\pm)}

. The primal problem above is equivalent to

\begin{matrix} maximize & c_{\pm}^{T} x_{\pm}^{+} - c_{\pm}^{T} x_{\pm}^{-} + c_{+}^{T} x_{+} w . r . t . x_{\pm}^{+} \in R_{+}^{n (\pm)}, x_{\pm}^{-} \in R_{+}^{n (\pm)}, x_{+} \in R_{+}^{n (+)} \\ subject to & + A_{=, \pm} x_{\pm}^{+} - A_{=, \pm} x_{\pm}^{-} + A_{=, +} x_{+} \leq + b_{=} \\ - A_{=, \pm} x_{\pm}^{+} + A_{=, \pm} x_{\pm}^{-} - A_{=, +} x_{+} \leq - b_{=} \\ + A_{\leq, \pm} x_{\pm}^{+} - A_{\leq, \pm} x_{\pm}^{-} + A_{\leq, +} x_{+} \leq + b_{\leq} \end{matrix}

3.7.d: Standard Dual Form

\begin{matrix} maximize & b_{=}^{T} y_{=}^{+} - b_{=}^{T} y_{=}^{-} + b_{\leq}^{T} y_{\leq} w . r . t . y_{=}^{+} \in R_{+}^{m (=)}, y_{=}^{-} \in R_{+}^{m (=)}, y_{\leq} \in R_{+}^{m (\leq)} \\ subject to & + A_{=, \pm}^{T} y_{=}^{+} - A_{=, \pm}^{T} y_{=}^{-} + A_{\leq, \pm}^{T} y_{\leq} \geq + c_{\pm} \\ - A_{=, \pm}^{T} y_{=}^{+} + A_{=, \pm}^{T} y_{=}^{-} - A_{\leq, \pm}^{T} y_{\leq} \geq - c_{\pm} \\ + A_{=, +}^{T} y_{=}^{+} - A_{=, +}^{T} y_{=}^{-} + A_{\leq, +}^{T} y_{\leq} \geq + c_{+} \end{matrix}

3.7.e: Dual Problem
Define

y_{=} = y_{=}^{+} - y_{=}^{-}

where

y_{=} \in R^{m (=)}

. The primal problem above is equivalent to

\begin{matrix} minimize & b_{=}^{T} y_{=} + b_{\leq}^{T} y_{\leq} with respect to y_{=} \in R^{m (=)}, y_{\leq} \in R_{+}^{m (\leq)} \\ subject to & A_{=, \pm}^{T} y_{=} + A_{\leq, \pm}^{T} y_{\leq} = c_{\pm} \\ A_{=, +}^{T} y_{=} + A_{\leq, +}^{T} y_{\leq} \geq c_{+} \end{matrix}

4: Zero Sum Matrix Games
4.a: Introduction
We are given two players

X

and

Y

and a matrix

A \in R^{m \times n}

If player

X

makes choice

j \in {1, \dots, n}

and player

Y

makes choice

i \in {1, \dots, m}

, player

X

wins (player

Y

looses)

A_{i, j}

units.

4.b: Strategy
We use the notation

P (n)

to denote the set of probability measures on the indices

{1, \dots, n}

; i.e.,

P (n) = {x \in R_{+}^{n} such that \sum_{i = 1}^{n} x_{i} = 1}

If player

X

adopts a strategy

x \in P (n)

and player adopts a strategy

y \in P (m)

, the expected winnings for player

X

(loosing for player

Y

) is

y^{T} A x = \sum_{i = 1}^{m} \sum_{j = 1}^{n} y_{i} A_{i, j} x_{j}

4.c: Lemma
For each

\hat{x} \in R^{n}

and

\hat{y} \in R^{m}

\begin{matrix} \inf_{y \in P (m)} y^{T} A \hat{x} & = & \min_{i = 1, \dots, m} \sum_{j}^{n} A_{i, j} {\hat{x}}_{j} \\ \sup_{x \in P (n)} {\hat{y}}^{T} A x & = & \max_{j = 1, \dots, n} \sum_{i}^{m} {\hat{y}}_{i} A_{i, j} \end{matrix}

4.c.a: Proof
Let

k \in {1, \dots, m}

be an index such that

\begin{matrix} \min_{i = 1, \dots, m} \sum_{j}^{m} A_{i, j} {\hat{x}}_{j} & = & \sum_{j}^{n} A_{k, j} {\hat{x}}_{j} \end{matrix}

Choosing

y \in P (m)

y_{k} = 1

and for

i \neq k

y_{i} = 0

, we conclude that

\begin{matrix} \inf_{y \in P (m)} y^{T} A \hat{x} & \leq & \sum_{j}^{n} A_{k, j} {\hat{x}}_{j} \\ \leq & \min_{i = 1, \dots, m} \sum_{j}^{n} A_{i, j} {\hat{x}}_{j} \end{matrix}

But it also follows that

\begin{matrix} y^{T} A \hat{x} & = & \sum_{i = 1}^{m} y_{i} \sum_{j = 1}^{n} A_{i, j} {\hat{x}}_{j} \\ \geq & \sum_{i = 1}^{m} y_{i} \sum_{j = 1}^{n} A_{k, j} {\hat{x}}_{j} \\ \geq & (\sum_{j = 1}^{n} A_{k, j} {\hat{x}}_{j}) \sum_{i = 1}^{m} y_{i} \\ \geq & \sum_{j = 1}^{n} A_{k, j} {\hat{x}}_{j} \\ \inf_{y \in P (m)} y^{T} A \hat{x} & \geq & \min_{i = 1, \dots, m} \sum_{j}^{n} A_{i, j} {\hat{x}}_{j} \end{matrix}

The completes the proof of the first equation in the lemma. The proof of the second equation is similar.

4.d: Primal Problem
Suppose that player

X

chooses her strategy first and then player

Y

gets to choose his strategy. It follows that player

X

should solve the problem the problem

maximize [\min_{y \in P (m)} y^{T} A x] w . r . t . x \in P (n)

This problem is equivalent to

\begin{matrix} maximize & z w . r . t . z \in R, x \in R_{+}^{n} \\ subject to & 1 = \sum_{j = 1}^{n} x_{j} \\ z \leq \sum_{j = 1}^{n} A_{i, j} x_{j} (i = 1, \dots, m) \end{matrix}

We use the notation

0_{m \times n}

(

1_{m \times n}

) to denote the

m \times n

matrix of zeros (ones). The problem above can be written as

\begin{matrix} maximize & (1, 0_{1 \times n}) (\begin{matrix} z \\ x \end{matrix}) w . r . t . z \in R, x \in R_{+}^{n} \\ subject to & (\begin{matrix} 0 & 1_{1 \times n} \\ 1_{m \times 1} & - A \end{matrix}) (\begin{matrix} z \\ x \end{matrix}) \begin{matrix} = & 1 \\ \leq & 0_{m \times 1} \end{matrix} \end{matrix}

4.e: Dual Problem
The dual of the problem above is

\begin{matrix} minimize & (1, 0_{1 \times m}) (\begin{matrix} w \\ y \end{matrix}) w . r . t . w \in R, y \in R_{+}^{m} \\ subject to & (\begin{matrix} 0 & 1_{1 \times m} \\ 1_{n \times 1} & - A^{T} \end{matrix}) (\begin{matrix} w \\ y \end{matrix}) \begin{matrix} = & 1 \\ \geq & 0_{n \times 1} \end{matrix} \end{matrix}

This problem is equivalent to

\begin{matrix} minimize & w w . r . t . w \in R, y \in R_{+}^{m} \\ subject to & 1 = \sum_{i = 1}^{m} y_{i} \\ w \geq \sum_{i = 1}^{m} y_{i} A_{i, j} (j = 1, \dots, n) \end{matrix}

Which is also equivalent to

minimize [\max_{x \in P (n)} y^{T} A x] w . r . t . y \in P (m)

This is the problem that player

Y

should solve if

Y

chooses his strategy first and then player

X

gets to choose here strategy.

4.f: Theorem
Duality theory leads us to the conclusion that if both players know linear programming, it does not matter which one goes first; i.e.,

\min_{y \in P (m)} [\max_{x \in P (n)} y^{T} A x] = \max_{x \in P (n)} [\min_{y \in P (m)} y^{T} A x]

4.g: Example
As an example of a matrix game, we solve for one of the optimal strategies in Problem15.1: 4.1 . As another example, we verify the optimal strategy for the game of Rock Paper Scissor: 4.2 .

4.1: Problem 15.1 of the Text
4.1.a: The Game
We consider a game where players

A

and

B

simultaneously choose a coin (which they own) of value

x

y

where

x \neq y

. If the chosen coins have the same value, player

A

gets to keep them. Otherwise player

B

gets to keep them. We use

P (b, a)

to denote the payoff from player

B

to player

A

when

A

chooses

a

and

B

chooses

b

. The following table gives the payoff from player

B

to player

A

for each of the possible cases:

\begin{matrix} P (x, x) = x & P (x, y) = - y \\ P (y, x) = - x & P (y, y) = y \end{matrix}

4.1.b: Strategy for A
The optimal strategy for

A

(if the strategy is discovered by

B

) solves the problem

maximize [\inf_{b \in P (2)} b_{1} x a_{1} - b_{1} y a_{2} - b_{2} x a_{1} + b_{2} y a_{2}] w . r . t . a \in P (2)

Applying the inf: 4.c part of the lemma for matrix games, we obtain the equivalent problem:

maximize [\min (x a_{1} - y a_{2}, - x a_{1} + y a_{2})] w . r . t . a \in P (2)

Because there are only two components of

a

, we can use

a_{2} = 1 - a_{1}

to replace

a_{2}

in the objective and obtain the equivalent problem

maximize [\min ((x + y) a_{1} - y, y - (x + y) a_{1})] w . r . t . a_{1} \in [0, 1]

The function

f (a_{1}) = \min ((x + y) a_{1} - y, y - (x + y) a_{1})

has constant derivative on the interval

[0, 1]

except for three points. Thus, there are three possible solutions for the maximum:

a_{1} = 0

a_{1} = 1

or the solution of the following equation

\begin{matrix} (x + y) a_{1} - y & = & y - (x + y) a_{1} \\ a_{1} & = & y / (x + y) \end{matrix}

The corresponding objective function values are

\begin{matrix} 0 & = & a_{1} & \Rightarrow & - y & = & \min ((x + y) a_{1} - y, y - (x + y) a_{1}) \\ 1 & = & a_{1} & \Rightarrow & - x & = & \min ((x + y) a_{1} - y, y - (x + y) a_{1}) \\ y / (x + y) & = & a_{1} & \Rightarrow & 0 & = & \min ((x + y) a_{1} - y, y - (x + y) a_{1}) \end{matrix}

It follows (from the fact that

x > 0

and

y > 0

) that the optimal strategy for

A

is to choose

x

with probability

y / (x + y)

and to choose

y

with probability

x / (x + y)

. In this case, the expected payoff of the game is zero (provided that player

B

plays in an optimal fashion).

4.1.c: Strategy for B
The optimal strategy for

B

(if the strategy is discovered by

A

) solves the problem

minimize [\sup_{a \in P (2)} b_{1} x a_{1} - b_{1} y a_{2} - b_{2} x a_{1} + b_{2} y a_{2}] w . r . t . b \in P (2)

Applying the sup: 4.c part of the lemma for matrix games, we obtain the equivalent problem:

minimize [\max (b_{1} x - b_{2} x, - b_{1} y + b_{2} y)] w . r . t . b \in P (2)

Because there are only two components of

b

, we can use

b_{2} = 1 - b_{1}

to replace

b_{2}

in the objective and obtain the equivalent problem

minimize [\max (x (2 b_{1} - 1), (1 - 2 b_{1}) y)] w . r . t . b_{1} \in [0, 1]

The function

g (b_{1}) = \max (x (2 b_{1} - 1), (1 - 2 b_{1}) y)

has constant derivative on the interval

[0, 1]

except for three points. Thus, there are three possible solutions for the maximum:

b_{1} = 0

b_{1} = 1

or the solution of the following equation

\begin{matrix} x (2 b_{1} - 1) & = & (1 - 2 b_{1}) y \\ 2 (x + y) b_{1} & = & (x + y) \\ b_{1} & = & 1 / 2 \end{matrix}

The corresponding objective function values are

\begin{matrix} 0 & = & b_{1} & \Rightarrow & y & = & \max (x (2 b_{1} - 1), (1 - 2 b_{1}) y) \\ 1 & = & b_{1} & \Rightarrow & x & = & \max (x (2 b_{1} - 1), (1 - 2 b_{1}) y) \\ 1 / 2 & = & b_{1} & \Rightarrow & 0 & = & \max (x (2 b_{1} - 1), (1 - 2 b_{1}) y) \end{matrix}

It follows (from the fact that

x > 0

and

y > 0

) that the optimal strategy for

B

is to choose

x

with probability

1 / 2

and to choose

y

with probability

1 / 2

. In this case, the expected payoff of the game is zero (provided that player

A

plays in an optimal fashion).

4.2: Rock Paper Scissor as a Zero Sum Matrix Game
4.2.a: The Game
We consider a game where players

X

and

Y

simultaneously (and repeatedly) choose one of the following options: Rock, Paper, or Scissor. The following table gives the payoff from player

Y

to player

X

for each of the possible cases

			$X$
		Rock	Paper	Scissor
	Rock	0	-1	1
$Y$	Paper	1	0	-1
	Scissor	-1	1	0

Each player tries to guess the others strategy and thereby improve their expected outcomes.

4.2.b: Mathematical Formulation
We define the matrix

A = (\begin{matrix} 0 & -1 & 1 \\ 1 & 0 & -1 \\ -1 & 1 & 0 \end{matrix})

The optimal strategy for

X

(if the strategy is discovered by

Y

) solves the following problem

maximize [\min_{y \in P (3)} y^{T} A x] w . r . t x \in P (3)

The optimal strategy for

Y

(if the strategy is discovered by

X

) solves the following problem

minimize [\max_{x \in P (3)} y^{T} A x] w . r . t y \in P (3)

4.2.c: Optimal Strategy
We guess that the optimal strategy for player

X

and

Y

are

\hat{x} = (1, 1, 1) / 3

and

\hat{y} = (1, 1, 1) / 3

. In this case we also think that the corresponding payoff value is zero; i.e.,

0 = {\hat{y}}^{T} A \hat{x}

We can check this by checking that

\hat{x}

is feasible for the corresponding primal problem,

\hat{y}

is feasible for the corresponding dual problem, and the primal and dual have the same object value zero at these points.

4.2.d: Primal Problem
The matrix game primal problem: 4.d is

\begin{matrix} maximize & (1, 0, 0, 0) (\begin{matrix} z \\ x \end{matrix}) w . r . t . z \in R, x \in R_{+}^{3} \\ subject to & (\begin{matrix} 0 & 1 & 1 & 1 \\ 1 & 0 & 1 & -1 \\ 1 & -1 & 0 & 1 \\ 1 & 1 & -1 & 0 \end{matrix}) (\begin{matrix} z \\ x_{1} \\ x_{2} \\ x_{3} \end{matrix}) \begin{matrix} = & 1 \\ \leq & 0 \\ \leq & 0 \\ \leq & 0 \end{matrix} \end{matrix}

Using the argument value

(\hat{z}, {\hat{x}}^{T}) = (0, 1 / 3, 1 / 3, 1 / 3)

, the corresponding primal objective is zero and the primal feasibility conditions check out because

\begin{matrix} (\begin{matrix} 0 & 1 & 1 & 1 \\ 1 & 0 & 1 & -1 \\ 1 & -1 & 0 & 1 \\ 1 & 1 & -1 & 0 \end{matrix}) (\begin{matrix} 0 \\ 1 / 3 \\ 1 / 3 \\ 1 / 3 \end{matrix}) \begin{matrix} = & 1 \\ \leq & 0 \\ \leq & 0 \\ \leq & 0 \end{matrix} \end{matrix}

4.2.e: Dual Problem
The matrix game dual problem: 4.e is

\begin{matrix} minimize & (1, 0, 0, 0) (\begin{matrix} w \\ y \end{matrix}) w . r . t . w \in R, y \in R_{+}^{3} \\ subject to & (\begin{matrix} 0 & 1 & 1 & 1 \\ 1 & 0 & -1 & 1 \\ 1 & 1 & 0 & -1 \\ 1 & -1 & 1 & 0 \end{matrix}) (\begin{matrix} w \\ y_{1} \\ y_{2} \\ y_{3} \end{matrix}) \begin{matrix} = & 1 \\ \geq & 0 \\ \geq & 0 \\ \geq & 0 \end{matrix} \end{matrix}

Using the argument value

(\hat{w}, {\hat{y}}^{T}) = (0, 1 / 3, 1 / 3, 1 / 3)

, the corresponding dual objective is zero and the primal feasibility conditions check out because

\begin{matrix} (\begin{matrix} 0 & 1 & 1 & 1 \\ 1 & 0 & -1 & 1 \\ 1 & 1 & 0 & -1 \\ 1 & -1 & 1 & 0 \end{matrix}) (\begin{matrix} 0 \\ 1 / 3 \\ 1 / 3 \\ 1 / 3 \end{matrix}) \begin{matrix} = & 1 \\ \geq & 0 \\ \geq & 0 \\ \geq & 0 \end{matrix} \end{matrix}

5: Part II: Selected Applications
5.a: Contents

Product Manufacturing: 5.1
Toll Booth Scheduling: 5.2
Electronics Company: 5.3
A Forestry Example: 5.4

5.1: Product Manufacturing
5.1.a: Problem Statement
A product can be made in three sizes, large, medium, and small, which yield a net unit profit of $12, $10, and $9, respectively. A company has three centers where this product can be manufactured and these centers have a capacity of turning out 550, 750, and 275 units of the product per day, respectively, regardless of the size or combination of sizes involved.

Manufacturing this product requires cooling water and each unit of large, medium, and small sizes produced require 21, 17, and 9 gallons of water, respectively. The centers 1, 2, and 3 have 10,000, 7000, and 420 gallons of cooling water available per day, respectively. Market studies indicate that there is a market for 700, 900, and 340 units of the large, medium, and small sizes, respectively, per day. By company policy, the fraction (scheduled production) / (center's capacity) must be the same at all the centers. How many units of each of the sizes should be produced at the various centers in order to maximize the profit?

5.1.b: Mathematical Formulation

5.1.b.a: Notation

Variable	Meaning
$c$	center index, $c \in {1, 2, 3}$
$s$	size index, 1 = large, 2 = medium, 3 = small
$x_{3 (c - 1) + s}$	units per day of size s produced at center c
$z$	net company profit in dollars per day
$p_{c}$	percentage of production capacity scheduled at center $c$
$r_{s}$	rate of production in units per day for size $s$
$w_{c}$	units of cooling water used per day at center $c$
$α$	we use $α$ to denote $1 / 5.50$
$β$	we use $β$ to denote $1 / 7.50$
$γ$	we use $γ$ to denote $1 / 2.75$

5.1.b.b: Relations

\begin{matrix} x_{1} & x_{2} & x_{3} & x_{4} & x_{5} & x_{6} & x_{7} & x_{8} & x_{9} \\ z = & 12 & 10 & 9 & 12 & 10 & 9 & 12 & 10 & 9 \\ p_{1} - p_{2} = & α & α & α & - β & - β & - β \\ p_{1} - p_{3} = & α & α & α & - γ & - γ & - γ \\ p_{1} = & α & α & α \\ r_{1} = & 1 & 1 & 1 \\ r_{2} = & 1 & 1 & 1 \\ r_{3} = & 1 & 1 & 1 \\ w_{1} = & 21 & 17 & 9 \\ w_{2} = & 21 & 17 & 9 \\ w_{3} = & 21 & 17 & 9 \end{matrix}

5.1.b.c: Problem Submitted to Neos
Maximize

z

with respect to

x \in R_{+}^{3 \times 3}

and subject to the following constraints:

The fact that the centers have the same percentage of capacity production rates is represented by $p_{1} - p_{2} = 0, p_{1} - p_{3} = 0$
Given the constraints above, the fact that none of the centers can exceed capacity is represented by $p_{1} \leq 100$
The fact that each center has a limit on how much cooling water it can use per day is represented by $w_{1} \leq 10^{4}, w_{2} \leq 7 \times 10^{3}, w_{3} \leq 420$
The formula for the net profit $z$ is only valid if we stay below the market capacities; i.e., $r_{1} \leq 700, r_{2} \leq 900, r_{3} \leq 340$

5.1.b.d: Standard Form
We define the vector

c \in R^{9}

, the vector

b \in R^{11}

and the matrix

A \in R^{11 \times 9}

\begin{matrix} c & = & (12, 10, 9, 12, 10, 9, 12, 10, 9)^{T} \\ b & = & (0, 0, 0, 0, 100, 700, 900, 340, 10^{4}, 7 \times 10^{3}, 420) \\ A & = & (\begin{matrix} α & α & α & - β & - β & - β \\ - α & - α & - α & β & β & β \\ α & α & α & - γ & - γ & - γ \\ - α & - α & - α & γ & γ & γ \\ α & α & α \\ 1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1 \\ 21 & 17 & 9 \\ 21 & 17 & 9 \\ 21 & 17 & 9 \end{matrix}) \end{matrix}

It follows that

\hat{x} \in R_{+}^{9}

solves the problem submitted to Neos: 5.1.b.c if and only if it solves

\begin{matrix} maximize & c^{T} x & with respect to x \in R_{+}^{9} \\ subject to & A x \leq b \end{matrix}

Furthermore

\hat{y} \in R_{+}^{11}

solves the corresponding dual problem if and only if it solves

\begin{matrix} minimize & b^{T} y & with respect to y \in R_{+}^{11} \\ subject to & A^{T} y \geq b \end{matrix}

5.1.c: Solution




%%%%%%%%%%%%%%%%%%%% CLP Results %%%%%%%%%%%%%%%%%%%%





Load Avg: ( 4.0 , 4.0 , 3.93 )


Coin LP version 1.02.02, build Aug  3 2005


At line 4 NAME          Manufacture


At line 5 ROWS


At line 16 COLUMNS


At line 68 RHS


At line 78 ENDATA


Problem Manufacture has 9 rows, 9 columns and 33 elements


Model was imported from ./clp.mps in 0 seconds


Switching to line mode


Clp:Clp:Clp:Presolve 9 (0) rows, 9 (0) columns and 33 (0) elements


Perturbing problem by 0.001 % of 540.46 - largest nonzero change 5.09017e-05 (% 0.000152787) - largest zero change 0


0  Obj -0 Dual inf 71.8437 (9)


6  Obj 3067.28


Optimal - objective value 3067.27


Optimal objective 3067.272727 - 6 iterations time 0.002


Clp:


      0 p1p2                  0                    -90


      1 p1p3                  0                    156


      2 p1            16.969697                     -0


      3 r1            220.60606                     -0


      4 r2                    0                     -0


      5 r3            46.666667                     -0


      6 w1                 1960                     -0


      7 w2            2672.7273                     -0


      8 w3                  420              7.3030303


      0 x1            93.333333         -8.7777262e-16


      1 x2                    0                     -2


      2 x3                    0                     -3


      3 x4            127.27273          5.2000976e-16


      4 x5                    0                     -2


      5 x6                    0                     -3


      6 x7                    0             -84.636364


      7 x8                    0             -57.424242


      8 x9            46.666667         -1.9160035e-15


Clp:





%%%%%%%%%%%%%%%%%%%% CLP Results %%%%%%%%%%%%%%%%%%%%

We use the notation

\hat{y} = ({\hat{z}}_{1}^{+}, z