Lecture 6&7 - Quadratic Programming

Resources: Chap. 16.1-2.4.5

---

A Quadratic Programming Problem on Standard Form looks like

Link to original

---

Active Set Method For Convex QP

Theorem 16.4 If $x^{\ast }$ satisfies the KKT (Karush–Kuhn–Tucker) Conditions and $G\ge 0$, then $x^{\ast }$ is a global solution

If we know which inequalities are active at the solution, QP can be solved as EQP

Examples

KKT for QP

Given

Link to original

and

Link to original

We first find the Lagrangian

$$\mathcal{L}(x,\lambda )=\frac{1}{2}{x}^{T}Gx+c^{T}x-\sum _{i\in \mathcal{E}\cup \mathcal{I}}{\lambda }_i(a_i^{T}x-b_i)$$

Stationary condition

$$G{x}^{\ast }+c-\sum _{i\in \mathcal{E}\cup \mathcal{I}}{\lambda }_i^{\ast }{a}_i=0$$

Primal feasibility

$$\begin{array}{rl}{a}_i^{T}x& =b_i,i\in \mathcal{E}\\ a_i^{T}x& \ge b_i,i\in \mathcal{I}\end{array}$$

Dual feasibility

$${\lambda }_i^{\ast }\ge 0,i\in \mathcal{I}$$

Complementary condition

$${\lambda }_i^{\ast }(a_i^T{x}^{\ast }-b_i)=0,i\in \mathcal{I}\cup \mathcal{E}$$

Alternatively we can formulate this as

$$\begin{array}{r}\mathcal{A}(x^{\ast })=\{i\in \mathcal{E}\cup \mathcal{I}|a_i^T{x}^{\ast }=b_i\}\end{array}$$

KKT:

$$G{x}^{\ast }+c-\sum _{i\in \mathcal{A}(x^{\ast })}{\lambda }_i^{\ast }{a}_i=0$$ $$\begin{array}{rl}{a}_i^T{x}^{\ast }& =b_i,i\in \mathcal{A}(x^{\ast })\\ a_i^T{x}^{\ast }& \ge b_i,i\in \mathcal{I}\setminus \mathcal{A}(x^{\ast })\end{array}$$

And

$${\lambda }_i^{\ast }\ge 0,i\in \mathcal{A}(x^{\ast })\cap \mathcal{I}$$