Lecture 9 - Linear Quadratic Control

page=68 From LinSys: LinSys; Linear Quadratic Regulator - LQR

---

Does not work with inequality constraints, and is a recursive approach.

We want to minimize the LQR

$$\underset{z}{\min }\sum _{t=0}^{N-1}\frac{1}{2}{x}_{t+1}^{T}Q{x}_{t+1}+\frac{1}{2}{u}_t^{T}R{u}_t,s.t.\{\begin{array}{ll}{x}_{t+1}=A{x}_t+B{u}_t,& t=0,1,\dots ,N-1\\ z=(u_0,x_1,u_1,\dots ,u_{N-1},x_N{)}^T& \end{array}$$

Which yields (for finite horizon)

$$\begin{array}{rl}{K}_t& =R_t^{-1}{B}_t^T{P}_{t+1}(\mathbb{I}+B_t{R}_t^{-1}{B}_t^T{P}_{t+1}{)}^{-1}{A}_t,t=0,\dots ,N-1\\ P_t& =Q_t+A_t^T{P}_{t+1}(\mathbb{I}+B_t{R}_t^{-1}{B}_t^T{P}_{t+1}{)}^{-1}{A}_t,t=0,\dots ,N-1\\ P_N& =Q_N\end{array}$$

Where $u_t=-K_t{x}_t$, $Q_t⪰0$, and $R_t\succ 0$, and the equation for $P_t$ is the Algebraic Ricatti Equation (?)

---

Proof

page=61 Solving the KKT (Karush–Kuhn–Tucker) Conditions

Link to original

Only the stationary and the primal feasibility is valid here (no ineq)

Stationary:

$$\mathcal{L}(z,{\lambda }_1,{\lambda }_2,\dots {\lambda }_N)=\sum _{t=0}^{N-1}\frac{1}{2}{x}_{t+1}^{T}Q{x}_{t+1}+\frac{1}{2}{u}_t^{T}R{u}_t-\sum _{t=0}^{N-1}{\lambda }_{t+1}^T(x_{t+1}-A{x}_t-B{u}_t)$$

Solving the following set of equations

$$\begin{array}{rl}\frac{\partial \mathcal{L}}{\partial u_t}& =R{u}_t+B^T{\lambda }_{t+1}=0\\ \frac{\partial \mathcal{L}}{\partial x_t}& =\cdots =0\\ \frac{\partial \mathcal{L}}{\partial x_N}& =\cdots =0\end{array}$$

And we postulate (a “guess” in reality) that the Lagrange multipliers ${\lambda }_t$ depend linearly on the states $x_t$

$${\lambda }_t=P_t{x}_t,P_t=P_t^T\succ 0$$

Which gives $P_N=Q_N$.