Linear–quadratic regulator

The settings of a (regulating) controller governing either a machine or process (like an airplane or chemical reactor) are found by using a mathematical algorithm that minimizes a cost function with weighting factors supplied by the operator. The cost function is often defined as a sum of the deviations of key measurements, like altitude or process temperature, from their desired values. The algorithm thus finds those controller settings that minimize undesired deviations. The magnitude of the control action itself may also be included in the cost function.

The LQR algorithm reduces the amount of work done by the control systems engineer to optimize the controller. However, the engineer still needs to specify the cost function parameters, and compare the results with the specified design goals. Often this means that controller construction will be an iterative process in which the engineer judges the "optimal" controllers produced through simulation and then adjusts the parameters to produce a controller more consistent with design goals.

The LQR algorithm is essentially an automated way of finding an appropriate state-feedback controller. As such, it is not uncommon for control engineers to prefer alternative methods, like full state feedback, also known as pole placement, in which there is a clearer relationship between controller parameters and controller behavior. Difficulty in finding the right weighting factors limits the application of the LQR based controller synthesis.

For a continuous-time linear system, defined on ${\displaystyle t\in [t_{0},t_{1}]}$ , described by:

${\displaystyle {\dot {x}}=Ax+Bu}$ 

where ${\displaystyle x\in \mathbb {R} ^{n}}$  (that is, ${\displaystyle x}$  is an ${\displaystyle n}$ -dimensional real-valued vector) is the state of the system and ${\displaystyle u\in \mathbb {R} ^{m}}$  is the control input. Given a quadratic cost function for the system, defined as:

${\displaystyle J=x^{T}(t_{1})F(t_{1})x(t_{1})+\int \limits _{t_{0}}^{t_{1}}\left(x^{T}Qx+u^{T}Ru+2x^{T}Nu\right)dt}$ 

where ${\displaystyle F}$  is the initial cost matrix, ${\displaystyle Q}$  is the state cost matrix, ${\displaystyle R}$  is the control cost matrix, and ${\displaystyle N}$  is the cross-term (control and state) cost matrix, the feedback control law that minimizes the value of the cost is:

${\displaystyle u=-Kx\,}$ 

where ${\displaystyle K}$  is given by:

${\displaystyle K=R^{-1}(B^{T}P(t)+N^{T})\,}$ 

and ${\displaystyle P}$  is found by solving the continuous time Riccati differential equation:

${\displaystyle A^{T}P(t)+P(t)A-(P(t)B+N)R^{-1}(B^{T}P(t)+N^{T})+Q=-{\dot {P}}(t)\,}$ 

with the boundary condition:

${\displaystyle P(t_{1})=F(t_{1}).}$ 

The first order conditions for J_min are:

1) State equation

${\displaystyle {\dot {x}}=Ax+Bu}$ 

2) Co-state equation

${\displaystyle -{\dot {\lambda }}=Qx+Nu+A^{T}\lambda }$ 

3) Stationary equation

${\displaystyle 0=Ru+N^{T}x+B^{T}\lambda }$ 

4) Boundary conditions

${\displaystyle x(t_{0})=x_{0}}$ 

and ${\displaystyle \lambda (t_{1})=F(t_{1})x(t_{1})}$ 

For a continuous-time linear system described by:

${\displaystyle {\dot {x}}=Ax+Bu}$ 

with a cost function defined as:

${\displaystyle J=\int _{0}^{\infty }\left(x^{T}Qx+u^{T}Ru+2x^{T}Nu\right)dt}$ 

the feedback control law that minimizes the value of the cost is:

${\displaystyle u=-Kx\,}$ 

where ${\displaystyle K}$  is given by:

${\displaystyle K=R^{-1}(B^{T}P+N^{T})\,}$ 

and ${\displaystyle P}$  is found by solving the continuous time algebraic Riccati equation:

${\displaystyle A^{T}P+PA-(PB+N)R^{-1}(B^{T}P+N^{T})+Q=0\,}$ 

This can be also written as:

${\displaystyle {\mathcal {A}}^{T}P+P{\mathcal {A}}-PBR^{-1}B^{T}P+{\mathcal {Q}}=0\,}$ 

with

${\displaystyle {\mathcal {A}}=A-BR^{-1}N^{T}\qquad {\mathcal {Q}}=Q-NR^{-1}N^{T}\,}$ 

For a discrete-time linear system described by: ^[3]

${\displaystyle x_{k+1}=Ax_{k}+Bu_{k}\,}$ 

with a performance index defined as:

${\displaystyle J=x_{H_{p}}^{T}Q_{H_{p}}x_{H_{p}}+\sum \limits _{k=0}^{H_{p}-1}\left(x_{k}^{T}Qx_{k}+u_{k}^{T}Ru_{k}+2x_{k}^{T}Nu_{k}\right)}$ , where ${\displaystyle H_{p}}$  is the time horizon

the optimal control sequence minimizing the performance index is given by:

${\displaystyle u_{k}=-F_{k}x_{k}\,}$ 

where:

${\displaystyle F_{k}=(R+B^{T}P_{k+1}B)^{-1}(B^{T}P_{k+1}A+N^{T})\,}$ 

and ${\displaystyle P_{k}}$  is found iteratively backwards in time by the dynamic Riccati equation:

${\displaystyle P_{k-1}=A^{T}P_{k}A-(A^{T}P_{k}B+N)\left(R+B^{T}P_{k}B\right)^{-1}(B^{T}P_{k}A+N^{T})+Q}$ 

from terminal condition ${\displaystyle P_{H_{p}}=Q_{H_{p}}}$ .^[4] Note that ${\displaystyle u_{H_{p}}}$  is not defined, since ${\displaystyle x}$  is driven to its final state ${\displaystyle x_{H_{p}}}$  by ${\displaystyle Ax_{H_{p}-1}+Bu_{H_{p}-1}}$ .

For a discrete-time linear system described by:

${\displaystyle x_{k+1}=Ax_{k}+Bu_{k}\,}$ 

with a performance index defined as:

${\displaystyle J=\sum \limits _{k=0}^{\infty }\left(x_{k}^{T}Qx_{k}+u_{k}^{T}Ru_{k}+2x_{k}^{T}Nu_{k}\right)}$ 

the optimal control sequence minimizing the performance index is given by:

${\displaystyle u_{k}=-Fx_{k}\,}$ 

where:

${\displaystyle F=(R+B^{T}PB)^{-1}(B^{T}PA+N^{T})\,}$ 

and ${\displaystyle P}$  is the unique positive definite solution to the discrete time algebraic Riccati equation (DARE):

${\displaystyle P=A^{T}PA-(A^{T}PB+N)\left(R+B^{T}PB\right)^{-1}(B^{T}PA+N^{T})+Q}$ .

This can be also written as:

${\displaystyle P={\mathcal {A}}^{T}P{\mathcal {A}}-{\mathcal {A}}^{T}PB\left(R+B^{T}PB\right)^{-1}B^{T}P{\mathcal {A}}+{\mathcal {Q}}}$ 

with:

${\displaystyle {\mathcal {A}}=A-BR^{-1}N^{T}\qquad {\mathcal {Q}}=Q-NR^{-1}N^{T}}$ .

Note that one way to solve the algebraic Riccati equation is by iterating the dynamic Riccati equation of the finite-horizon case until it converges.

In practice, not all values of ${\displaystyle x_{k},u_{k}}$  may be allowed. One common constraint is the linear one:

${\displaystyle C\mathbf {x} +D\mathbf {u} \leq \mathbf {e} .}$ 

The finite horizon version of this is a convex optimization problem, and so the problem is often solved repeatedly with a receding horizon. This is a form of model predictive control.^[5][6]

If the state equation is quadratic then the problem is known as the quadratic-quadratic regulator (QQR). The Al'Brekht algorithm can be applied to reduce this problem to one that can be solved efficiently using tensor based linear solvers.^[7]

If the state equation is polynomial then the problem is known as the polynomial-quadratic regulator (PQR). Again, the Al'Brekht algorithm can be applied to reduce this problem to a large linear one which can be solved with a generalization of the Bartels-Stewart algorithm; this is feasible provided that the degree of the polynomial is not too high.^[8]

Model predictive control and linear-quadratic regulators are two types of optimal control methods that have distinct approaches for setting the optimization costs. In particular, when the LQR is run repeatedly with a receding horizon, it becomes a form of model predictive control (MPC). In general, however, MPC does not rely on any assumptions regarding linearity of the system.

• Kwakernaak, Huibert; Sivan, Raphael (1972). Linear Optimal Control Systems (1st ed.). Wiley-Interscience. ISBN 0-471-51110-2.
• Sontag, Eduardo (1998). Mathematical Control Theory: Deterministic Finite Dimensional Systems (2nd ed.). Springer. ISBN 0-387-98489-5.

• MATLAB function for Linear Quadratic Regulator design
• Mathematica function for Linear Quadratic Regulator design