Minimum Effort Servomechanism:

Suppose I can influence the evolution of a vector of n state variables x(t) = (x1(t), ..., xn(t))^T with a vector of control variables u(t) = (u1(t), ..., ur(t))^T  (here the symbol "^T" represents the transpose of a vector) as given by:

dx / dt = A * x(t) + B * u(t)

where A is an n x n matrix and B is an n x r matrix.

At each instance of time t, I incur a cost that depends on the values of the state variables and control variables at that time. I want to minimize the total costs over a relevant planning interval of time. Formally, the problem is as follows:

Given symmetric, nonnegative, semidefinite matrices D, (nxn), M, (nxn), and R, (nxr), and a planning interval [t₀, t], choose a piecewise smooth function u from [t₀, t] to R^r so as to minimize the sum of the terminal cost and running cost given by:

min J[u] = x(t)^T*D*x(t) + (x(t)^T*M*x(t) + x(t)^T*R*u(t) + u(t)^T*R^T*x(t) + u(t)^T*u(t) ) dt

dx / dt = A * x(t) + B * u(t)

x(t₀) = x0

Because the problem above has no constraint on x(t), it is a free final state problem, with a feedback optimal control.

With a fixed final state constraint of the form x(t) = xt, the optimal control will be an open-loop control.

Finite Horizon Time Problems

Matrix Ricatti Equation:

With a finite planning horizon, t < , linear-quadratic optimal control problems are solved by finding the symmetric n x n matrix K = K(t) that satisfies the Matrix Ricatti Equation (MRE):

dK / dt + A^T * K + K * A - (K * B + R) * (B^T* K + R^T) + M = 0,   K(t) = D

Optimal Closed Loop (Feedback) Control:

If K(t) solves MRE, then the optimal closed loop control is:

u*(t) = - (B^T * K + R^T) * x(t)

Evolution of the Optimal State Variables:

dx(t) / dt = A * x(t) - B * (B^T * K + R^T) * x(t)

          = ( A - B * (B^T * K + R^T) ) * x(t)

Solving these differential equations with the initial conditions x(t₀) = x0 yields x = (x1, ..., xn)^T as a function of time and x0.

Example 1 (Pinch 167):

Consider a system with two state variables, and A = 0, B = I, D = I, M = 0, and R = 0, where I is the identity matrix. Then

dx1 / dt = u1(t)

dx2 / dt = u2(t)

min J[u] = x1²(3) + x2²(3) + (u1²(s) + u2²(s) ) ds

      1. MRE:

      2. Optimal Closed Loop Control:

u*(t) = - B^T * K * x(t)

      3. MRE Differential Equations:

      4. Integrating the differential equations:

The time paths of the state variables, x(t), and the control variables, u(t), are shown in the diagram below.

Infinite Time Horizon Problems

If the planning horizon extends to infinity, then K(t) must remain finite as t -> , requiring dK / dt = 0. To find the equilibrium K(t) = K^e, solve the Algebraic (Steady-State) Ricatti Equation (ARE):

Algebraic (Steady-State) Ricatti Equation (ARE):

A^T * K + K * A - (K * B + R) * (B^T* K + R^T) + M = 0

Optimal Feedback Control Law:

u*(x(t)) = - (B^T * K + R^T) * x*(t)

Minimum Cost Solution:

J[u*] = x(0)^T * K * x(0)

Example 2 (Hocking 124-25):

Consider the Rocket-Car problem with costs that depend both on position, x1(t), and energy expended, u(t).

dx1 / dt = x2(t)

dx2 / dt = u(t)

min J[u] = (4 * x1(t) * x1(t) + u(t) * u(t) ) dt

with t₀ = 0, and

      1. MRE Differential Equations:

      2. ARE Solution:

k3 = 2, k1 = +/- 4, k2 = +/- 2

      For K to be a positive definite matrix:

      3. Optimal Feedback Control Law:

u*(x(t)) = - (B^T * K) * x*(t) = (-2, -2) * (x1(t), x2(t))^T

      4. Optimal Evolution of State Variables:

dx / dt = A * x(t) + B * u*(x(t))

      5. Optimal Trajectories of the State Variables:

     The autonomous linear differential equation above has the solution:

x*(t) = expm(Â * t) * x(0),

      where expm(Â * t) is the matrix exponential, e^{Â * t}. The explicit solution is:

      6. Minimum Cost Solution with (x1(0), x2(0)) = ( 2, 1 ):

J[u*] = x(0)^T * K * x(0) = 26

In the diagrams below, you can see the trajectories of the state and control variables plotted as a function of time, and the phase diagram of x2(t) plotted against x1(t), with an initial position of (x1(0), x2(0)) = ( 2, 1 ).

You can change the rocket car's initial situation by entering new values for x1(0) and x2(0) in the form below the phase diagram. See how the trajectories shift, and how the phase diagram changes after you click Fire Rockets. Try x1(0) = -2.5 and x2(0) = -3, for example. When u(t) is positive, the rear rocket is firing; when u(t) is negative, the front rocket is firing. (Go to the Rocket-Car page for a full description of this optimal control problem).