3.6 Stochnstie Regulntor nnd Tracking Problems
253
and lim P , = p,
3303
1.m
provided P, is so chosen that A,=ASP,
3304
is asymptotically stable. This means that the convergence of the scheme is assured if the initial estimate is suitably chosen. If the initial estimate is incorrectly selected, however, convergence to a different solution of the algebraic Riccati equation may occur, or no convergence at all may result. If A is asymptotically stable, a safe choice is P, = 0. If A is not asymptotically stable, the initial choice may present difficulties. Wonham and Cashman (1968), Man and Smith (1969), and Kleinman (1970h) give methods for selecting Po when A is not asymptotically stable. The main problem with this approach is 3292, which must be solved many times over. Although it is linear, the numerical effort may still he rather formidable, since the number of linear equations that must be solved a t each iteration increases rapidly with the dimension of the problem (for 11 = 15 this number is 120). In Section 1.11.3 several numerical approaches to solving 3292 are referenced. In the literature favorable experiences using the NewtonRaphson method lo solve Riccati equations has been reported with up to 15dimensional problems (Blackburn, 1968; Kleinman, 1968, 1970a).
3.6 S T O C H A S T I C L I N E A R O P T I M A L R E G U L A T O R AND T R A C K I N G P R O B L E M S 3.6.1 Regulator Problems with DisturbancesThe Stochastic Regulator Problem
In the preceding sections we discussed the deterministic linear optimal regulator problem. The solution of this problem allows us to tackle purely transient problems where a linear system has a disturbed initial state, and it is required to return the system to the zero state as quickly as possible while limiting the input amplitude. There exist practical problems that can be formulated in this manner, hut much more common are problems where there are disturbances that act uninterruptedly upon the system, and that tend to drive the state away from the zero state. The problem is then to design a feedback configuration through which initial offsets are reduced as quickly as possible, but which also counteracts the effects of disturbances as much as possible in the steadystate situation. The solution of this problem will bring us into a position to synthesize the controllers that have been asked for in
254
Optimal Linear State Feedback Control Systems
Chapter 2. For the time being we maintain the assumption that the complete state of the system can be accurately observed at each instant of time. The effect of the disturbances can be accounted for by suitably extending the system description. We consider systems described by
where ~ ( t is) the input variable, z(t) is the controlled variable, and v(t) representsdisturbances that act upon the system. We mathematically represent the disturbances as a stochastic process, which we model as the output of a linear system driven by white noise. Thus we assume that u(t) is given by
where w(t) is white noise. We furthermore assume that both z(to) and zd(t,) are stochastic variables. We combine the description of the system and the disturbances by defining an augmented state vector ?(I) = col [z(t),x,(t)], which from 3305, 3306, and 3307 can be seen to satisfy
In terms of the augmented state, the controlled variable is given by We note in passing that 3308 represents a system that is not completely controllable (from 11). We now turn our attention to the optimization criterion. I n the deterministic regulator problem, we considered the quadratic integral criterion
For a given input n(t), to 2 t 2 t,, and a given realization of the disturbances u(t), to 5 t 2 t l , this criterion is a measure for the deviations z(t) and ~ ( t ) from zero. A priori, however, this criterion cannot be evaluated because of the stochastic nature of the disturbances. We therefore average over all possible realizations of the disturbances and consider the criterion
3.6 Stochastic Regulator and Tracking Problems
255
In terms of the'augmented state E(t) = col [x(t),z,(t)], this criterion can be expressed as
+ T ( i ) ~ 2 ( i ) (dt ) + (
I
t 1 ( ) ,
3312
where
I t is obvious that the problem of minimizing 3312 for the system 3308 is nothing but a special case of the general problem of minimizing
for the system
x(t) = A(t)z(t)
+ B(t)u(t) + ~ ( t ) ,
3315
where ~ ( tis)white noise and where x(t,) is a stochastic variable. We refer to this problem as the stochastic linear optimal regulator problem: Definition 3.4. Consider the s ~ ~ s t edescribed m by the state differential eqnation 3316 x(t) = A(t)x(t) B(t)u(t) ~ ( 1 ) with initial state 3317 4 t 0 )= % and controlled variable 3318 ~ ( t= ) D(t)x(t).
+
+
61 3316 ~ ( tis)i~hitenoise iaith intensity V(t). The initiolstate xuis a stochastic uoriable, independent of the ililrite noise w , with
E{x,xOT}= Qo. Consider the criterion
< <
where R,(t) and R,(t) are positivedefinite sy~innetricmatrices for to t tl and Pl is ~tomregativedefirzitesynnnetric. Then the problem of deiern~iningfor each t , to t tl. the irtplrt u(t)as a$rnction of all informationfr.on~thepast such that the criterion is minbnized is called the stochastic linear optioral regrrlator problem. If all matrices in the problem fornndation are constant, we refer to it as the tinreirruariant stochastic linear optirnal regrrlator problem.
< <
The solution of this problem is discussed in Section 3.6.3.
256
Optimal Lincnr Stato Feedback Control Systems
Example 3.11. Stirred tmili In Example 1.37 (Section 1.11.4), we considered an extension of the model of the stirred tank where disturbances in the form of fluctuationsin the concentrations of the feeds are incorporated. The extended system model is given by
where ~ ( t is) white noise with intensity
Here the components of the state are, respectively, the incrementalvolume of fluid, the incremental concentration in the tank, the incremental concentration of the feed &, and the incremental concentration of the feed F3. Let us consider as previously the incremental outgoing flow and the incremental outgoing concentration as the components of the controlled variable. Thus we have
The stochastic optimal regulator problem now consists in determining the ) that a criterion of the form input ~ ( tsuch

~
~
~~~~~~~~~.
3.6 Stocltnstic Rcgulntor and Trncking Problems
257
is minimized. We select the weighting matrices R, and Rz in exactly tbe same manner as in Example 3.9 (Section 3.4.1), while we choose P, to be the zero matrix. 3.6.2
Stochastic Tracking Problems
We have introduced the stochastic optimal regulator problem by considering regulator problems with disturbances. Stochastic regulator problems also arise when we formulate stocl~asticoptiriial trackirrgprobteri~s.Consider the linear system 3325 x(t) = A(t)x(t) B(f)lr(f),
+
with the controlled variable z(t) = D(t)x(t).
3326
Suppose we wish the controlled variable to follow as closely as possible a refirerice uariable z,(t) which we model as the output of a linear differential system driven by white noise:
Here ~ ( t is) white noise with given intensity V(t). The system equations and the reference model equations can be combined by defining the augmented state Z(t) = col [x(t), x,(t)], which satisfies
In passing, we note that this system (just as that of 3308) is not completely controllable from 11. TO obtain an optiriial tracking system, we consider the criterion
where R,(t) and R,(t) are suitable weighting matrices. This criterion expresses that the controlled variable should be close to the reference variable, while the input amplitudes should be restricted. In fact, for R,(t) = WJt) and R,(t) = pW,,(t), the criterion reduces to J:[cO(i)
+ PC,~(~)] &
3331
where C,(t) and C,,(i) denote the mean square tracking error and the mean
258
Optirnnl Linear State Feedback Control S y ~ t c r n ~
square input, respectively, as defined in Chapter 2 (Section 2.3): C,(t) Here e(t) is the tracking error
= ~{rr'(t)
~~(t)rr(t)}.
e(t) = z(t)  z,(t). 3333 The weighting coefficient p must be adjusted so as to obtain the smallest possible mean square tracking error for a given value of the mean square input. The criterion 3330 can be expressed in terms of the augmented state x(t) as follows:
+
w ~ [ J ; > ~ o ~ d t ) z ( t ) u r ( t ) ~ i t ) ~ d],
3334
where 3335 i(t) = (D(t), D,(t))Z(t). Obviously, the problem of minimizing the criterion 3334 for the system 3329 is a special case of the stochastic linear optimal regulator problem of Definition 3.4. Without going into detail we point out that tracking problems with disturbances also can be converted into stochastic regulator problems by the state augmentation technique. In conclusion, we note that the approach of this subsection is entirely in line with the approach of Chapter 2, where we represented reference variables as having a variable part and a constant part. I n the present section we have set the constant part equal to zero; in Section 3.7.1 we deal with nonzero constant references.
Example 3.12. A~zgrclnrvelocity tracking system Consider the angular velocity control system of Example 3.3 (Section 3.3.1). Suppose we wish that the angular velocity, which is the controlled variable 5(t). follows as accurately as possible a reference variable t,(t), which may be described as exponentially correlated noise with time constant 0 and rms value o. Then we can model the reference process as (see Example 1.36, Section 1.1 1.4) 3336 t&) = Mt), where CJt) is the solution of
The white noise ~ ( t has ) intensity 2u2/0. Since the system state differential equation is

3.6 Stochwtic Regulator nnd Trucking Problems
259
the augmented state differential equation is given by
with .?(I) = col I&), C,(t)]. For the optimization criterion we choose
where p is a suitable weighting factor. This criterion can be rewritten as
where The problem of minimizing 3341 for the system described by 3339 and 3342 constitutes a stochastic optimal regulator problem. 3.6.3 Solution of the Stochastic Linear Optimal Regulator Problem
In Section 3.6.1 we formulated the stochastic linear optimal regulator problem. This problem (Definition 3.4) exhibits a n essential difference from the deterministic regulator problem because the white noise makes it impossible to predict exactly how the system is going to behave. Because of this, the best policy is obviously not to determine the input tr(t) over the control period [ t o , tl] apriori, but to reconsider the situation at each intermediate instant t on the basis of all available information. At the instant t the further behavior of the system is entirely determined by the present state x ( t ) , the input U ( T )for T 2 t , and the white noise W ( T )for T 2 t . All the information from the past that is relevant for the future is contained in the state x ( t ) . Therefore we consider control laws of the form
which prescribe an input corresponding to each possible value of the state at time t . The use of such control laws presupposes that each component of the state can be accurately measured at all times. As we have pointed out before, this is an unrealistic assumption. This is even more so in the stochastic case where the state in general includes components that describe the disturbances or the reference variable; it is very unlikely that these components can be easily measured. We postpone the solution of this difficulty until after
260
Optimnl Linear Stntc Feedback Control Systems
Chapter 4, however, where the reconstruction of the state from incomplete and inaccurate measurements is discussed. In preceding sections we have obtained the solution of the deterministic regulator problem in the feedback form 3343. For the stochastic version of the problem, we have the surprising result that the presence of the white noise term w(t) in the system equation 3316 does not alter the solution except to increase the minimal value of the criterion. We first state this fact and then discuss its proof: Theorem 3.9. The opti~nallinear sol~rtionof the stochastic linear optir~lal regulatorprobler~iis to choose the input accorrlirlg to the linear control law
Here P(t) is the solution of the niatrix Riccati equatiocl
with the ternnir~alconditiorl P ( t 3 = Pp Here we abbreviate as t ~ s t ~ a l R,(t) = D T ( t ) ~ , ( r ) ~ ( t ) . The ni~ir~irnal ual~teof the criterion is giuerl by
I t is observed that this theorem gives only the best li~zearsolution of the stochastic regulator problem. Since we limit ourselves to linear systems, this is quite satisfactory. It can be proved, however, that the linear feedback law is optimal (without qualification) when the white noise is(t) is Gaussian (Kushner, 1967, 1971; Astrom, 1970). To prove the theorem let us suppose that the system is controlled through the linear control law lr(t) = F(t)x(t). 3350 Then the closedloop system is described by the differential equation
and we can write for the criterion 3320
3.6 Stochnstic
Regulator nnd Tracking Problems
261
We know from Theorem 1.54 (Section 1.11.5) that the criterion can be exmessed as
where P(t) is the solution of the matrix differential equation P(t) = [ ~ ( t ) ~(t)~(f)]~F(t)
+ &)[4f)
+ R,(O + P ( t ) ~ d t ) ~ ( t )3354 ,
 B(t)~(f)l
with the terminal condition P(tJ =PI.
3355
Now Lemma 3.1 (Section 3.3.3) states that P(t) satisfies the inequality for all to j t j f,, where P(t) is the solution of the Riccati equation 3346 with the terminal condition 3347. The inequality 3356 converts into an equality if F i s chosen as F"(r) = R;'(T)B~(T)P(T), The inequality 3356 implies that
f jT j
tp
3357
3358 tr [ ~ ( t ) r 2 ] tr [ ~ ( t ) r ] for any nonnegativedefinite matrix r. This shows very clearly that 3353 is minimized by choosing Faccording to 3357. For this choice of F, the criterion 3353 is given by 3349. This terminates the proof that the control law 3345 is the optimal linear control law.
Theorem 3.9 puts us into a position to solve various types of problems. In Sections 3.6.1 and 3.6.2, we showed that the stochastic linear optimal regulator problem may originate from regulator problems for disturbed systems, or from optimal tracking prohlems. In both cases the problem has a special structure. We now briefly discuss the properties of the solutions that result from these special structures. In the case of a regulator with disturbances, the system state differential and output equations take the partitioned form 3308, 3309. Suppose that we partition the solution P(t) of the Riccati equation 3346 according to the partitioning E(t) = col [x(t), x,(t)] as
If, accordingly, the optimal feedback gain matrix is partitioned as
262
Optimal Linear Stnte Feedback Control Systcm~
it is not difficult to see that FLt)
=~?(t)~~(t)P~i(t),
3361 ~ , ( t )= ~ ; ~ ( t ) ~ ~ ( t ) ~ ~ , ( t ) . Furthermore, it can be found by partitioning the Riccati equation that P,,, PI%,and PC%are the solutions of the matrix differential equations
...
P,,(t)
=
~ $ ( t ) ~ ( t ) ~ ; ~ ( t ) ~ ~ ( t ) ~ , ? ( + t ) D,~(~)P,,(I) +~$((f)~,(i) + A,,'(t)P.,(t) + P,,(t)A,(t), 3364
P,,(t,) = 0. We observe that P,,, and therefore also fi, is completely independent of the properties of the disturbances, and is in fact obtained by solving the deterministic regulator problem with the disturbances omitted. Once Pll and Fl have been found, 3363 can be solved to determine P I , and from this F,. The control system structure is given in Fig. 3.14. Apparently, the feedhack link, w h i t e noise
L feedforword Link
dynornics
feedbock Link
Pig. 3.14. Structure of the optimal state feedback regulator with disturbances,
3.6 Stochnstic Rcgulotor and Tracking Problems
263
that is, the link from the state x to the input rr is co~q~lete!y independent of the properties of the disturbances. Thefeedforward link, that is, the link from the state of the disturbances x,, to the input u, is of course dependent upon the properties of the disturbances.
A similar conclusion can be reached for optimal tracking problems. Here it turns out that with the structures 3329 and 3335 of the state differential and output equations the feedback gain matrix can be partitioned as
(note the minus sign that has been introduced), where
Here the matrices PI,, PI,, and P,, are obtained by partitioning the matrix P according to the partitioning Z(t) = col [x(t), x,(r)]; they satisfy the matrix differential equations
We conclude that for the optimal tracking system as well the feedback link is ii~depei~denf of tlreproperties of the reference variable, while the feedforward link is of course influenced by the properties of the reference variable. A schematic representation of the optimal tracking system is given in Fig. 3.15. Let us now return to the general stochastic optimal regulator problem. In practice we are usually confronted with control periods that are very long, which means that we are interested in the case where ti m. In the deterministic regulator problem, we saw that normally the Riccati equation 3346 has a steadystate solution P(t) as ti m, and that the corresponding steadystate control law P(t) is optimal for halfinfinite control periods. I t is not difficult to conjecture (Kushner, 1971) that the steadystate control law
264
Optimnl Linear Stnte Feedbnck Control Systems f eedforword
white
U feedbock Link
Rig. 3.15.
Structure of the optimal state feedback tracking system.
is optimal for the stochastic regulator in the sense that it minimizes
if this expression exists for the steadystate control law, with respect to all other control laws for which 3370 exists. For the steadystate optimal control law, the criterion 3370 is given by
if it exists (compare 3349). Moreover, it is recognized that for a timeinvariant stochastic regulator problem and an asymptotically stable timeinvariant control law the expression 3370 is equal to lim E{zT(t)n,z(t)
+ uT(t)~,u(t)}.
3372
tm
From this it immediately follows that the steadystate optimal control law minimizes 3372 with respect to all other timeinvariant control laws. We see from 3371 that the minimal value of 3372 is given by We observe that if R, = Wo and R, = p W,,, where W, and PV,, are the weighting matrices in the mean square tracking error and the mean square input (as introduced in Section 2.5.1), the expression 3372 is precisely Here C., is the steadystate mean square tracking error and C,, the steadystate mean square input. To compute C., and C,, separafe[y,as usually is required, it is necessary to set up the complete closedloop system equations and derive from these the differential equation for the variance matrix of the
3.6 Stochnstic Regulator and Tmdting Problems
265
state. From this variance matrix all mean square quantities of interest can be obtained. Example 3.13. Stirred rank regttlator In Example 3.11 we described a stochastic regulator problem arising from the stirred tank problem. Let us, in addition to the numerical values of Example 1.2 (Section 1.2.3), assume the following values:
Just as in Example 3.9 (Section 3.4.1), we choose the weighting matrices R, and R, as follows.
where p is to be selected. The optimal control law has been computed for p = 10, 1, and 0.1, as in Example 3.9, but the results are not listed here. I t turns out, of course, that the feedback gains from the plant state variables are not affected by the inclusion of the disturbances in the system model. This means that the closedloop poles are precisely those listed in Table 3.1. In order to evaluate the detailed performance of the system, the steadystate variance matrix
has been computed from the matrix equation
+
+
3378 0 = (A  [email protected] O(A  B F ) ~ ' if. The steadystate variance matrix of the input can be found as follows:
lim E{a(t)uT(t)} = lim ~ { F z ( t ) x ~ ( t ) F=q FOPT. 1m
3379
tm
From these variance matrices the rms values of the components of the controlled variable and the input variable are easily obtained. Table 3.2 lists the results. The table shows very clearly that as p decreases the fluctuations in the outgoing concentration become more and more reduced. The fluctuations in the outgoing flow caused by the control mechanism also eventually decrease with p. All this happens of course at the expense of an increase in the fluctuations in the incoming feeds. Practical considerations must decide which value of p is most suitable.
266
Optimal Linear Stntc Reedbnck Control Systems
Table 3.2 Rms Values for StirredTank Regulator Steadystate nns values of
P
Incremental outgoing flow (m%)
Incremental feed Incremental concentration (kmol/m3)
No. 1 (m3/s)
No. 2 (ds)
Example 3.14. Angrrlar velocity trackiug system Let us consider the angular velocity trackingproblem as outlined in Example 3.12. To solve this problem we exploit the special structure of the tracking problem. It follows from 3365 that the optimal tracking law is given by
The feedback gain Fl(t) is independent of the properties of the reference variable and in fact has already been computed in previous examples where we considered the angular velocity regulation problem. From Example 3.7 (Section 3.4.1), it follows that the steadystate value of the feedback gain is given by

while the steadystate value of Pll is
By using 3368, it follows that the steadystate value of PI,can be solved from 3383
Solution yields
3.6 Stochastic Regulator and Trucking Problems
267
so that
IC
Finally, solution of 3369 for p2, gives
Let us choose the following numerical values:
This yields the following numerical results:
From 3373 it follows that
+
lim [E{%?(t)} pE{pZ(t)}]= tr ( P V ) ,
tm
3390
where <(I)= [(I)  5,(t). Since in the present problem
we find that lim [E{?(I)} 1m
+ pE{ti2(t)}] = 291.8 rad2/s2.
3392
We can use 3392 to obtain rough estimates of the rms tracking error and rms input voltage as follows. First, we have from 3392 lim E { f 3 ( t ) }< 291.8 radZ/s'. t m
3393
268
Optimal Linear Stnte Feedback Control Systems
I t follows that steadystate rms tracking error
< 17.08 rad/s.
3394
Similarly, it follows from 3392 lim E{,u'(!)} tm
< 791.8 = 0.2918 V2. P
3395
We conclude that steadystate rms input voltage < 0.5402 V. 3396 Exact values for the rms tracking error and rms input voltage can be found by computing the steadystate variance matrix of the state Z ( t ) of the closedloop augmented system. This system is described by the equation
As a result, the steadystate variance matrix matrix equation
0 of Z ( t ) , is the solution of the
3399
Numerical solution yields
The steadystate mean square tracking error can be expressed as lim E { [ & f )  C,(!)]" = Qe,,  2Q,,
tm
+ &,,
= 180.7 rad3/s',
3401 where the 0 , are the entries of 0. Similarly, the mean square input is given by
3.6
Stochnstic Rcgulntor
and
Trncking Problems
269
I n Table 3.3 the estimated and actual rms values are compared. Also given are the openloop rms values, that is, the rms values without any control at all. I t is seen that the estimated rms tracking error and input voltage are a little on the large side, but that they give a very good indication of the orders of magnitude. We moreover see that the control is not very good since the rms tracking error of 13.44 rad/s is not small as compared to the rms value of the
Table 3.3 Numerical Results for the Angular Velocity Tracking System Steadystate tracking error (radls)
Steadystate rms input voltage (v)
30 117.08 13.44
0 <0.5402 0.3333
m s
Openloop Estimated closedloop Actual closedloop
reference variable of 30 rad/s. Since the rms input is quite small, however, there seems to be room for considerable improvement. This can be achieved by choosing the weighting coe5cient p much smaller (see Problem 3.5). Let us check the reference variable and closedloop system bandwidths for the present example. The reference variable break frequency is 118 = 1 rad/s. Substituting the control law into the system equation, we find for the closedloop system equation
This is a firstorder system with break frequency
Since the power spectral density of the reference variable, which is exponentially correlated noise, decreases relatively slowly with increasing frequency, the difference in break frequencies of the reference variable and the closedloop system is not large enough to obtain a sufficiently small tracking error.
270
Optimal Linear State Feedback Control Systems
3.7 REGULATORS AND TRACKING SYSTEMS WITH NONZERO SET POINTS AND CONSTANT DISTURBANCES 3.7.1 Nonzero Set Points
In our discussion of regulator and tracking problems, we have assumed up to this point that the zero state is always the desired equilibrium state of the system. In practice, it is nearly always true, however, that the desired equilibrium state, which we call the set poiilt of the state, is a constant point in state space, different from the origin. This kind of discrepancy can be removed by shifting the origin of thz state space to this point, and this is what we have always done in our examples. This section, however, is devoted to the case where the set point may he variable; that is, we assume thal the set point is constant over long periods of time but that from time to time it is shifted. This is a common situation in practice. We limit our discussion to the timeinvariant case. Consider the linear timeinvariant system with state differential equation where the controlled variable is given by Let us suppose that the set point of the controlled variable is given by 2,. Then in order to maintain the system at this set point, a constant input u, must be found (dicaprio and Wang, 1969) that holds the state at a point x, such that z, = Dx,. 3407 It follows from the state differential equation that x, and u, must be related 0 = Ax,
+ BII,.
Whether or not the system can be maintained at the given set point depends on whether 3407 and 3408 can be solved for u, for the given value of z,. We return to this question, but let us suppose for the moment that a solution exists. Then we define the shifted input, the shifted state, and the slrifted co~itrolledvariable, respectively, as
3.7 Nonzero Sct Points nnd Constant Disturbances
271
I t is not d~fficultto find, by solvingthese equations for 11, x, and 2 , substituting the result into the state differential equation 3405 and the output equation 3406, and using 3407 and 3408, that the shifted variables satisfy the eauations
Suppose now that at a given time the set point is suddenly shifted from one value to another. Then in terms of the shifted system equations 3410, the system suddenly acquires a nonzero initial state. I n order to let the system achieve the new set point in an orderly fashion we propose to effect the transition such that an optimization criterion of the form [z"'(t)R,zl(t)
+ L I ' ~ ( ~ ) R ~dtL I+' (~~ ') ~] ( t ~ ) ~ ~ x ' (3411 tJ
is minimized. Let us assume that this shifted regulator problem possesses a steadystate solution in the form of the timeinvariant asymptotically stable steadystate control law d ( t ) = Pxl(t). 3412 Application of this control law ensures that, in terms of the original system variables, the system is transferred to the new set point as quickly as possible without excessively large transient input amplitudes. Let us see what form the control law takes in terms of the original system variables. We write from 3412 and 3409:
This shows that the control law is of the form u(t) = Fx(t)
+
3414
I!;,
where the constant vector u; is to be determined such that in the steadystate situation the controlled variable z(t) assumes the given value 2,. We now study the question under what conditions u; can be found. Substitution of 3414 into the system state differential equation yields
Since the closedloop system is asymplotically stable, as t reaches a steadystate values x, that satisfies 0 = Ai,
Here we have abbreviated
+ BII;.
K=ABE

m the state
272
Optimnl Lincnr State Feedback Control Systems
Since the closedloop system is asymptotically stable, Khas all of its characteristic values in the lefthalf complex plane and is therefore nonsingular; consequently, we can solve 3416 for z,: z, = (A)"Ba;. 3418 If the set point z, of the controlled variable is to be achieved, we must therefore have z, = D(K)'BU;. 3419 When considering the problem of solving this equation for 11; for a given value of z,, three cases must be distinguished: of z is greater tho11 that of 11: Then 3419 has a solution (a) The di~rte~tsiort for special values of z, only; in general, no solution exists. In this case we attempt to control the variable z(t) with an input u(t) of smaller dimension; since we have too few degrees of freedom, it is not surprising that no solution can generally be found. (b) The dime~tsiortsof tr and z are the sortie, that is, a sufficient number of degrees of freedom is available to control the system. In this case 3419 can be solved for 11; provided D(X)lB is nonsingular; assuming this to be the case (we shall return to this), we find 11;
= [D(K)'B]lz,,,
3420
which yields for the optimal input to the tracking system u(t) =
&t)
+ [D(2)lB]'2,.
3421
(c) Tlre diir~ensio~t of z is less than that of u: In this case there are too many degrees of freedom and 3419 has many solutions. We can choose one of these solutions, but it is more advisable to reformulate the traclting problem by adding components to the controlled variable. On the basis of these considerations, we henceforth assume that [email protected]) = dim (n),
3422
so that case (b) applies. We see that D(A)'B
= flC(0),
3423
where H&) = D(SI  K1l~.
3424
We call H,(s) the closeclloop transfer ~rrotrix,since it is the transfer matrix from rr1(t) to z(t) for the system i ( t ) = Ax([)
+ Bu(t),
z(t) = Dx(t), u(t) = Fz(t)
3425
+ ll'(t).
3.7 Nonzero Set Poinh and Constant Disturbances
273
In terms of HJO) the optimal control law 3421 can be written as
As we have seen, this control law has the property that after a step change in the set point z, the system is transferred to the new set point as quickly a s possible without excessively large transient input amplitudes. Moreover, this control law of course makes the system return to the set point from any initial state in an optimal manner. We call 3426 the nonzero set point optimal c o n l m l l a ~ tI~t .has the property that it statically decouples the control system, that is, the transmission T ( s ) of the control system (the transfer matrix from the set point z, to the controlled variable z) has the property that T ( 0 ) =I. We now study the question under what conditions HJO) has an inverse. I t will be proved that this property can be directly ascertained from the openloop system equations
Consider the following string of equalities det [H,(s)] = det [D(sI  A = det
[D(sI
+ BF)'B]
 A)'{I
+ B F ( d  A)'1'B]  E(s1  A + BE)'B] det [I  ( s l  A + BF)'BF] det [(sI  A + BE)'] det (sI  A) det (sI  A )
= det [ D ( d  A)'B] det [I = det [D(sI  A)"B] = det [D(sl  A)'B] 
det [D(sl  A)'B] det (sI  A
+ BF)
Here we have used Lemma 1.1 (Section 1.5.3) twice. The polynomial y(s) is defined by
where H ( s ) is the openloop transfer matrix
274
Optimal Lincnr Stnte Feedback Control Systems
and $(s) the openloop characteristic polynomial $(s) = det (sI  A).
3431
Finally, &(s) is ilk closedloop characteristic polynomial $,(x)
= det (sI  A
+ BF).
3432
We see from 3428 that the zeroes of the closedloop transfer matrix are the same as those of the openloop transfer matrix. We also see that
[D(Z)'B]= det [H,(O)]
=YJ(~)
3433 $O(O) is zero if and only if y~(0)= 0. Thus the condition y(0) # 0 guarantees that D(ii)'B is nonsingular, hence that the nonzero set point control law exists. These results can be summarized as follows. det
Theorem 3.10. Consider the timeinuariant system
wlrere z arid u have the same rlinlensions. Consider any asynytatically stable tili?ei?~uaria~lt control laiv ~ i ( t )= Fx(t)
+ ul(t).
3435
Let H(s) be the openloop transfer n1atri.x H(s) = D(sI  A)lB, and H,(s) the closedloop transfer niatrix HJs) = D(sI  A
+ BF)'B.
T11ei1 HJO) is n a ~ ~ s i q y l aam/ r the controlled variable z(t) con under steodystate cor~clitioiisbe ~ilaintainedat any constant va111ez, by choosing
if and only if H(s) lras a nonzero nwnerator polynon~ialthat has no zeroes at the origin. I t is noted that the theorem is stated for any asymptotically stable control law and not only for the steadystate optimal control law. The discussion of this section has been c o n h e d to deterministic regulators. Of course stochastic regulators (including tracking problems) can also have nonzero set points. The theory of this section applies to stochastic regulators
3.7 Nonzero Set Points and Constant Disturbnnccs
275
without modification; the nonzero set point optimal control law for the stochastic regulator is also given by
Example 3.15. Positiori control sj~stenl Let us consider the position control system of Example 3.4 (Section 3.3.1). In Example 3.8 (Section 3.4.1), we found the optimal steadystate control law. I t is not &cult to find from the results of Example 3.8 that the closedloop transfer function is given by
If follows from 3435 and 3438 that the nonzero set point optimal control law is given by
, is the set point for the angular position. This is precisely the control where I law 3171 that we found in Example 3.8 from elementary considerations. Example 3.16. Stirred tank As an example of a multivariable system, we consider the stirredtank regulator problem of Example 3.9 (Section 3.4.1). For p = 1 (where p is defined as in Example 3.9), the regulator problem yields the steadystate feedback gain matrix
It is easily found that the corresponding closedloop transfer matrix is given
'JY
276
Optimal Linear State Feedback Control Systems
From this the nonzero set point optimal control law can be found to be n(t) = Fz(t)
+
i
10.84 1.931
0.1171
.)
3444
0.07475
Figure 3.16 gives the response of the closedloop system lo step changes in the components of the set point 2., Here the set point of the outgoing flow is
incremental
outgoing concentrotion
Fig. 3.16. Thcresponses of the stirred lank as a nonzero set point regulating system. Left column: Responses of the incremental outgoing flow and concentration to a step of 0.002m0/s in the set point of the flow. Right column: Responses of the incremental outgoing Row and concentration to a step of 0.1 kmol/ma in the set point of the concentration.
changed by 0.002 d / s , which amounts to 10 % of the nominal value, while the set point of the outgoing concentration is changed by 0.1 kmol/m3, which is 8 % of the nominal value. We note that the control system exhibits a certain amount of dynamic corrpling or irzteroction, that is, a change in the set point of one of the components of the controlled variable transiently affects the other component. The effect is small, however.
3.7 Nonzero Set Points and Constont Dislurbnnccs
3.7.2*
277
Constant Disturbances
I n this subsection we discuss a method for counteracting the effect of constant disturbances in timeinvariant regulator systems. As we saw in Chapter 2, in regulators and tracking systems where high precision is required, it is important to eliminate the effect of constant disturbances completely. This can be done by the application of integrating action. We introduce integrating action in the context of state feedback control by first extending the usual regulator problem, and then consider the effect of constant disturbances in the corresponding modified closedloop control system configuration. Consider the timeinvariant system with state differential equation
with x(t,) given and with the controlled variable
We add to the system variables the "integral state" q(t) (Newell and Fisher, 1971; Shih, 1970; Porter, 1971), defined by
with ~(1,)given. One can now consider the problem of minimizing a criterion of the form
where R,, Rj, and R, are suitably chosen weighting matrices. The first term of the integrand forces the controlled variable to zero, while the second term forces the integral state, that is, the total area under the response of the controlled variable, to go to zero. The third term serves, as usual, to restrict the input amplitudes. Let us assume that by minimizing an expression of the form 3448, or by any other method, a timeinvariant control law
is determined that stabilizes the augmented system described by 3445, 3446, and 3447. (We defer for a moment the question under which conditions such an asymptotically stable control law exists.) Suppose now that a constant disturbance occurs in the system, so that we must replace the state differential equation 3445 with
+
+
3450 ~ ( 1= ) A 4 t ) Bu(t) v,, where u, is a constant vector. Since the presence of the constant disturbance
278
Optinlnl Linenr Stntc Ferdbnck Control Systcms
does not affect the asymptotic stability of the system, we have lim q(t) = 0, or, from 3447,
tm
lim z(t) = 0. tm
This means that the control sj~sfeinwit11 the as~tnrptoticallystable control la11, 3449 has tliepropert~rthat the effect of comtant disttirbances on the cor~tro/led
uariable eue~itliallyvanislres. Since this is achieved by the introduction of the integral state g, this control scheme is a form of integral control. Figure 3.17 depicts the integral control scheme.
Fig. 3.17. State feedback integral
control.
Let us now consider the mechanism that effects the suppression of the constant disturbance. The purpose of the multivariahle integration of 3447 is to generate a constant contribution t i , to the input that counteracts the effect of the constant disturbance on the controlled variable. Thus let us consider the response of the system 3450 to the input Substitution of this expression into the state differential equation 3450 yields
+
3454 ~ ( t=) ( A  BFl)x(t) Bu, + 0,. In equilibrium conditions the state assumes a constant value a, that must satisfy the relation 3455 0 = xx, Bu, u,, where
+
+
K=ABF,. Solution for x, yields x, = (  x )  l B ~ i ,
3456
+ (x)'v,,
provided K is nonsingular. The corresponding equilibrium value
3457 2,
of the
3.7 Nonzero Set Points nnd
Constant Disturbances
279
controlled variable is given by z, = DX, = D(A)~BII,
+ D(x))~~,.
3458
When we now consider the question whether or not a value of 11, exists that makes z, = 0, we obviously obtain the same conditions as in Section 3.7.1, broken down to the three following cases. (a) The di~itertsiortof
2: is greater
tltan tlrot of
11:
In this case the equation
0 = D(L)~BII~ID(L)~Z+,
3459
represents more equations than variables, which means that in general no solution exists. The number of degrees of freedom is too small, and the steadystate error in z cannot be eliminated. (b) The rlintmtsio~zof z eyrtals that of n: In this case a solution exists if and only if D(L)'B = HJO) 3460 is nonsingular, where HJs) = D(sI  L)'B 3461 is the closedloop transfer matrix. As we saw in Theorem 3.10, HJO) is nonsingular if and only if the openloop transfer matrix H(s) = D(sI  A)'B has no zeroes at the origin. (c) The dimension of z is less than that of tr: In this case there are too many degrees of freedom and the dimension of z can be increased by adding components to the controlled variable. On the basis of these considerations, we from now on restrict ourselves to the case where dim (z) = dim (11). Then the present analysis shows that a necessary condition for the successful operation of the integral scheme under consideration is that the openloop transfer matrix H(s) = D(sI A)'B have no zeroes at the origin. In fact, it can be shown, by a slight extension of the argument of Power and Porter (1970) involving the controllability canonical form of the system 3445, that necessary and sufficient conditions for the existence of an asymptotically stable control law of the form 3449 are that (i) the system 3445 is stabilizable; and (ii) the openloop transfer matrix H(s) = D(sI  A)"B the origin.
has no zeroes a t
Power and Porter (1970) and Davison and Smith (1971) prove that necessary and sufficient conditions for arbitrary placement of the closedloop system poles are that the system 3445 be completely controllable and that the openloop transfer matrix have no zeroes at the origin. Davison and Smith (1971) state the latter condition in an alternative form.
280
Optimnl Linenr State Feedbnek Control Systems
In the literature alternative approaches to determining integral control schemes can be found (see, e.g., Anderson and Moore, 1971, Chapter 10; Johnson, 1971h). Example 3.17. Ii~tegralcontrol of the positioning system Let us consider the positioning system of previous examples and assume that a constant disturbance can enter into the system in the form of a constant torque T, on the shaft of the motor. We thus modify the state differential equation 359to
where y = I/J, with J the moment of inertia of all the rotating parts. As before, the controlled variable is given by We add to the system the scalar integral state q(t), defined by
From Example 3.15 we know that the openloop transfer function has no zeroes at the origin; moreover, the system is completely controllable so that we expect no difficultiesin finding an integral control system. Let us consider the optimization criterion
As in previous examples, we choose Inspection of Fig. 3.9 shows that in the absence of integral control q(t) will reach a steadystate value of roughly 0.01 rad s for the given initial condition. Choosing A = 10 srP 3467 can therefore be expected to affect the control scheme significantly. Numerical solution of the corresponding regulator problem with the numerical values of Example 3.4 (Section 3.3.1) and y = 0.1 kgr1 mr2 yields the steadystate control law with
P(t) = F9(t)
 F,q(t),
3468
3.8 Asymptotic Propcrtics
281
The corresponding closedloop characteristic values are 9.519 & j9.222 sI and 3.168 s1. Upon comparison with the purely proportional scheme of Example 3.8 (Section 3.4.1), we note that the proportional part of the feedback, represented by ' F ~lias , hardly changed (compare 3169), and that the corresponding closedloop poles, which are 9.658 & j9.094 sl in Example 3.8 also have moved very little. Figure 3.18 gives the response of the integral
t input voltage
0
P
01
1
is1 2 I
Fig.3.18. Response of theintegrnl position control system to a constant torque of 10 N m on the shaft of the motor.
control system from zero initial conditions to a constant torque r0of 10 N m on the shaft of the motor. The maximum deviation of the angular displacement caused by this constant torque is about 0.004 rad.
3.8*
3.8.1*
ASYMPTOTIC PROPERTIES O F TIMEINVARIANT O P T I M A L CONTROL LAWS Asymptotic Behavior of the Optimal ClosedLoop Poles
In Section 3.2 we saw that the stability of timeinvariant linear state feedback control systems can be achieved or improved by assigning the closedloop poles to suitable locations in the lefthalf complex plane. We were not able to determine which pole patterns are most desirable, however. In Sections 3.3 and 3.4, the theory of linear optimal state feedback control systems was developed. For timeinvariant optimal systems, a question of obvious interest concerns the closedloop pole patterns that result. This section is devoted to a study of these patterns. This wiU supply valuable information about the response that can be expected from optimal regulators.
282
Optimnl Linear Stnte ficdbnck Control Systems
Suppose that in the timeinvariant regulator problem we let
where N is a positivedefinite symmetric matrix and p a positive scalar. With this choice of R?,the optimization criterion is given by
The parameter p determines how much weight is attributed to the input; a large value of p results in small input amplitudes, while a small value of p permits large input amplitudes. We study in this subsection how the locations of the optimal closedloop regulator poles vary as a function of p. For this investigation we employ root locus methods. In Section 3.4.4 we saw that the optimal closedloop poles are the lefthalf plane characteristic values of the matrix 2 , where
Using Lemma 1.2 (Section 1.5.4) and Lemma 1.1 (Section 1.5.3), we expand det ( s l  2) as follows: sl
1
A
 BN'BT
det ( s l  2) = det = det ( s l
 A)
= det ( s l  A)
= det ( s l
det ( s l
 A)(1).
+ A'')  A)
det (  s t
r
= (l)"+(s)$(s)
det
4
\
N'HT(\
s)R,H(s)],
3473
3.8 Asymptotic Properties
283
where n is the dimension of the state x, and '
$(s) = det (sI  A), H(s) = D(s1 A)lB.
3474
For simplicity, we first study the case where both the input u and the controlled variable z are scalars, while We return to the multiinput multioutput case at the end of this section. I t follows from 3473 that in the singleinput singleoutput case the closedloop poles are the lefthalf plane zeroes of
where H(s) is now a scalar transfer function. Let us represent H(s) in the form
where y(s) is the numerator polynomial of H(s). It follows that the closedloop poles are the lefthalf plane roots of
We can apply two techniques in determining the loci of the closedloop poles. The first method is to recognize that 3478 is a function of s\ to substitute s" s', and to find the root loci in the s'plane. The closedloop poles are then obtained as the lefthalf plane square roots of the roots in the s'plane. This is the rootsquare loclis method (Chang, 1961). For our purposes it is more convenient to trace the loci in the splane. Let us write
where the vi, i = 1,2, . . . , p , are the zeroes of H(s), and the ri,i = 1, 2, . . . ,11, the poles of H(s). To bring 3478 in standard form, we rewrite it with 3479 as
284
Optimnl Linenr Stnte Feedbock Control Systems
Applying the rules of Section 1.5.5, we conclude the following. (a) As p t 0 , of the 211 roots of 3480 a total number of 2p asymptotically approach the p zeroes si,i = 1,2, . . . , p , and their negatives %I,., i = 1,  2 , . . . ,p. (b) As p .0 , the other 2(n  p) roots of 3480 asymptotically approach straight lines which intersect in the origin and make angles with the positive real axis of brr rl
(c) As p distance
p

li
= 0 , 1,2,
0 , the 2(n
 . . ,211  2p  1,
n
 p odd, 3481
 p) faraway roots of 3480 are asymptotically at a
from the origin. (d) As p m, the 211 roots of 3480 approach the 2 , . . . ,n, and their negatives rr,., i = 1,2, . ,n.
..
11
poles rrj, i = 1,
Since the optimal closedloop poles are the lefthalf plane roots of 3480 we easily conclude the following (Kalman, 1964). Theorem 3.11. Consider the steadystate solution of the singleinput singleoutput regulator problen~with R, = 1 mld R, = p. Asslrtne that the openloop system is stabilizable oriddetecfablea~ldletits transferjifitnctior be giuerl by
where the rr,, i = 1, 2 , . . . ,11, are the characteristic ualries of the systmil. Then we haue thefollowing.
(a) As p 4 0 , p of the n optintal closedloop characteristic ualttes asyniptoticolly approacl~the nlrmbers gi, i = 1,2, . . , p , where
.
(h) As p 0, the rernoining 11  p optimal closedloop characteristic values osyrytotically approach straight lines ivl~ichintersect in the origin and iiiake
3.8 Asymptotic Properties
285
a~lg[es11,itlzthe ciegatiue real axis of
* (1 + 4 . h , 11
p
I1
l=O,l;..,
p 7
1,
n
p
euen.
Tllesefaraway closerlloop cl~aracteristicualues are asyiilpfoficollyat a clistarlce
., . f,om the origin. (c) As p . m, tlie 11 closedloop characteristic ualries approach the 11an1bers +.$ 3 i = I , ? , .. . ,n, iv11ere wciri= wi if R e ( d l 0 , 3487 if Re (n,) > 0 . The configuration of poles indicated by (b) is known as a Btctter~sortlr co~Ifig~cratian of order 11  p with radius w, (Weinberg, 1962). In Fig. 3.19
I
Fig. 3.19. Butterworth pole configurations of orders one through five and unit radii.
286
Optimnl Linenr Stnte Feedback Cantrol Systems
some loworder Butterworth configurations are indicaled. In the next section we investigate what responses correspond to such configurations. Figure 3.20 gives an example of the behavior of the closedloop poles for a fictitious openloop polezero configuralion. Crosses mark the openloop poles, circles the openloop zeroes. Since the excess of poles over zeroes is two, a secondorder Butterworth configuration results as p 1 0. The remaining
Fig. 3.20. Root loci of the chamctcristic values of the matrix Z (dashed and solid lines) and of the closedloop poles (solid lines only) for a singleinput singleoutput system with a fictitious openloop polezero configuration.

closedloop pole approaches the openloop zero as p 10. For p m the closedloop poles approach the single lefthalf plane openloop pole and the mirror images of the two righthalf plane openloop poles. We now return to the multiinput case. Here we must investigate the roots of
The problem of determining the root loci for this expression is not as simple
3.8 Asymptotic Properties
287
as in the singleinput case. Evaluation of the determinant leads to an expression of the form
.
where the functions a,(l/p), i = 0, 1,2, . . , I T are polynomials in lip. Rosenau (1968) has given rul'es that are helpful in obtaining root loci for such an expression. We are only interested in the asymptotic behavior of the roots as p + 0 and p m. The roots of 3488 are also the roots of

$(s)$(S)
det [pI
+ N1HT(s)R3~(s)]
= 0.
3490
As p t 0 some of the roots go to infinity; those that stay finite approach the zeroes of $(s)$(S) det [NlIf T (  s ) ~ 3~ ( s ) ] .
3491
provided this expression is not identically zero. Let us suppose that H(s) is a square transfer matrix (in Section 3.7 we saw that this is a natural assumption). Then we know from Section 1.5.3 that
.
Y(S) det [H(s)] = 
$W
where ?/J(s)is a polynomial at most of degree 11  k, with 11 the dimension of the system and k the dimension of 11 andz. As a result, we can write for 3491
det (N)
1p(s)yl(s)
Thus it follows tliat as p 1 0 those roots of 3490 tliat stay finite approach the zeroes of the transfer matrix H(s) and their negatives. This means that those oplimal closedloop poles of the regulntor that stay finite approach those zeroes of H(s) that have negative real parts and the negatives of the zeroes that have nonnegative real parts. I t turns out (Rosenau, 1968) that as p j, 0 the faroff closedloop regulator poles, that is, those poles that go to infinity, generally do not form a single Butterwortli configuration, such as in the singleinput case, but that they group into seuerolButterworth configurations of digerent orders and different radii (see Examples 3.19 and 3.21). A rough estimate of the distance of the faraway poles from the origin can be obtained as follows. Let $,(s) denote the closedloop characteristic polynomial. Then we have $r(s)$o(s) = $(s)$(s)
det
288
Optimal Linear Stnte Feedback Control Systems
For small p we can approximate the righthand side of this expression by
where k is the dimension of the input 11.Let us write
Then the leading term in 3491 is given by
This shows that the polynomial $,(s)$,(s) contains the following terms
..
$o(S)$o(S)= (l)nS2n+ . . . + u2 det (R3) (1)'~~' +  . 3498 p" det ( N ) The terms given are the term with the highest power of s and the term with the highest power of lip. A n approximation of the faraway roots of this polynomial (for small p) is obtained from det (R3) (l)ns?x = 0, det (N) I t follows that the closedloop poles are approximated by the lefthalf plane solutions of det (RJ j / c i l ~ t  d i , (  l ) ( n  n  ~ i / ~ n  n ~(=z ~ 3500 Pk det (N) This first approximation indicates a Butterworth configuration of order 11  p . We use this expression to estimate the distance of the faraway poles to the origin; this (crude) estimate is given by (l)nsz"
+
P"
iuz
det (RJ j""nwll, pkdet (N) We consider h a l l y the behavior of the closedloop poles for p m. In this case we see from 3494 that the characteristic values of the matrix Z approach the roots of $(s)$(s). This means that the closedloop poles i = 1,2, ,ti, as given by 3487. approach the numbers 6, We summarize our results for the multiinput case as follows.

...
Theorem 3.U. Consider the steadystate solrrtiori of the rilultii~iput timeinvariant regulator probleril. Assirrile that the openloop system is stabilizable arid detectable, that the irlpirt tr arid the controlled variable 2 have the same
3.8 Asymptotic Properties
di~i~eiision k , and tliat the state z 110sdi~irension11. Let H(s) be the lc loop trarrsfer matrix H(s) = D(sI  A)lB.
289
x lc open3502
Stppose tl~ot$(s) is the openloop cl~aracteristicpo[yno~i~iol arid write
 ..
Assume that a # 0 and talce R, = pN witlt N > 0, p > 0. (a) Tlien as p 0, p of the optin~alclosedloop regrilator poles approach the ualt~es11,, i = 1, 2, . ,p, ivlrere =
[
1"
if if
Re (45 0
3504 vi Re (I:.) > 0. Tlre remai~ringclosedloop poles go to iifinity arid grorp into several Butter~ ~ o r tconfgt~rotio~is lt of dnferent orders and diferertt radii. A rorrglt estimate of the distance of the faraway closedloop poles to the origin is
Tx";"',
ia2
det (RJ (N) (b) As p * m, tlie 11 closedloop replator poles approaclr the rrranbers 6, i = 1,2;..,n, ivlrere if Re ( R J 0 pk det
<

if
3506
Re(rJ.0.
We conclude this section with the following comments. When p is very small, large input amplitudes are permitted. As a result, the system can move fast, which is reflected in a great distance of the faraway poles from the origin. Apparently, Butterworth pole patterns give good responses. Some of the closedloop poles, however, do not move. away hut shift to the locations of openloop zeroes. As is confirmed later in this section, in systems with lefthalf plane zeroes only these nearby poles are "canceled" by the openloop zeroes, which means that their effect in the controlled variable response is not noticeable. The case p = m corresponds to a very heavy constraint on the input amplitudes. I t is interesting to note that the "cheapest" stabilizing control law ("cheap" in terms of input amplitude) is a control law that relocates the unstable system poles to their mirror images in the lefthalf plane. Problem 3.14 gives some information concerning the asymptotic behavior of the closedloop poles for systems for which dim (u) # dim (a).
290
Optimnl Linear Stnte Reedbnck Control Systems
Example 3.18.
Positior~control system In Example 3.8 (Section 3.4.1), we studied the locations ofthe closedloop poles of the optimal position control system as a function of the parameter p. As we have seen, the closedloop poles approach a Butterworth configuration of order two. This is in agreement with the results of this section. Since the openloop transfer function
has no zeroes, both closedloop poles go to iniinity as p
10.
Example 3.19.
Stirred tank As an example of a multiinput multioutput system consider the stirred tank regulator problem of Example 3.9 (Section 3.4.1). From Example 1.15 (Section 1.5.3), we know that the openloop transfer matrix is given by
For this transfer matrix we have det [H(s)] =
0.01
(s
+ O.Ol)(s + 0.02)
'
Apparently, the transfer matrix has no zeroes; all closedloop poles are therefore expected to go to m as p 10. With the numerical values of Example 3.9 for R, and N , we find for the characteristic polynomial of the matrix Z
Figure 3.21 gives the behavior of the two closedloop poles as p varies. Apparently, each pole traces a firstorder Butterworth pattern. The asymptotic behavior of the roots for p 10 can be found by solving the equation
which yields for the asymptotic closedloop pole locations 0.1373 
JP
and
0.07280  .
JP
3512
3.8 Asymptotic Propcrtics
291
Fig. 3.21. Loci of the closedloop roots for the stirred tank regulator. The locus on top originates from 0.02, the one below from 0.01.
The estimate 3505 yields for the distance of the faraway poles to the origin
We see that this is precisely the geometric average of the values 3512. Exnmple 3.20. Pitch coutrol of an airplane As an example of a more complicated system, we consider the longik tudinal motions of an airplane (see Fig. 3.22). These motions are character! ized by the velocity it along the xaxis of the airplane, the velocity 111 alongI the zaxis of the airplane, the pitch 0, and the pitch rate q = 8. The x and zaxes are rigidly connected to the airplane. The xaxis is chosen to coincide with the horizontal axis when the airplane performs a horizontal stationary flight.
I
Fig. 3.22. The longitudinal motions of an airplane.
292
Optimal Linear Stntc Feedbnck Control Systems
The control variables for these motions are the engine thrust T and the elevator deflection 8. The equations of motion can he linearized around a nominal solution which consists of horizontal Eght with constant speed. I t can he shown (Blakelock, 1965) that the linearized longitudinal equations of motion are independent of the lateral motions of the plane. We choose the components of the slate as follows: incremental speed along xaxis, fl(t) = tr(t), speed along zaxis, f?(t) = ~ ( t ) , Mt) = W , pitch, f4(t) = q(t), pitch rate. The input variable, this time denoted by c, we d e k e as incremental engine thrust,
3515
elevator deflection. With these definitions the state differential equations can be found from the inertial and aerodynamical laws governing the motion of the airplane (Blakelock, 1965). For a particular mediumweight transport aircraft under cruising conditions, the following linearized state differential equation results:
Here the following physical units are employed: u and IV in m/s, 0 in rad, q in rad/s, T i n N, and 8 in rad. In this example we assume that the thrust is constant, so that the elevator deflection S(r) is the only control variable. With this the system is described
293
3.8 Asymptotic Properties
by the state differential equation /0.01580
0.02633
As the controlled variable we choose the pitch
O(t):
It can be found that the transfer function from the elevator deflection to the pitch O ( t ) is given by
B(t)
The poles of the transfer function are 0.006123 ijO.09353, 1.250 ijl.394, while the zeroes are given by 0.02004
and
0.9976.
3520
3521
The loci of the closedloop poles can be found by machine computation. They are given in Fig. 3.23. As expected, the faraway poles group into a Butterworth pattern of order two and the nearby closedloop poles approach the openloop zeroes. The system is further discussed in Example 3.22.
Example 3.21. The control of the longiludinal riiotions of an airplone In Example 3.20 we considered the control of the pitch of an airplane through the elevator deflection. In the present example we extend the system by controlling, in addition to the pitch, the speed along the xaxis. As an additional control variable, we use the incremental engine thrust T(t). Thus we choose for the input variable =
(
incremental eopinetinst, elevator deflection,
3522
294
Optimal Linear State Peedback Control Systems
b
Fig. 3.23. Loci of the closedloop poles of the pitch stabilization system. (a) Faraway poles: (b) nearby poles.
and for the controlled variable incrementalspeed along the zaxis,
3523
pitch. From the syslem state differential equation 3516, it can be computed that the system transfer matrix has the numerator polynomial ~ J ( s= ) 0.003370(s
+ 1.002),
3524
3.8 Asymptotic Properties
295
which results in a single openloop zero at 1.002. The openloop poles are at 0.006123 jO.09353 and 1.250 & jl.394. Before analyzing the,problem any further, we must establish the weighting matrices R, and N. For both we adopt a diagonal form and to determine their values we proceed in essentially the same manner as in Example 3.9 (Section 3.4.1) for the stirred tank. Suppose that R, = diag (u,, u ~ ) Then .
+
+
3525 zT(t)R,z(t) = ulti2(t) uzBZ(t). Now let us assume that a deviation of 10 m/s in the speed along the xaxis is considered to be about as bad as a deviation of 0.2 rad (12") in the pitch. We therefore select u, and u, such that
 0.0004. u1 Thus we choose
UE
where for convenience we have let det (R,) = 1. Similarly, suppose that N = diag (p,, p,) so that
+
3529 cT(t)Nc(t) = plT2(t) pz a2(t). To determine p, and p,, we assume that a deviation of 500 N in the engine thrust is about as acceptable as a deviation of 0.2 rad (12") in the elevator deflection. This leads us to select
which results in the following choice of N:
With these values of R, and N, the relation 3505 gives us the following estimate for the distance of the faroff poles:
The closedloop pole locations must be found by machine computation. Table 3.4 lists the closedloop poles for various values of p and also gives the estimated radius on. We note first that one of the closedloop poles approaches the openloop zero at 1.002. Furthermore, we see that w, is
296
Optimnl Linear Stnte Feedback Control Systems
only a very crude estimate for the distance of the faraway poles from the origin. The complete closedloop loci are sketched in Fig. 3.24. I t is noted that the appearance of these loci is quite different from those for singleinput systems. Two of the faraway poles assume a secondorder Butterworth configuration, while the third traces a fistorder Butterworth pattern. The system is further discussed in Example 3.24.
Fig. 3.24. Loci of the closedloop poles for the longitudinal motion control system. (a) Faraway poles; (6) nearby pole and one faraway pole. For clarity the coinciding portions of the loci on the renl axis ate represented as distinct lines; in reality they coincide with the real axis.
3.8
Asymptotic Properties
297
Table 3.4 ClosedLoop Poles for the Longitudinal Motion Stability Augmentation System Closedloop poles (sl)
3.82" Asymptotic Properties of the SingleInput SingleOutput Nonzero Set Point Regulator
In this section we discuss the singleinput singleoutput nonzero set point optimal regulator in the light of the results of Section 3.8.1. Consider the singleinput system 3533 i ( f ) = Ax(t) bp(t)
+
with the scalar controlled variable Here b is a column vector and d a row vector. From Section 3.7 we know that the nonzero set point optimal control law is given by =
wheref'is the row vector
1 f'm +50, H m
f'=1 bTP,
3535
3536
P
with P the solution of the appropriate Riccati equation. Furthermore, H,(s) is the closedloop transfer function and C0 is the set point for the controlled variable. In order to study the response of the regulator to a step change in the set point, let us replace 5, with a timedependent variable [,(t). The interconnection of the openloop system and the nonzero set point optimal
298
Optimal Linear State Weedback Control Systems
control law is then described by
c(t) = d x (1).
Laplace transformation yields for the transfer function T(s)from the variable set point co(t)to the controlled variable 5 0 ) :
Let us consider the closedloop transfer function d(s1 A Obviously,
+ by)lb.
+
where &(s) = det (ST  A by) is the closedloop characteristic polynomial and y,(s) is another polynomial. Now we saw in Section 3.7 (Eq. 3428) that the numerator of the determinant of a square transfer matrix D(sI  A BF)'B is independent of the feedback gain matrix F and is equal to the numerator polynomial of the openloop transfer matrix D(sI  A)'B. Since in the singleinput singleoutput case the determinant of the transfer function reduces lo the transfer function itself, we can immediately conclude that y~&) equals yt(s), which is defined from
+
Here H(s) = d(s1 A)'b is the openloop transfer function and $(s) = det (s1  A) the openloop characleristic polynomial. As a result of these considerations, we conclude that
Let us write
where the v,., i = 1,2, . . . , p , are the zeroes of H(s). Then it follows from Theorem 3.11 that as p 0 we can write for the closedloop characteristic polynomial
where the fli, i = 1, 2, . . . , p , are defined by 3484, the qi, i= 1, 2,
. .. ,
3.8
Asymptotic Properties
299
I I  p , form a Butterworth configuration of order n  p and radius 1, and where
3545
Substitution of 3544 into 3542 yields the following approximation for T(s):
where x,,(s) defined by
is a Bufterwor.thpo/~~noiniol of order
11
 p , that is, ~,,(s) is
Table 3.5 lists some loworder Butterworth polynomials (Weinberg, 1962). Table 3.5 Butterworth Polynomials of Orders One through Five )I&) x&) x,(s) x4(s) &(s)
+1 + 1.414s + 1 = s3 + 2s3 + 2s + 1 = s1 + 7.613s3 + 3.414sD + 2.613s + 1 = s5 + 3.236s4 + 5.7369 + 5.236s3 + 3.236s + 1 =s = s2
The expression 3547 shows that, if the openloop transfer function has zeroes in the leftlrolfplane 0114, the control system transfer function T(s) approaches 1 %.,(~l%)
3549
as p 10. We call this a Butterworth tr.ansfer fiiitction of order n  p and break frequency a,. In Figs. 3.25 and 3.26, plots are given of the step responses and Bode diagrams of systems with Butterworth transfer functions
300
Optimnl Linear State Feedback Control Systems
step 1 response

I
Rig. 3.25. Step responses of systems with Butlerworth transfer functions of orderi one through five with break frequencies 1 rad/r;.
of various orders. The plots of Fig. 3.25 give an indication of the type of response obtained to steps in the set point. This response is asymptotically independent of the openloop system poles and zeroes (provided the latter are in the lefthalf complex plane). We also see that by choosing p small enough the break frequency w , can he made arbitrarily high, and conespondingly the settling time of the step response can he made arbitrarily smaU. An extremely fast response is of course obtained at the expense of large input amplitudes. This analysis shows that the response of the controlled variable to changes in the set point is dominated by the faroff poles iliwa, i = 1,2, . . . ,n  p. The nearby poles, which nearly coincide with the openloop zeroes, have little effect on the response of the controlled variable because they nearly cancel against the zeroes. As we see in the next section, the faroff poles dominate not only the response of the controlled variable to changes in the set point but also the response to arbitrary initial conditions. As can easily he seen, and as illustrated in the examples, the nearby poles do show up in the iilput. The settling time of the tracking error is therefore determined by the faraway poles, but that of the input by the nearby poles. The situation is less favorable for systems with righthalfplane zeroes. Here the transmission T(s) contains extra factors of the form s+% s  17,
3550
3.8 Asymptotic Properties
0.01 0
0.1
I
10
uIrodlsl
301
100
90 180
270 360
450
Pig. 3.26. Modulus and phase of Butterworth transfer functions of orders one through five with break frequencies 1 rad/s.
and the tracking error response is dominated by the nearby pole at lli. This points to a n inherent limitation in the speed of response of systems with righthalf plane zeroes. I n the next subsection we further pursue this topic. First, however, we summarize the results of this section:
Theorem 3.13. Consider the nonzero set point optimal control law 3535for the timeinuariant, singleinplrt singleoutprrt, stabilizable and detectable svstem
wlrere R, = 1 and R, = p. Then as p 0 the control sjuten1 tmnsniission T(s) (i.e., the closedloop transfer firnctian from the uariable set point [,(t)
302
Optirnnl Linear State Peedback Conlrol Systcrns
to the confrolled uariable i ( t ) ) approacltes
11t11erex,,(s) is a Butterivorfl~poiynoii~ialof order 11  p ai~dradills 1, n is the order of the system, p is the ntriilber of zeroes of the openloop fraiuj%r firilctioil of tile systenl, w, is the asJJlllptoticradius of the Butterluorfll conjigwatiorl of thefara~sajlclosedlooppoles asgiuel~by 3486, r i , i = 1 , 2, . . . , p, are the zeroes of the openloop transfer jirnction, and gi, i = 1, 2, . . , p , are the openloop frartsferjirnctio,~zeroes rnirrored info the leftha[fco~q~lex plane.
.
Example 3.22. Pitch control Consider the pitch control problem of Example 3.20. For p = 0.01 the steadystate feedback gain matrix can be computed to be
The corresponding closedloop characteristic polynomial is given by The closedloop poles are 0.02004,
0.9953,
and
0.5239 & j5.323.
3558
We see that the first two poles are very close to the openloop zeroes at 0.2004 and 0.9976. The closedloop transfer function is given by
so that HJO) = 0.1000. As a result, the nonzero set point control law is given by 3560 6(t) = yx(t)  IO.OOO,(~), where B,(t) is the set point of the pitch. Figure 3.27 depicts the response of the system to a step of 0.1 rad in the set point O,(t). I t is seen that the pitch B quickly settles at the desired value; its response is completely determined by the secondorder Butterworth configuration at 5.239 lt j5.323. The pole at 0.9953 (corresponding to a time constant of about 1 s) shows up most clearly in the response of the speed along the zaxis 111 and can also be identified in the behavior of the elevator deflection 6. The very slow motion with a time constant of 50 s, which
3.8 Asymptotic Properties
303
i""'""t"1 sped along xoxis
speed .,long zaxis W
I
im/sr
pitch 9
l
Irodl
O'r,
0 0
tl51
5
I
Irodl 1

Fig.3.27. Responsc of the pitch control system to a step oF0.1 rad in the pitch angle set point.
corresponds to the pole at 0.02004, is represented in the response of the speed along the xaxis ti, the speed along the zaxis 111,and also in the elevator deflection 6, although this is not visible in the plot. It takes about 2 min for 11 and is to settle at the steadystate values 49.16 and 7.7544s. Note that this control law yields an initial elevator deflection of 1 rad which, practically speaking, is far too large.
304
Optimal Linear State Feedback Control Systems
Example 3.23. System 11dt11a righthalfplane zero As a second example consider the singleinput system with state differential eauation
Let us choose for the controlled variable c(t) = (1, l)x(t).
3562
This system has the openloop transfer function
and therefore has a zero in the righthalf plane. Consider for this system the criterion
I t can be found that the corresponding Riccati equation has the steadystate solution
&
l+Jl+4p+2fi
JP
42
+
.
3565
\ia)
The corresponding steadystate feedback gain vector is

T h g  ~ l m o o poles p can be found to he
FigureG.28 gives a sketch of the loci of the closedloop poles. As expected, one of the closedloop poles approaches the mirror image of the righthalf plane zero, while the other pole goes to m along the real axis. For p = 0.04 the closedloop characteristic polynomial is given by
and the closedloop poles are located at 0.943 and 5.302. The closedloop
Wig. 3.28.
Loci of the closedloop poles for a system with a righthalf plane zero.
Fig. 3.29. Response of a closedloop system with a righlhalf plane zero to a unit step in the set poinl.
306
Optimnl Linenr Slntc Rcdbnck Control Systems
transfer function is
so that HJO) = 0.2. The steadystate feedback gain vector is
f = (5, 4.245).
3570
As a result, the nonzero set point control law is
Figure 3.29 gives the response of the closedloop system to a step in the set point c,(t). We see that in this case the response is dominated by the closedloop pole at 0.943. It is impossible to obtain a response that is faster and at the same time has a smaller integrated square tracking error. 3.8.3'
The Maximally Achievable Accuracy of Regulators and Tracking Systems
In this section we study the steadystate solution of the Riccati equation as p approaches zero in R, = p N . 3572 The reason for our interest in this asymptotic solution is that it will give us insight into the maximally achievable accuracy of regulator and tracking systems when no limitations are imposed upon the input amplitudes. This section is organized as follows. First, the main results are stated in the form of a theorem. The proof of this theorem (Kwakernaak and Sivan, 1972), which is long and technical, is omitted. The remainder of the section is devoted to a discussion of the results and to examples. We fust state the main results: Theorem 3.14. Consider the timei~luariantstabilizable and detectable linear system
elle ere B and D are assn~nedto hauefi~llrank. Consider also the criterion
where R,
> 0 , Ril > 0 . Let R, = p N ,
1vit11N
> 0 and p apositiue scalar, and let Fpbe the steadystate sol~~tiorl of
3.8 Asymptotic Properlies
307
the Riccati eqz~ation
Tlten the foUoiving facts hold. (a) The limit
lim F,
= Po
3577
exists. (h) Let z,(t), t 2 to, denote the response of the controlled variable for the reglilator that is steadystate optirnalfor R, = pN. Then
(c) Ifdim (z) > dim (ti), tl~enPo # 0. (d) If dim ( 2 ) = dim (11)and the nunlerotor polynomial y(s) of the openloop transfer n1atri.v H(s) = D ( d  A)lB is nonzero, Po = 0 if and o n b if y (s) has zeroes i~'itlrnorlpositiue realparts only. (e) Ifdim (2) < dim (I,),tl~ena sr~flcientcondition for Po to be 0 is that there y(s) of exists a rectangdar matrix M sirclr tlmt the nlouerator poly~ton~ial the syuare transfer matrix D(sI  A)IBM is nonzero and has zeroes isith nonpositiue realparts only. A discussion of the significance of the various parts of the theorem now follows. Item (a) states that, as we let the weighting coe5cient of the input p decrease, the criterion
t , identify ). R, with W , and N with V',,, the approaches a limit ~ ~ ( t , ) ~ ~If( we expression 3579 can he rewritten as
where C,,,(t) = z,,T(t)V',zp(t) is the weighted square regulating error and C,,,(t) = ~ i , ' ( t W,,tr,(t) ) the weighted square input. It follows from item (b) of the theorem that as p 10, of the two terms in 3580 the first term, that is, the integrated square regulating error, fully accounts for the two terms together so that in the limit the integrated square regulating error is given by
308
Optimnl Linenr Slnto Fecdbnck Control Systems
If the weighting coefficient p is zero, no costs are spared in the sense that no limitations are imposed upon the input amplitudes. Clearly, under this condition the greatest accuracy in regulation is achieved in the sense that the integrated square regulation error is the least that can ever be obtained. Parts (c), (d), and (e) of the theorem are concerned with the conditions under which P, = 0, which means that ultimately perfect regulation is approached since lim J"m~o,u) ,'lU
dt
= 0.
3582
10
Part (c) of the theorem states that, if the dimension of the controlled variable is greater than that of the input, perfect regulation is impossible. This is very reasonable, since in this case the number of degrees of freedom to control the system is too small. In order to determine the maximal accuracy that can be achieved, P, must be computed. Some remarks on how this can be done are given in Section 4.4.4. In part (d) the case is considered where the number of degrees of freedom is sufficient, that is, the input and the controlled variable have the same dimensions. Here the maximally achievable accuracy is dependent upon the properties of the openloop system transfer matrix H(s). Perfect regulation is possible only when the numerator polynomial y(s) of the transfer matrix has no righthalf plane zeroes (assuming that y(s) is not identical to zero). This can be made intuitively plausible as follows. Suppose that at time 0 the system is in the initial state xu. Then in terms of Laplace transforms the response of the controlled variable can he expressed as Z(s) = H(s)U(s)
+ D(sI  A)Ix,,
3583
where Z(s) and U(s) are the Laplace transforms of z and u, respectively. Z(s) can be made identical to zero by choosing U(s)
=
Hl(s)D(sI
 A)lx,.
3584
The input u ( f )in general contains delta functions and derivatives of delta functions at time 0. These delta functions instantaneously transfer the system from the state xu at time 0 to a state x(0+) that has theproperty that z(0f) = Dx(O+) = 0 and that z(t) can be maintained at 0 for t > 0 (Sivan, 1965). Note that in general the state x(t) undergoes a delta function and derivative of delta function type of trajectory at time 0 hut that z(t) moves from z(0) = Dx, to 0 directly, without infinite excursions, as can be seen by inserting 3584 into 3583. The expression 3584 leads to a stable behavior of the input only if the inverse transfer matrix H'(s) is stable, that is if the numerator polynomial y(s) of H(s) has no righthalf plane zeroes. The reason that the input 3584
3.8 Asyrnptolic Properties
309
cannot be used in the case that H'(s) has unslable poles is that although the input 3584 drives the controlled variable z(t) to zero and maintains z ( t ) at zero, the input itself grows indefinitely (Levy and Sivan, 1966). By our problem formulation such inputs are ruled out, so that in this case 3584 is not the limiting input as p 0 and, in fact, costless regulation cannot be achieved. Finally, in part (e) of the theorem, we see that if dim (2) < dim ( 1 0 , then Po = 0 if the situation can be reduced to that of part (d) by replacing the input u with an input 11' of the form
I
The existence of such a matrix M is not a necessary condition for Po to be zero, however. Theorem 3.14 extends some of the results of Section 3.8.2. There we found that for singleinput singleoutput systems without zeroes in the righthalf complex plane the response of the controlled variable to steps in the set point is asymptotically completely determined by the faraway closedloop poles and not by the nearby poles. The reason is that the nearby poles are canceled by the zeroes of the system. Theorem 3.14 leads to more general conclusions. It states that for multiinput multioutput systems without zeroes in the righthalf complex plane the integrated square regulating error goes to zero asymptotically. This means that for small values of p the closedloop response of the controlled variable to any initial condition of the system is very fast, which means that this response is determined by the faraway closedloop poles only. Consequently, also in this case the effect of the nearby poles is canceled by the zeroes. The slow motion corresponding to the nearby poles of course shows up in the response of the input variable, so that in general the input can be expected to have a much longer settling time than the controlled variable. For illustrations we refer to the examples. I t follows from the theory that optimal regulator systems can have "hidden modes" which do not appear in the controlled variable but which do appear in the state and the input. These modes may impair the operalion of the control system. Often this phenomenon can be remedied by redefining or extending the controlled variable so that the requirements upon the system are more faithfully reflected. I t also follows from the theory that systems with righthalf plane zeroes are fundamentally deficient in their capability to regulate since the mirror images of the righthalf plane zeroes appear as nearby closedloop poles which are not canceled by zeroes. If these righthalf plane zeroes are far away from the origin, however, their detrimental effect may be limited. I t should he mentioned that ultimate accuracy can of course never be
310
Optimnl Lincnr State Feedhnek Control Systems
i
chieved since this would involve i n h i t e feedback gains and infinite input mplitudes. The results of this section, however, give an idea of the ideal erformance of which the system is capable. In practice, this limit may not early be approximated because of the constraints on the input amplitudes. So far the discussion bas been confined to the deterministic regulator problem. Let us now briefly consider the stochastic regulator problem, which includes tracking problems. As we saw in Section 3.6, we have for the stochastic regulator problem
+
3586 Cam,, PC,,,,, = tr (PV), where C,, and C,,, indicate the steadystate mean square regulation error and the steadystate mean square input, respectively. I t immediately follows that
+ PC,,,,)
lim (C,,,,
= tr
(PoV).
3587
P!U
I t is not difficult to argue [analogously to the proof of part (b) of Theorem 3.141 that of the two terms in 3587 the first term fully accounts for the lefthand side so that lim C,,,, = tr (POI'). I'
!0
This means that perfect stochastic regulation (Po= 0) can be achieved under the same conditions for which perfect deterministic regulation is possible. It furthermore is easily verified that, for the regulator with nonwhite disturbances (Section 3.6.1) and for the stochastic tracking problem (Section 3.6.2), perfect regulation or tracking, respectively, is achieved if and only if in both cases the plant transfer matrix H(s) = D(sI  A)lB satisfies the conditions outlined in Theorem 3.14. This shows that it is the plant alone that determines the maximally achievable accuracy and not the properties of the disturbances or the reference variable. I n conclusion, we note that Theorem 3.14 gives no results for the case in which the numerator polynomial 7p(s) is identical to zero. This case rarely seems to occur, however. Example 3.24. Control of the loi~gitrrrlii~al i~iotior~s of an airplarie As an example of a mnltiinput system, we consider the regulation of the longiludinal motions of an airplane as described in Example 3.21. For p = 100 we found in Example 3.21 that the closedloop poles are 1.003, 4.283, and 19.83 & jl9.83. The 6rst of these closedloop poles practically coincides with the openloop zero at 1.002. Figure 3.30 shows the response of the closedloop system to an initial deviation in the speed along the xaxis u, and to an initial dev~ationin the pitch 0 . I t is seen that the response of the speed along the xaxis is determined
Fig. 3.30. Closedloop responses of a longitudinal stability augmentation system for an airplane. Leit column: Responses to the initial state u(O)=l m/s, while all other components or the initial slale are zero. Right column: Response to the initial state O(0) = 0.01 rad, while all other components of the initial state are zero.
312
Optimal Linear State Feedback Control Systems
mainly by a time constant of about 0.24 s which corresponds to the pole a t 4.283. The response of the pitch is determined by the Butterworth configuration at 19.83 & jl9.83. The slow motion with a time constant of about 1 s that corresponds to the pole at  1.003 only affects the response of the speed along the zaxis IIJ. We note that the controlled system exhibits very little interaction in the sense that the restoration of the speed along the xaxis does not result in an appreciable deviation of the pitch, and conversely. Finally, it should be remarked that the value p = lo' is not suitable from a practical point of view. It causes far too large a change in the engine thrust and the elevator angle. In addition, the engine is unable to follow the fast thrust changes that this control law requires. Further investigation should take into account the dynamics of the engine. The example confirms, however, that since the plant has no righthalf plane zeroes an arbitrarily fast response can be obtained, and that the nearby pole that corresponds to the openloop zero does not affect theresponse of the controlled variable. Example 3.25. A system with a righthalfplane zero In Example 3.23 we saw that the system described by 3561and 3562with the openloop transfer function
has the following steadystate solution of the Riccati equation i+J1+4p+2JP
JP
JP 4,+,)
. 1
3590
7
As p approaches zero, P approaches Po, where
As we saw in Example 3.23, in the limit p I 0 the response is dominated by the closedloop pole at 1. 3.9*
S E N S I T I V I T Y O F L I N E A R S T A T E FEEDBACK CONTROL SYSTEMS
I n Chapter 2 we saw that a very important property of a feedback system is its ability to suppress disturbances and to compensate for parameter changes.
I n this section we investigate to what extent optimal regulators and tracking systems possess these properties. When we limit ourselves to timeinvariant problems and consider only the steadystate case, where the terminal time is at infinity, the optimal regulator and tracking systems we have derived have the structure of Fig. 3.31. The optimal control law can generally be represented
 L+I Fig.
3.31. The
in the form
structure
of a
timeinvariant linear state feedback control system.
+
3592 tl(f) = Fx(i) ~,.x,(t) + Fa%y where xJf) is the state of the reference variable, 2, the set point, and P , Fr, and F, are constant matrices. The matrix F is given by
where P is the nonnegativedefinite solution of the algebraic Riccati equation
In Chapter 2 (Section 2.10) we saw that the ability of the closedloop system to suppress disturbances or to compensate for parameter changes as compared to an equivalent openloop configurationis determined by the behavior of the return difference malrix J(s). Let us derive J(s) in the present case. The transfer matrix of the plant is given by (sI  A)'B, while that of the feedback link is simply F. Thus the return difference matrix is Note that we consider the complete state x(t) as the controlled variable (see Section 2.10). We now derive an expression for J(s) starting from the algebraic Riccati equation 3594.Addition and substraction of an extra term sP yields after
314
Optimal Lincnr Stntc Fccdbnck Control Systcms
rearrangement
 PBR;'BTP  (PSI  AT)F  P(sI  A). 3596 Premultiplication by BT(sI  AT)' and postmultiplication by (sI  A)IB 0 = DTR,D
gives
o = B*(SI  A ~ )  ~ (  P B R ; ~ B ~+P D~'R,D~)(sI A)'B  PP(sI
 A)'B
 B~(SI  A ~ )  ~ P B .3597
This can be rearranged as follows:
After substitution of R;'B~P = E, this can be rewritten as
where H(s) = D(sI  A)'&
Premultiplication of both sides of 3599 by
ET and postmultiplication by f yields after a simple manipulation
If we now substitute s = jw, we see that the second term on the righthand side of this expression is nonnegativedefinite Hermitian; tbis means that we can write JZ'(jw)WJ(jw) 2 W for all real w, 3602 where w = fl'~,E. 3603 We know from Section 2.10 that a condition of the form 3602 guarantees disturbance suppression and compensation of parameter changes as compared to the equivalent openloop system for all frequencies. This is a useful result. We know already from Section 3.6 that the optimal regulator gives aptirital protection against white noise disturbances entering a t the input side of the plant. The present result shows, however, that protection against disturbances is not restricted to tbis special type of disturbances only. By the same token, compensation of parameter changes is achieved. Thus we have obtained the following result (Kreindler, 1968b; Anderson and Moore, 1971).
3.9 Sensitivity
315
Theorem 3.15. Consider the systern c01figwatio11 of Fig. 3.31, ivlrere the "plant" is the detectable and stabilizable timeir~uariantsystem
Let the feedback gain motrix be given by ivlrere P is the rronrtegatiuedefi~tifesol~rfionof the algebraic Riccati eqrration 0 = DTR,D Then the return dl%fererice
 FER;~E*F
J(S) = I satisfies the ineqtrality JT(jo)WJ(jw)
+ A ~ F+ FA.
+ (s1  A)~BE 2
W
for all real
3606
3607 w,
3608
i14er.e
W =PT~DF. 3609 For an extension of this result to timevarying systems, we refer the reader to Kreindler (1969). I t is clear that with the configuration of Fig. 3.31 improved protection is achieved only against disturbances and parameter variations inside the feedback loop. In particular, variations in D fully affect the controlled variable z(t). It frequently happens, however, that D does not exbibit variations. This is especially the case if the controlled variable is composed of components of the state vector, which means that z ( f ) is actaally inside the loop (see Fig. 3.32).
Rig. 3.32. Example of a situation in which the controlled variable is inside the feedback
loop.
316
Optimnl Linear State Fcedbnck Control Systems
Theorem 3.15 has the shortcoming that the weighting matrix F T ~ , Fis known only after the control law has been computed; this makes it difficult t o choose the design parameters R, and R, such as to achieve a given weighting matrix. We shall now see that under certain conditions it is possible to determine an asymptotic expression for PV. I n Section 3.8.3 it was found that if dim (2) = dim (10,and the openloop transfer matrix H(s) = D(sI  A)lB does not have any righthalf plane zeroes, the solution P of the algebraic Riccati equation approaches the zero matrix as the weighting matrix R3 approaches the zero matrix. A glance at the algebraic Riccati equation 3594 shows that this implies that
as R,

P B R ; ~ B ~ F +D'R,D
3610
0, or, since R ; ~ B ~ '= P F , that
FT~,FD~R,D
3611

as R, t 0. This proves that the weighting matrix Win the sensitivity criterion 3608 approaches DZ'R,D as Re 0. We have considered the entire state x(t) as the feedback variable. This means that the weighted square tracking error is sT(l)Wx(l).
From the results we have just obtained, it follows that as R, replaced with x ' ( ~ ) D ~ R , D x (= ~ )t T ( t ) ~ , z ( t ) .


3612 0 this can he
3613
This means (see Section 2.10) that in the limit R , 0 the controlled variable receives all the protection against disturbances and parameter variations, and that the components of the controlled variable are weighted by R,. This is a useful result because it is the controlled variable we are most interested in. The property derived does not hold, however, for plants with zeroes in the righthalf plane, or with too few inputs, because here P does not approach the zero matrix. We summarize oar conclusions: Theorem 3.16. Consider the iseighting matrix
3.9
If the condifions are satisfied (Tlreorem 3.14) tlrerz

as R,
0.
w

D'R, D
Sensitivity

317
r~nllrr~eliicliP * 0 as R9
0,
3617
The results of this section indicate in a general way that state feedback systems offer protection against disturbances and parameter variations. Since sensitivity matrices are not very convenient to work with, indications as to what to do for specific parameter variations are not easily found. The following general conclusions are valid, however. 1. As the weighting matrix R, is decreased the protection against disturbances and parameter variations improves, since the feedback gains increase. For plants with zeroes in the lefthalf complex plane only, the break frequency up to which protection is obtained is determined by the faraway closedloop poles, which move away from the origin as Rz decreases. 2. For plants with zeroes in the lefthalf plane only, most of the protection extends to the controlled variable. The weight attributed to the various components of the controlled variable is determined by the weighting matrix R,. 3. For plants with zeroes in the righthalf plane, the break frequency up to which protection is obtained is limited by those nearby closedloop poles that are not canceled by zeroes. Example 3.26. Positiort confrol system As an illustration of the theory of this section, let us perform a brief sensitivity analysis of the position control system of Example 3.8 (Section 3.4.1). With the numerical values given, it is easily found that the weighting matrix in the sensitivity criterion is given by
This is quite close to the limiting value
To study the sensitivity of the closedloop system to parameter variations, in Fig. 3.33 the response of the closedloop system is depicted for nominal and offnominal conditions. Here the offnominal conditions are caused by a change in the inertia of the load driven by the position control system. The curves a correspond to the nominal case, while in the case of curves b and c the combined inertia of load and armature of the motor is # of nominal
318
Oplirnnl Linear Stntc ficdbock Contn~lSystems
Fig. 3.33. The effccl of parameter variations on the response or the position control
system:
(a)Nominal
load; (6) inertial load
+ of nominal;
(e)
inertial load +of nominal.
and 3 of nominal, respectively. A change in the total moment of inertia by a certain factor corresponds to division of the constants a and K by the same factor. Thus of the nominal moment of inertia yields 6.9 and 1.18 for a and K , respectively, while of the nominal moment of inertia results in the values 3.07 and 0.525 for a and K , respectively. Figure 3.33 vividly illustrates the limited effect of relatively large parameter variations.
+
3.10
CONCLUSIONS
This chapter has dealt with state feedback control systems where all the components of the state can be accurately measured at all times. We have discussed quite extensively how linear state feedback control systems can be designed that are optimal in the sense of a quadratic integral criterion. Such systems possess many useful properties. They can be made to exhibit a satisfactory transient response to nonzero initial conditions, to an external reference variable, and to a change in the set point. Moreover, they have excellent stability characteristics and are insensitive to disturbances and parameter variations.
3.11 Problems
319
All these properties can be achieved in the desired measure by appropriately choosing the controlled variable of the system and properly adjusting the weighting matrices R, and R2.The results of Sections 3.8 and 3.9, which concern the asymptotic properties and the sensitivity properties of steadystate control laws, give considerable insight into the influence of the weighting matrices. A major objection to the theory of this section, however, is that very often it is either too costly or impossible to measure all components of the state. To overcome this difficulty, we study in Chapter 4 the problem of reconstructing the state of the system from incomplele and inaccurate measurements. Following this in Chapter 5 it is shown how the theory of linear state feedback control can be integrated with the theory of state reconstruction to provide a general theory of optimal linear feedback control. 3.11 PROBLEMS 3.1. Stabilization of the position control system Consider the position control system of Example 3.4 (Section 3.3.1). Determine the set of all linear control laws that stabilize the position control system. 3.2. Positiorz control ofof,.ictio~~lessrlc motor A simplification of the regulator problem of Example 3.4 (Section 3.3.1) occurs when we neglect the friction in the motor; the state differential equation then takes the form
where z(t) = col Ifl([), Cz(t)]. Take as the controlled variable H ) = (1, O)x(t), and consider the criterion J:[L"(l)
+ PPW] dt.
3622
(a) Determine the steadystate solution P of the Riccati equation. (b) Determine the steadystate control law. (c) Compute the closedloop poles. Sketch the loci of the closedloop poles as p varies. (d) Use the numerical values rc = 150 rad/(V s2) and p = 2.25 rad2/Vz and determine by computation or simulation the response of the closedloop system to the initial condition [,(0) = 0.1 rad, Cz(0) = 0 rad/s. ~
320
Optimal Linear State Feedback Control Systems
3.3. Regulatioiz of an a~iiplidy/~e Consider the amplidyne of Problem 1.2. (a) Suppose tbat the output voltage is to be kept at a constant value e,, Denote the nominal input voltage as e,, and represent the system in terms of a shifted state variable with zero as nominal value. (b) Choose as the controlled variable 3623 &) = 4 )  QO, and consider the criterion J:[L%)
+ pp1'(t)1&
3624
where 3625 p'(0 = 4 )  eon. Find the steadystate solution of the resulting regulator problem for the following numerical values:
  I s1, R, 10 s1, L1 L? Rl = 5 9, R, = 10 9, lcl = 20 VIA, lc, = 50 VIA, p = 0.025. (c) Compute the closedloop poles. (d) Compute or simulate the response of the closedloop system to the initial conditions z(0) = col (1,O) and z(0) = col(0, I). R1
3.4. Stochastic position control system Consider the position control problem of Example 3.4 (Section 3.3.1) but assume that in addition to the input a stochastically varying torque operates upon the system so tbat the state differentialequation 359 must be extended as follows:
) the effect of the disturbing torque. We model v(t) as Here ~ ( t represents exponentially correlated noise :
+(t) =
10 v(t) + d t ) ,
where w(t) is white noise with intensity 7.0~18. (a) Consider the controlled variable 5(0 = (1, O)z'(t)
3628
3.11 Problems
321
and the criterion
Find the steadystate solution of the corresponding stochastic regulator problem. (b) Use the numerical values K
= 0.787 rad/(V s2),
a = 4.6 sl,
3631
a = 5 rad/s5,
8=ls. Compute the steadystate rms values of the controlled variable 5(t) and the input p(t) for p = 0.2 x radVV. 3.5. Ar~g~rlar velocity trocking system Consider the angular velocity tracking problem of Examples 3.12 (Section 3.6.2) and 3.14 (Section 3.6.3). In Example 3.14 we found that the value of p that was chosen (p = 1000) leaves considerable room for improvement.
(a) Vary p and select that value of p that results in a steadystate rms input voltage of 3 V. (b) Compute the corresponding steadystate rms tracking error. (c) Compute the corresponding break frequency of the closedloop system and compare this to the break frequency of the reference variable. 3.6. Norizero set poir~tregtrlafor for an ariiplidyr~e Consider Problem 3.3 where a regulator has been derived for an amplidyne.
(a) Using the results of this problem, find the nonzero set point regulator. (b) Simulate or calculate the response of the regulator to a step in the output voltage set point of 10 V. 3.7. Extermion of the regulator probleiit Consider the linear timevarying system
i(t) = A(t)x(t)
+ B(t)u(t)
with the generalized quadratic criterion
where Rl(t), R,,(f), and R,(t) are matrices of appropriate dimensions.
3632
322
Optimal Lincnr Slntc Fmdhnck Control Systems
(a) Show that the problem of minimizing 3633 for the system 3632 can be reformulated as minimizing the criterion
+
+
~ ~ [ z ~ t ) ~ ~ t )d z~ (f t) )R ~ u r ( dt t ) ] zl'(tJPli(tl) for the system i ( t ) = A'(t)z(t) where
+ B(t)ri1(t),
R;W = ~ , ( t ) R,,(~)R;Y~)Rz(~), rr'(1) = u(t) R;'(t)RE(t)z(t), ~ y t= ) ~ ( t) ~(t)~;l(t)~g(t) (Kalman, 1964; Anderson, 1966a; Anderson and Moore, 1971). (b) Show that 3633 is minimized for the system 3632 by letting
+
u(t) = P(t)z(t),
3634 3635
3636
3637
where
+
FU(t)= Rd(t)[BT(t)P(t) Rg(t)], with P(t) the solution of the matrix Riccati equation P(t) = [A(t)
3638
 B(t)R;'(t)RZ(t)lT~(t)
+ P(t)[A(t)  B(t)R;'(t)RZ(t)] + RLt)  R~z(t)R;'(t)R~(t)
3639
 P(t)B(t)Ryl(t)BT(t)P(t), t 1 tl, P(fl) = P,. (c) For arbitrary F(t), t 1 t,, let F(t) be the solution of the matrix differential equation &t) = [A(t)  B(t)F(f)lTF(t) P ( t ) [ ~ ( t ) B(t)F(t)]
+
+ R L ~ ) R&)F(~)  ~ ~ ( f ) ~ g ( t ) + ~ ~ ( t ) ~ ? ( t ) ~ ( tt )1 , t,,
3640
P(tJ = P,. Show that by choosing F(t) equal to Fo(t), B(t) is minimized in the sense that F(t) 2 P(t), t 5 t,, where P(t) is the solution of 3639. Rwiark: The proof of (c) follows from (b). One can also prove that 3637 is the best linear control law by rearranging 3640 and applying Lemma3.1 (Section3.3.3) to it. 3.8". Sohitiom of tlie algebraic Riccati equation (O'Donnell, 1966; Anderson, 1966b; Potter, 1964) Consider the algebraic Riccati equation 0 = R,
 PBR$B~P + FA + A ~ F .
3641
3.11 Problems
323
Let Z be the matrix A R1
BR;~B~
AT
3642
Z can always he represented as
where J is the Jordan canonical form of Z. I t is always possible to arrange the columns of W such that J can be partitioned as
Here J,,, J,, and J,, are 11 x
11
blocks. Partition W accordingly as
(a) Consider the equality Z W = WJ,
3646
and show by considering the 12 and 22blocks of this equality that if W12 is nonsingular P = WzCK: is a solution of the algebraic Riccati equation. Note that in this manner many solutions can be obtained by permuting the order of the characteristic values in J. (b) Show also that the characteristic values of the matrix A BR;1B1'W,2K: are precisely the characteristic values of J,, and that the (generalized) characteristic vectors of this matlix are the columns of VIE. Hint: Evaluate the 12block of the identity 3646. 3.9*. Steadystate solrrtion of the Riccati eqtiation by clingo~~alization Consider the 2n x 2n matrix Z as given by 3247 and suppose that it cannot be diagonalized. Then Z can be represented as
where J i s the Jordan canonical form ofZ, and Wis composed of the characteristic vectors and generalized characteristic vectors of Z. I t is always possible to arrange the columns of W such that J can be partitioned as follows
where the n x 11 matrix J, has as diagonal elements those characteristic values of Z that have positive real parts and half of those that have zero
324
Optimal Linear State flcedbaclc Conhol Systems
real parts. Partition Wand V = WI accordingly as
Assume that {A, B) is stahilizable and {A, D) detectable. Follow the argument of Section 3.4.4 closely and show that for the present case the following conclusions hold. (a) The steadystate solution P of the Riccati equation satisfies
P(t) = R,
 P(t)BR;lBTP(t) + ATp(t) + P(t)A V,
+ Vl,P = 0.
(b) W13 is nonsingular and
P
=
3650 3651
w,,wz.
(c) The steadystate optimal behavior of the state is given by Hence Z has no characteristic values with zero real parts, and the steadystate closedloop poles consist of those cbaracterstic values of Z that have negative real parts. Hint: Show that
where the precise form of X(t) is unimportant. 3.10*. Bass' relation for P (Bass, 1967) Consider the algebraic Riccati equation
and suppose that the conditions are satisfied under which it has a unique nonnegativedefinite symmetric solution. Let the matrix Z be given by
It follows from Theorem 3.8 (Section 3.4.4) that Z has no characteristic values with zero real parts. Factor the characteristic polynomial of Z as follows det (sI  Z) = $(s)$(s) 3657 such that the roots of $(s) have strictly negative real parts. Show that P
3.11 Problems
325
satisfies the relation:
Hint: Write $(Z) = $(WJWI) = W $ ( a W1 = W$(.7)lT where V = WI and J = diag (A, A ) in the notation of Section 3.4.4. so111tionof the Riccati eqlrotiorl (Vaughan, 3.11'. Negative espo~~entiol 1969) using the notation of Section 3.4.4, show that the solution of the timeinvariant Riccati equation
can be expressed as follows: where
+
+ WllG(tl 
p(t) = [b W&(tl  t)][W,,
f)]l,
3660
G(t) = e"'~e"',
3661
+ VI,P~)(VZI+ V:,Pl)l.
3662
with 8 = (VII
Show with the aid of Problem 3.12 that S can also be written in terms of Was 3.12*. The re loti or^ between Wand V Consider the matrix Z as defined in Section 3.4.4.
(a) Show that if e = col (e', e"), where e' and e" both are ndimensional vectors, is a right characteristic vector o f 2 corresponding to the characteristic value 1, that is, Ze = Ae, then (e"', elT) is a left characteristic vector o f Z corresponding to the characteristic value 1, that is, (ellT,  d T ) z = ,l(ellz',
3664
(b) Assume for simplicity that all characteristic values At, i = 1, 2 , . . . ,211, of Z are distinct and let the corresponding characteristic vectors be given by e. i = 1,2, . . . ,2n. Scale the e j such that if the characteristic vector e = col (e', e'') corresponds to a characteristic value 1, and f = col (7, f") corresponds to A, then y T e t  pe,, = 1. 3665 Show that if W is a matrix of which the columns are e,, i = 1,2: . . . , 211, and we partition $9
326
Optimnl Linear Stnte Feedback Control Systems
then (O'Donnell, 1966; Walter, 1970)
Hint: Remember that left and right characteristic vectors for different characteristic values are orthogonal.
3.13'. Freqltericy do~iiainsol~ttionof regdator problenis For singleinput timeinvariant systems in phasevariable canonical form, the regulator problem can be conveniently solved in the frequency domain. Let 3668 i ( t ) = Ax(i) bp(t)
+
be given in phasevariable canonical form and consider the problem of minimizing l;[~'(t)
+ PPV)] dt,
3669
where ((t) = dx(t).
3670
(a) Show that the closedloop characteristic polynomial can be found by
3671 P where H(s) is the openloop transfer function H(s) = d(sI  A)'b. (b) For a given closedloop characteristic polynomial, show how the corresponding control law p(t) =  J w ) 3672 can be found. Hint: Compare Section 3.2.
3.14*. The riiinini~rmnuniber of faraway closedloop poles Consider the problem of minimizing
+
p ~xi T( (O t ) ~ ~ ! (dt, t)l ~ ~ [ x ~ ( t ) ~ ~ where R, 2 0, N > 0, and p
3673
> 0, for the system
i ( t ) = Ax(t)
+ Bu(t).
3674
(a) Show that as p L O some of the closedloop poles go to infinity while the others stay finite. Show that those poles that remain finite approach the lefthalf plane zeroes of det [BT(sI
 AT')'R,(sl  A)'B].
3675

(b) Prove that at least k closedloop poles approach inhity, where k is the dimension of the input a. Hint: Let Is1 m to determine the maximum number of zeroes of 3675.Compare the proof of Theorem 1.19 (Section 1.5.3). (c) Prove that as p + m the closedloop poles approach the numbers 7i.I, i = 1,2, . ,11, which are the characteristic values of the matrix A mirrored into the lefthalf complex plane.
..
3.15*. Estimation of the radius of the faraway closedloop poles from the Bode plot (Leake, 1965; Schultz and Melsa, 1967, Section 8.4) Consider the problem of minimizing
Jlo
for the singleinput singleoutput system
Suppose that a Bode plot is available of the openloop frequency response function [email protected]) = d ( j o I  A)lb. Show that for small p the radius of the faraway poles of the steadystate optimal closedloop system canbeestimated: as the frequency w, for which IH(jw.)I =
JP.