Efficient On-Line Computation of Constrained Optimal Control

Transcription

1 Efficient On-Line Computation of Constrained Optimal Control Francesco Borrelli, Mato Baotic, Alberto Bemporad, Manfred Morari Automatic Control Laboratory, ETH Zentrum - ETL, CH-89 Zürich, Switzerland tel: , fax: +--6, borrelli,baotic,bemporad,morari@aut.ee.ethz.ch Dip. Ingegneria dell Informazione, Università di Siena, Via Roma 56, I-5 Siena, Italy tel: , fax: , bemporad@dii.unisi.it Abstract For discrete-time linear time-invariant systems with constraints on inputs and outputs, the constrained finite time optimal controller can be obtained explicitly as a piecewise affine function of the initial state via multiparametric programming [, ]. Exploiting the properties of the value function, we present two algorithms that efficiently perform the online evaluation of the explicit optimal control law both in terms of storage demands and computational complexity.the algorithms are particularly effective when used for Model Predictive Control (MPC) where an open loop constrained finite time optimal control problem has to be solved at each sampling time. Introduction Recently in [] and in [] the authors have shown how to compute the solution of the constrained finite time optimal control (CFTOC) problem as a explicit piecewise affine function of the initial state.such a function is computed off-line by using a multiparametric programming solver [, ], which divides the state space into polyhedral regions, and for each region determines the linear gain which produces the optimal control action. This method reveals its effectiveness when applied to Model Predictive Control (MPC) [8, 6].MPC requires to solve at each sampling time an open-loop CFTOC problem.the optimal command signal is applied to the process only during the following sampling interval.at the next time step a new optimal control problem based on new measurements of the state is solved over a shifted horizon.clearly the explicit solution reduces the on-line computation of the MPC control law to the evaluation of a piecewise affine function instead of the on-line solution of a quadratic or linear program. The only drawback of such explicit optimal control laws is that the number of polyhedral regions could grow exponentially with the size of the optimal control problem. In this paper we focus on efficient on-line methods for the evaluation of this piecewise affine control law.the simplest algorithm would require: (i) the storage of the list of polyhedral regions and of the corresponding linear control laws, (ii) a sequential search through the list of polyhedra for the i-th polyhedron that contains the current state in order to implement the i-th control law. By exploiting the properties of the value function, for CFTOC based on LP and QP, we propose two new algorithms that avoid storing the polyhedral regions.the new algorithms significantly reduce the on-line storage demands and computational complexity of the evaluation of the explicit CFTOC control law and consequently of the explicit MPC control law. CFTOC, MPC and Their Explicit Solution. CFTOC problem formulation Consider the linear time-invariant system { x t+ = Ax t + Bu t () y t = Cx t subject to the constraints y min y t y max, u min u t u max () at all time instants t. In () (), x t R n, u t R m, and y t R p are the state, input, and output vector respectively, y min y max (u min u max ) are bounds on outputs (inputs), and the pair (A, B) is stabilizable. Let x(t) be the state at the generic time t and consider the constrained finite time optimal control problem N J (x(t)) = min U xt+n P p + (x t+k) Q p + u t+k R p k= subj. to y min y t+k y max, k =,...,N u min u t+k u max, k =,...,N x t+k+ = Ax t+k + Bu t+k, k () In()wedenotewithU [u t,...,u t+n ] the optimization vector and with x M p the p-norm of the vector x weighted with the matrix M, in particular x M p = x Mx for p =, x M p = Mx p for p =,. In the following, we will assume that Q = Q, R = R, P, for p =,andthatq, R, P are full column rank matrices for p =,. Depending on the norm p used in the cost function, the optimization problem () can be translated into an LP Although the form () is very common in practical implementations of CFTOC, the results of this paper also hold for the more general mixed constraints Ex t + Lu t M arising, for example, from constraints on the input rate u t u t u t.

2 (p =orp =+ ) orintoaqp(p = ).The optimizer U (t) ={u t,...,u t+n } of problem () is a function of the initial state x(t).it can be computed by solving aqporanlponcex(t) is fixed or it can be computed explicitly for all x(t) within a given range of values as explained in the following.. MPC formulation The solution U (t) ={u t,...,u t+n } of problem () is an open loop optimal control trajectory over a finite horizon.mpc uses it together with a receding horizon strategy to obtain an infinite horizon feedback control law. Consider the problem of regulating to the origin the discrete-time linear time-invariant system () while fulfilling the constraints ().MPC solves such a constrained regulation problem in the following way.assume that a full measurement of the state x(t) is available at the current time t.then, the CFTOC problem () is solved at each time t for t. Let U (t) ={u t,...,u t+n } be the optimal solution of ().Then at time t u(t) =u t () is applied as input to system (). The two main issues regarding this policy are the feasibility of the optimization problem () for all t and the stability of the resulting closed-loop system. We will assume that the matrices Q, R, P, the horizon length N and the terminal constraints in () have been chosen to guarantee the stability and the feasibility of MPC control law ()-().For a detailed discussion see, e.g. [,, 6].. Explicit Solution Based on LP Consider the case of p =orp =+ in problem (). Is well known (e.g. []) that by introducing the vector z {ε x,...,εx N,εu,...,εu N,u t,...,u t+n } R s, s (m +)N, ε x k Qx t+k, ε x N Px t+n, ε u k Ru t+k, and substituting x t+k = A k x(t) + k j= Aj Bu t+k j in (), the optimization problem () can be rewritten in the form J (x) =min f T z z (5) subj.to Gz S + Fx where x = x(t), and the matrices f R s, G R q s, F R q n, S R q are easily obtained from Q, R, P and (), as explained in []. Because the problem depends on x = x(t) the implementation of MPC can be performed in two different ways: solving the LP (5) on-line at each time step or by solving problem (5) off-line for all x within a given range of values, i.e., by considering (5) as a multi-parametric Linear Program (mp-lp). The mp-lp problem consists of computing the optimizer z (x) and the value function J (x) for all possible vectors x in a given compact set X.The solution of mp-lp problems can be approached as proposed in [] or [5]. Once the multi-parametric problem (5) has been solved off-line for a polyhedral set X R n of states, the CFTOC explicit solution z (x) of (5) is available as a piecewise affine function of x, and the model predictive controller ()-() is also available explicitly, as the optimal input u(t) consists simply of m components of z (x) u(t) =[... I m... ]z (x). (6) In [5] the following results about the properties of the solution are proved: Theorem Consider the multi-parametric linear program (5). Then the set of feasible parameters X f is convex,the optimizer (if unique) z (x) :X f R s is continuous and piecewise affine,and the optimal solution J (x) : X f R is continuous,convex and piecewise linear. Corollary The MPC control law (6) u(t) = f(x), f : R n R m,defined by the optimization problem () and () is continuous and piecewise affine.. Explicit Solution Based on QP Consider the case p = in problem ().By substituting x t+k t = A k x(t) + k j= Aj Bu t+k j in (), the optimization problem () can be rewritten as the QP J (x) = x Yx+ min U U HU + x FU subj.to GU W + Ex (7) where x = x(t), the column vector U [u t,...,u t+n ] R s, s mn, is the optimization vector, H = H, and H, F, Y, G, W, E are easily obtained from Q, R, P and ().As only the optimizer U is needed, the term involving Y is usually removed from (7). As in the -norm or -norm case, because the problem depends on x(t), the implementation of MPC can be performed in two different ways: solving the QP (7) on-line at each time step or, as recently proposed in [], by solving problem (7) off-line for all x(t) within a given range of values, i.e., by considering (7) as a multi-parametric Quadratic Program (mp-qp). The mp-qp problem consists of computing the optimizer U (x) and the value function J (x) for all possible vectors x in a given set X.Once the multi-parametric problem (7) has been solved off line, i.e., the CFTOC solution U (x) of (7) has been found, the explicit MPC law is simply u(t) =[I m... ]U (x). (8) In [] the authors prove the following results about the properties of the solution Theorem Consider the multi-parametric quadratic program (7) and let H. Then the set of feasible parameters X f is convex,the optimizer U (x) :X f R s is continuous and piecewise affine,and the optimal solution J (x) :X f R is continuous,convex and piecewise quadratic.

3 Corollary The MPC control law (8) u(t) = f(x), f : R n R m,defined by the optimization problem () and () is continuous and piecewise affine. Corollary states that by using an mp-qp solver the computational effort for MPC is reduced to a piecewise affine function evaluation.in the next section we propose a method to efficiently evaluate such a piecewise affine function without storing the polyhedral regions of the feasible domain X f. Efficient On-Line Algorithms Let the explicit optimal control law be: u (x) =F i x + G i, x P i, i =,...,N r (9) { where F i R m n, G i R m, and} P i = x R n H i x K i,h i R N i c n,k i R N i c, i =,...,N r is a polyhedral partition of X f.in the following H j i denotes the j-row of the matrix H i.the online implementation of the control law (9) can be simply executed according to the following steps: Algorithm. Measure the current state x. Search for the j-th polyhedron that contains x, i.e. H j x K j. Implement the j-th control law, i.e. u (x) =F j (x)+g j In Algorithm, step (.) is critical and it is the only step whose efficiency can be improved.a trivial implementation of step (.) would consist of searching for the polyhedral region that contains the state x as in the following algorithm Algorithm. i =, notfound=;. while i N r and notfound.. j =, feasible=.. while j Nc i and feasible... if H j i x>kj i then feasible=... else j = j +.. end.. if feasible= then notfound=. end Algorithm requires the storage of all polyhedra P i,i.e., (n +)N C real numbers and in the worst case (the state is contained in the last region of the list) it will give a solution after nn C multiplications, (n )N C sums and N C comparisons, where N C N r i= N c i. By using the properties of the value function we show how Algorithm can be replaced by a more efficient algorithm that avoids storing the polyhedral regions P i, i =,...,N r and is computationally faster. In the following we need to distinguish between optimal control based on LP and optimal control based on QP. fx () 5 f f f P P P P x f Figure : Example for Algorithm in one-dimensional case. For a given point x P (x =5)wehave f (x) =max(f (x),f (x),f (x),f (x)).. Efficient Implementation - LP case From Theorem, the value function J (x) corresponding to the explicit solution of CFTOC () based on LP is convex and piecewise affine: J (x) =T i x + V i, x P i, i =,...,N r. () The value function can be used to avoid storing the polyhedral regions P i.from the equivalence of the representations of piecewise linear convex functions [9], the function J (x) in equation () can be represented alternatively as J (x) =max{t i x + V i,i=,...,n} for x X f () Exploiting the equivalence of () and (), the polyhedral region P j containing the state x can simply be identified by searching for the maximum among the list {T i x + V i, } Nr i= : x P j T jx + V j =max{t i x + V i, i =,...,N r }. () Therefore, instead of searching for the polyhedron j that contains the point x, we store the expression of the piecewise-linear value function and identify the region j by searching for the maximum in the list of numbers composed of the single affine function T i x+v i evaluated at x (see Figure ). Algorithm is then substituted by the following: Algorithm. Compute the vector n R Nr, where n i = T i x + V i. Find j such that n j =max(n) Algorithm does not require the storage of any polyhedra P i, but it only requires the storage of the value function,i.e.,(n +)N r real numbers, and it will give a solution after nn r multiplications, (n )N r sums, and N r comparisons.in Table we compare the complexity of Algorithm against Algorithm in terms of storage demand and number of flops. Table : Complexity of Algorithm and Algorithm Algorithm Algorithm Storage (real numbers) (n +)N C (n +)N r Flops (worst case) nn C nn r

4 . Efficient Implementation - QP case Consider the explicit solution of CFTOC () based on QP.From Theorem, the value function J (x) isconvex and piecewise quadratic.the simple Algorithm described in the previous subsection cannot be applied. Therefore in a new algorithm is proposed. Definition Two polyhedra P i, P j are called neighboring polyhedra if they share a facet. Let {P i } Nr i= be the polyhedral partition obtained by solving the mp-qp (7) and denote by C i = {j P j is a neighbor of P i, j =,...,N r,j i} the list of all neighboring polyhedra of P i. Theorem Let f(x) be a continuous real-valued PWA function f(x) ={f i(x) =A ix + B i x P i, i =,...,N r}, () with A i A j, j C i, i =,...,N r. () Define the list O i (x) = { o i j (x) j C i},where { o i j(x) = + f i (x) f j (x) f i (x) <f j (x) Then O i (x) has the following properties: (i) O i (x) =S i = const, x P i, i =,...,N r. (5) (ii) O i (x) S i, x / P i, i =,...,N r. Proof: Let F = P i P j be the common facet of P i and P j.define the linear function gj i (x) =f i(x) f j (x). (6) From the continuity of f(x) it follows that gj i(x) =, x F.From convexity of P i and P j it follows that gj i(x) is the separating hyperplane between P i and P j. (i) Because gj i (x) = is a separating hyperplane, the function gj i(x) does not change its sign for all x P i, i.e., o i j (x) =si j, x P i with s i j =+orsi j =.The same reasoning can be applied to all neighbors of P i to get S i = {s i j, j C i, x P i }.(ii) x / P i, j C i such that H j i x>kj i.since gi j (x) = is a separating hyperplane, o i j (x) = si j. Equivalently, Theorem states the following property x P i O i (x) =S i (7) where S i uniquely characterizes P i.therefore to check on-line if the polyhedral region i contains the state x it is sufficient to compute the binary vector O i (x) and compare it with S i.list S i = {s i j j C i} is calculated off-line, by comparing the values of f i (x) andf j (x) for j C i in a point x belonging to P i, for instance, the Chebychev center of P i. Figure illustrates the procedure with regions.the list of neighboring regions C i and the list S i can be constructed by simply looking at the figure: C = {}, C = {, }, C = {, }, C = {}, S = { }, S = {, }, S = {, }, S = { }.The point x = is in region and we have O (x) = {, } = S, while O (x) ={, } S, O (x) = fx () 5 f f x f f P P P P Figure : Example for Algorithm in one-dimension. {} S, O (x) ={} S.Obviously, if we are in the wrong region, the checking procedure gives us information about the right search direction(s).the solution can be found by searching in the direction where constraint(s) are violated, i.e., we should check the neighboring region P j for which o i j (x) si j. The overall procedure is composed of two parts:. (off-line) Construction of the PWA function f(x) in () satisfying () and computation of the list of neighbors C i and the list S i,. (on-line) Execution of the following algorithm Algorithm. i =, notfound=;. while notfound.. compute O i (x).. if O i (x) =S i then notfound=.. else i = q, where o i q (x) si q, q C i. end Algorithm does not require the storage of the polyhedra P i, but it only requires the storage of one linear function f i (x) per polyhedron, i.e., (n +)N r real numbers and the list of neighbors C i which requires N C integers.in the worst case Algorithm terminates after nn r multiplications, (n )N r sums and N C comparisons.in Table we compare the complexity of Algorithm against Algorithm in terms of storage demand and number of flops. Remark Note that the computation of O i (x) inalgorithm requires the evaluation of Nc i linear functions, but the overall computation never exceeds N r linear function evaluations.consequently, Algorithm will outperform Algorithm, since typically N C N r. Table : Complexity of Algorithm and Algorithm Algorithm Algorithm Storage (real n.) (n +)N C (n +)N r Flops (worst case) nn C (n )N r + N C The PWA function f(x) in () satisfying () will be referred to as PWA descriptor function.in the rest of the section we propose a way to construct a PWA descriptor function from the PWQ value function.in the following we will assume that the mp-qp (7) is not degenerate (refer to [7] for a detailed discussion on the degeneracy of mp-qp).

5 j qx j( ) qx i( ) (a) i j qx j( ) (b) qx i( ) Figure : Two convex piecewise quadratic functions: (a) not differentiable one, (b) differentiable one. Let J (x) be the convex and piecewise quadratic (CPWQ) value function corresponding to the explicit solution of CFTOC () based on QP: } J (x) = {q i (x) x Q i x + T i x + V i, for x P i, i =,...,N r, (8) We will prove that the J (x) isac () function and we will obtain a PWA descriptor function by differentiating it.before going further we recall the following result [, 7]: Theorem Consider the set of active constraints at the optimum of QP (7) and assume there are no degeneracies: A (x) ={i G i U (x) =W i + S i x}, (9) then. A (x) is constant x P i, i =,...,N r,i.e. A (x) A i x P i. If P i and P j are neighboring polyhedra then A i A j or A j A i. Theorem 5 Consider the value function J (x) in (8). Let P i, P j be two neighboring polyhedra then Q i Q j or Q i Q j and Q i Q j () and Q i Q j iff A i A j () Proof: The proof is given in []. We recall a property of convex piecewise quadratic functions [9]: Proposition Consider the value function J (x) in (8) satisfying () and its quadratic expression q i (x) and q j (x) on two neighboring polyhedra P i, P j with common boundary a x = b. Then there exist constants γ R/{} and b such that q i (x) =q j (x)+(a x b)(γa x b) () Equation () states that the neighboring expressions q i (x) andq j (x) of a CPWQ function satisfying () either intersect on two parallel hyperplanes a x = b and γa x = b if b γb (see Figure (a)), or they are tangent in one hyperplane a x = b if b = γb (see Figure (b)). Theorem 6 Under the hypothesis of Theorem,the value function J (x) in (8) is C (). i Proof: We will prove that b and b in () satisfy b = γb by contradiction.suppose there exists i {,...,N r } and j C i such that b γb.without loss of generality we assume that (i) Q i Q j and (ii) P i is on the side a x b of the common boundary.let F ij be the common facet between P i and P j and F ij its interior. From (i) and from (), γ<andγ =iffq i Q j =. Take x F ij, for sufficiently small ε, the point x = x aε belongs to P i. Let J (ε) =J (x aε) then q i (ε) =q j (ε)+(a aε)(γa aε ( b γb)) () From convexity of J (ε), J (ε) J + (ε)wherej (ε) and J + (ε) are the left and right derivatives of J (ε). This implies q j (ε) q i (ε) whereq j (ε) andq i (ε) arethe derivatives of q j (ε) andq i (ε).from () the latter is true if and only if (a a)( b γb)) γ(a a) ε,that implies b <γbsince γ<andε>. From () q i (ε) <q j (ε) for all ε (, b γb γa a ). Thus there exists x P i with q j (x) <q i (x).this is a contradiction since from Theorem, A i A j. Note that in case of degeneracy the value function J (x) in (8) is not C (), counterexamples are given in [5]. Combining Theorem 6 and (), we obtain q i (x) =q j (x)+γ(a x b) () Theorem 7 Consider the value function J (x) in (8). Let m(x) J (x),be the gradient vector of J (x), i.e., m(x) ={m i (x) q i (x) =Q i x+t i x P i,i=,...,n r },where m : x R n m(x) R n satisfies the following properties (i) m(x) is continuous; (ii) w R n such that for each i and j with P i, P j neighboring polyhedra gj i(x) w (m i (x) m j (x)) for any x which is not an element of their common hyperplane. Proof: (i) Follows easily from Theorem 6.(ii) Consider two neighboring polyhedra P i, P j and their common hyperplane a x = b, a.from equation () we get gj(x) i =γw a(a x b). (5) Since γ,a,anda x b it follows that gj i(x) = if and only if w a = (i.e., vectors w and a are orthogonal).in order to have gj i (x) forall neighboring polyhedra P i and P j, the vector w must not be parallel to any of the common hyperplanes.as the number of common hyperplanes defining the polyhedral partition is finite, such a vector w exists and can be easily computed. It follows from Theorem 7 that an appropriate PWA descriptor function f(x) can be calculated from the gradient of the value function.it is sufficient to find off-line

6 a vector w that is not parallel to any hyperplane of the polyhedral partition.then f i (x) = A i x + B i, where A i =w Q i and B i = w T i. Remark Note that Algorithm applies also to the LP case by using the value function J (x) in () as descriptor function.algorithm is less simple than Algorithm but it requires the evaluation of N r linear functions only in the worst case. Example We compare the performance of Algorithm, Algorithm and Algorithm on an CFTOC problem for the linear system y(t) = u(t), (6) s or its equivalent discrete-time (T s = ) state-space representation x(t +) = x(t)+.5 u(t) [ ] y(t) =.8... x(t) (7) subject to the input constraint u(k) (8) and the output constraint y(k) (9). Explicit MPC based on LP To regulate (7), we design an MPC controller based on the optimization problem () where p =, N =, Q = diag{[5 ]}, R =.8, P =.The explicit solution of the mp-lp problem consist of a polyhedral partition of the state-space consist in 6 regions.in Table we report the comparison between the complexity of Algorithm and Algorithm for this example. The average on-line MPC computation for a set of random points in the state space is 59 flops (Algorithm ), and 88 flops (Algorithm ).The implicit solution with Matlab Optimization Toolbox LP solver takes 559 flops on average. Table : Complexity comparison of Algorithm andalgorithm for example in Section. Algorithm Algorithm Storage (real numbers) Flops (worst case) Explicit MPC based on QP To regulate (7), we design an MPC controller based on the optimization problem () where p =,N =7, Q = diag{[]}, R =., P =.The explicit solution of the mp-qp problem consists of a polyhedral partition of the state-space consist in regions.for this example the choice of w =[] is satisfactory.in Table we report the comparison between Algorithm and Algorithm for this example. The average on-line MPC computation for a set of random points in the state space is flops (Algorithm ), and 75 flops (Algorithm ).The implicit solution with Matlab s QP solver takes 5 flops on average. Table : Complexity comparison of Algorithm andalgorithm for example in Section. Algorithm Algorithm Storage (real numbers) Flops (worst case) Conclusion By exploiting the properties of the value function of MPC, we presented two algorithms that significantly improve the efficiency of the on-line explicit MPC (both based on LP and QP) in terms of storage demands and computational complexity.the following improvements are achieved.there is no need to store the polyhedral partition of the state space for computing the optimal control law..in the worst case, the optimal control law is computed after the evaluation of one linear function per polyhedron. References [] A. Bemporad, F. Borrelli, and M. Morari. Explicit solution of constrained / -norm model predictive control. In Proc. 9th IEEE Conf. on Decision and Control, December. [] A. Bemporad, M. Morari, V. Dua, and E. N. Pistikopoulos. The explicit linear quadratic regulator for constrained systems. Automatica, toappear. [] F. Borrelli, M. Baotic, A. Bemporad, and M. Morari. Efficient on-line computation of explicit model predictive control. Technical Report AUT-5, Automatic Control Laboratory, ETH Zurich, Switzerland, August. [] F. Borrelli, A. Bemporad, and M. Morari. A geometric algorithm for multi-parametric linear programming. Technical Report AUT-6, Automatic Control Laboratory, ETH Zurich, Switzerland, February. [5] T. Gal. Postoptimal Analyses, Parametric Programming, and Related Topics. de Gruyter, Berlin, nd edition, 995. [6] D.Q. Mayne, J.B. Rawlings, C.V. Rao, and P.O.M. Scokaert. Constrained model predictive control: Stability and optimality. Automatica, 6(6):789 8, June. [7] P. Tøndel, T.A. Johansen, and A. Bemporad. An algorithm for multi-parametric quadratic programming and explicit MPC solutions. In Proc. th IEEE Conf. on Decision and Control, December. [8] S.J. Qin and T.A. Badgwell. An overview of industrial model predictive control technology. In Chemical Process Control -V, volume 9, no. 6, pages 56. AIChE Symposium Series - American Institute of Chemical Engineers, 997. [9] M. Schechter. Polyhedral functions and multiparametric linear programming. Journal of Optimization Theory and Applications, 5():69 8, May 987.