Numerical Resolution of optimal control problems by a Piecewise Linear continuation method

Transcription

1 Numerical Resolution of optimal control problems by a Piecewise Linear continuation method Pierre Martinon november 25

2

3 Shooting method and homotopy

4 Introduction We would like to solve numerically the following problems: Low thrust orbital transfer problem (bang-bang control, huge number of revolutions) Problems with singular arcs Direct methods (state / control discretization) Non linear optimisation problem Not a good choice for a large number of commutations a priori. Indirect methods (Necessary conditions) Based on Pontriaguine s Maximum Principle. Fast and accurate in favorable cases.

5 Shooting method Optimal control problem Non linear equation system Starting Problem (P) Boundary Value Problem (BVP) Initial Value Problem (IVP) Shooting function S Solve S(z) = Find a solution of (P)

6 6 Difficulties of the studied problems Orbital transfer: S is not smooth The shooting function is not differentiable (even not defined) everywhere. We need a good starting point for the simple shooting convergence. Singular arcs: S is set valued Multiple shooting: requires some knowledge of the control structure (number of singular arcs typically). How to solve these difficulties? We use a homotopic method to obtain the required information (initial point and control structure), with no a priori knowledge about the solution of the problem.

7 Homotopy and zero path Parametrize the problem (P) with λ [, ]: Family of problems (P λ ) such that - (P ) is simple enough to solve. - (P ) is the original problem (P). We define the homotopy H by H : (z, λ) S λ (z) Homotopic method (continuation) - Start from a known zero z of H(, ) - Follow the zero path of H until we reach λ = - Then we have a zero z of H(, ) = S = S.

8 8 Convergence results (cf J.Gergaud) Homotopy convergence We denote y = (x, p) R n the state-costate pair. H : [a, b] R n U R continuous and convex with respect to u. Let Γ(t, x, p) be the set of the solutions of min u U H(t, x, p, u). Under the correct assumptions (in particular Γ has compact convex nonempty values and is usc), from every sequence of solutions (y λk ) of (BVP) λk (λ k ), we can extract a subsequence (y k ) satisfying: (i) (y k ) converges uniformly to y solution of (BVP). (ii) (ẏ k ) converges weakly-* to ẏ in L n ([, t f ]). et (iii) (u k ) converges weakly-* in L n ([, t f ]).

9 Simplicial algorithm

10 Principle: piecewise linear approximation of the path We consider a homotopy H : R n+ R n. Simplex and face A simplex is the convex hull of n + 2 affinely independant points of R n+. A k-face is the convex hull of k vertices of a simplex (face for k = n + ). Triangulation A triangulation is a countable family T of simplices of R n+ satisfying: The intersection of two simplices of T is either a k-face or empty. T is locally finite. Illustration of triangulations K and J 3

11 Vertices labeling We define the labeling l by l(v i ) = H(z i, λ i ), with v i = (z i, λ i ). We define H T by affine interpolation over the vertices of the simplices of T. Completely labeled face A face [v,.., v n+ ] is completely labeled iff it contains a zero of H T, this being true under a certain perturbation (precisely, it contains a solution v ɛ of the equation H T (v) = ɛ for all sufficiently small ɛ >, with ɛ = (ɛ,.., ɛ n )). Fundamental property Every simplex has either zero, or exactly two completely labeled faces.

12 2 General simplicial algorithm Given starting simplex (completely labeled face at λ = ). Repeat until we reach λ = - Determine the second compl. labeled face of the current simplex. - Build the unique simplex sharing this face (pivot rules) new current simplex. End repeat x* x followed zero path zero of homotopy PL approximation completely labeled face transverse simplex Schematic illustration in dimension 2

13 3 Properties The followed path does not cycle It is possible to follow a path with non-maximal rank Possible adaptation to the set-valued case Convergence for a set-valued H (cf Allgower-Georg) We consider a simplicial algorithm using a selection of H as labeling and a refining triangulation of R n [, [. Under the following assumptions: (i) the generated faces remain in K [, ] (K compact). (ii) the algorithm does not go back to λ =. Then if H is usc and convex compact valued, the algorithm generates a sequence (z i, λ i ) such that λ i, and there exists a subsequence that converges to (z, ) such that H(z, ).

14 4 Examples of path following for several triangulations.8 Lambda = X = Simplx: 3.8 Lambda = X = Simplx: 33.8 Lambda = X = Simplx: 32 Lambda.6.4 Lambda.6.4 Lambda Z 2 () Z () Z 2 () Z () Z 2 () Z () Uniform triangulations K, J, D.8 Lambda = X = Simplx:.8 Lambda = X = Simplx: 53 Lambda.6.4 Lambda Z 2 ().55 Z () Z 2 () Z () Refining triangulations J 3 et J 4

15 Junction homotopy Objective: We would like to be able to find a completely labeled face for a given triangulation meshsize, at a certain level λ j. Application: Path following initialisation (first face at λ = ) Change the triangulation during the following Refine the solution at λ = (sequence of triangulations with decreasing meshsize) Principle: Intermediate homotopy at level λ j : we try to connect a well chosen affine application to H(, λ j ), using the wanted meshisze. The choice of the affine application is such that the junction homotopy is easy to initialise.

16 Adaptive triangulation Objective: improve the speed / precision of the path following Refining triangulations? Difficult to use in practice (meshsize is fixed regardless of the path followed) Idea: dynamically adapt the triangulation to the followed path At certain levels λ = λ i during the following: - we determine the new meshsize δ. - we perform a junction homotopy to find a completely labeled face of meshsize δ at the λ i level. - the following continues with the new meshsize until the next level. 2 mechanisms to choose the new meshsize: Deviation control: adapt the meshsize according to the accuracy of the following. Anisotropic deformation: try to take into account the direction of the path.

17 Deviation control Norm of H at the zeros of H T : hint of the accuracy of the following. Depending on the quality of this hint, we uniformly increase or decrease the meshsize Expectations: Illustration on a simple example Speed up the following, without degrading the accuracy too much.

18 8 Anisotropic deformation We estimate the relative weight of each dimension in the path followed since the previous level. We increase (decrease) the meshize along the dimensions in the majority (minority). δ = δ 2 δ 2 2δ 2 δ δ 2 (δ, δ 2 ) ( δ 2, 2δ 2 ) Expectations: Better accuracy, can balance the side effects of the deviation control part.

19 Solution refining Objective: improve the solution at λ =. First idea: refining triangulation before λ = to finish the following Lambda Lambda = X = Simplx: 3.3 Flaw: we encounter the difficulties related to the use of refining triangulations Z 2 () Z () Second idea: perform a sequence of junctions at λ = with triangulations of decreasing meshsize (cf Merril s restart algorithm) Generally effective, but the cost of the junctions is often high.

20 Conclusion The use of junction homotopies allows an easy initialisation of the simplicial algorithm, and performs the meshsize changes for the adaptive triangulation. Concerning the adaptive triangulation, we tried to preserve the robustness of the simplicial algorithm (no derivatives, same numerical settings for all problems)

21 Orbital transfer 2

22 Problem statement Orbital transfer problem (CNES) Initial orbit: ellipse, slight inclination (7 degrees) Final orbit: geostationnary, geosynchronous Criterion: final mass maximisation (payload) We consider low thrust propulsors Difficulties: Mass criterion bang-bang control Low thrust huge number of revolutions

23 23 Problem formulation Forces: central force + propulsor thrust { r = r µ + u r 3 m ṁ = T Max I spg u µ = Gm T Earth gravitationnal constant. T Max the maximal thrust ( u ). I sp the specific impulsion of the propulsor. For the practical resolution (cf T.Haberkorn): - we minimize the fuel consumption. - non cartesian coordinates, form orbital parameters. - integration along longitude instead of time. - final time and longitude are fixed.

24 24 Criterion homotopy Criterion energy to mass We start from a quadratic criterion (minimization of an energy ) to the consumption criterion: J λ = Min tf t λ u(t) + ( λ) u(t) 2 dt. Homotopy: corresponding shooting function for λ [, ] H : (z, λ) S λ (z) λ < : continuous control (Hamiltonian is strictly convex) λ = : bang-bang control (the original problem)

25 The following converges to λ = without any major difficulties. The number of simplices does not seem related to the thrust T Max. The shooting results are consistent with the ones obtained by 25 Path following and results Initialisation at λ = : We perform a shoot with a fixed step integration (Runge Kutta 4). This solution is sufficient for the first junction of the simplicial algorithm. Results T max Initial norm Simplices Time Objective Final norm N N N N N N

26 t f L Transfer for T Max = Newtons Solution at λ = e y P h x m e x h y STATE x x COSTATE CONTROL Evolution of the control along the path CONTROL NORM u λ= λ=.5 λ= TIME

27 RKF45 Integrators comparison: Trajectories for,,.n DOP ODEX

28 Integration stepsize with respect to the thrust T Max For each integrator, we have: Integration steps T Max C te RKF45, DOP835 and ODEX Integration steps vs T Max 5 RKF45 DOP853 ODEX STEPS 4 3 T max (N) Note: logarithmic scale on both axes Confirms the regularity of the problem with respect to T max (we already had t f T Max C te, L f T Max C te )

29 29 Trajectory and stepsize - RKF45 RKF45: embedded Runge Kutta formulas (4,5) RKF45: SWITCHING FUNCTION and INTEGRATION STEPSIZE SWITCH STEPSIZE RKF45: Integrator Stepsize h TIME Time Regular spreading of the dots, 3 values scopes for the stepsize Larger stepsize over the arcs without thrust (only L changes) Extremely small stepsize at the commutations ( 9 )

30 3 Trajectory and stepsize - DOP853 DOP853: embedded Runge Kutta formulas (8,5-3) DOP853: SWITCHING FUNCTION and INTEGRATION STEPSIZE.2 SWITCH STEPSIZE DOP853: Integrator Stepsize h TIME Time Similarity with RKF45 concerning the spreading of the dots Globally, larger stepsize than RKF45 (higher order) Still very small stepsize at the commutations

31 3 Trajectory and stepsize - ODEX ODEX: Gragg-Bulirsch-Stoer extrapolation (variable order) 4 2 ODEX: SWITCH, STEPSIZE AND ORDER SWITCH STEPSIZE ORDER ODEX: Integrator Stepsize 2 Order h TIME Time Less regular spreading of the dots Some very large steps (extrapolation, high order) Again, very small stepsize at the commutations Order is generally high ( 8), decreases at the commutations

32 N UNIFORM AND ADAPTIVE TRIANGULATIONS (K. /.),5N UNIFORM AND ADAPTIVE TRIANGULATIONS (K. /.) 5N UNIFORM AND ADAPTIVE TRIANGULATIONS (K.),2N UNIFORM AND ADAPTIVE TRIANGULATIONS (K. /.) N UNIFORM AND ADAPTIVE TRIANGULATIONS (K. /.) Adaptative triangulation Number of simplices followed along the path (uniform / adaptive) for T Max =, 5,,.5,.2 et.n 25 UNIFORM ADAPTIVE 2 8 UNIFORM ADAPTIVE 8 6 UNIFORM ADAPTIVE SPLX 5 SPLX 2 8 SPLX LAMBDA LAMBDA LAMBDA 35 3 UNIFORM ADAPTIVE 8 6 UNIFORM ADAPTIVE 5,N UNIFORM AND ADAPTIVE TRIANGULATIONS (K. /.) UNIFORM ADAPTIVE SPLX 2 5 SPLX 8 SPLX LAMBDA LAMBDA LAMBDA Less simplices followed (and smoother following) Better final accuracy (faster shooting) BUT: the cost of the junctions can sometimes offset this gain! 32

33 33 Conclusion Resolution of the probleme up to T Max =.N, results are consistent with the ones obtained with a predictor-corrector method (for an execution time 3 to 5 times longer). The 3 integrators tested manage to perform the integration, but the commutations (more than 5 at.n) require extremely small steps ( 9 ). Adaptative mode is qualitatively satisfying (less simplices, more accurate following) but sometimes suffers from the high numerical cost of the junctions.

34 Singular arcs

35 35 Introduction Singular arc: Interval (non reduced to a point) over which the minimisation of H does not determine u uniquely. Differential inclusion: ϕ the dynamic of the state-costate pair y = (x, p). Γ the expression of the optimal control, Γ is set valued and we have { ẏ Φ(y) = ϕ(y, Γ(y)) ae over [t, t (BVP) f ] Boundary conditions Resolution with multiple shooting: Typically requires some knowledge of the singular structure (number, approximate location and nature of the arcs).

36 Problem : optimal fishing (P ) Min ( c x(t) E) u(t) U max dt ẋ(t) = rx(t) ( x(t) u(t) t [, ] x() = 7. 6 x() free k ) u(t) U max E =, c = , r =.7, k = , U max = 2. 6 Switching function and optimal control: ψ(t) = c x(t) E p(t), u (t) = if ψ(t) > u (t) = if ψ(t) < u (t) [, ] if ψ(t) =. Over a singular arc: via ψ = ( ) using = k r c 2 ( c x p) Umax x c k p + 2px k 2px2 k 2

37 Problem 2: quadratic regulator (P 2 ) 5 Min 2 (x 2(t) + x 2 2 (t)) dt x (t) = x 2 (t) x 2 (t) = u(t) u(t) t [, 5] x() = (, ) x(5) free. Switching function and optimal control: { u ψ(t) = p 2 (t), (t) = sign p 2 (t) if ψ(t) u (t) [, ] if ψ(t) =. Over a singular arc: via ψ = u sing (t) = x (t)

38 Homotopy on the criterion: quadratic perturbation Add a quadratic perturbation to the criterion (λ [, ]) We regularize these problems by adding a u 2 term to the criterion c (note: x E < ). (J λ ) Min (J λ 2 ) Min 2 ( c x E ) (u ( λ) u 2 ) U max dt 5 (x 2 + x 2 2 ) + ( λ)u 2 dt λ < : the Hamiltonian is strictly convex, u is a continuous function. λ = : we have the two original problems.

39 Path following Initialisation at λ = without any difficulties, the following begins well. Evolution of the control and switching function? CONTROL EVOLUTION SWITCHING FUNCTION EVOLUTION U.7 ψ TIME TIME (P ): u and ψ for λ =,.5,.75,.9 and.95 We can predict the appearance of a singular arc when λ : - the control u remains non extremal - ψ gets closer to

40 But the following degrades brutally when λ comes closer to Numerical instability when ψ is close to..8 CROSSING TURNING BACK SWITCHING FUNCTION.9 CONTROL EVOLUTION ψ.2 U.6.5 CROSSING TURNING BACK TIME TIME λ = - Two control structures - (P ) Depending on the exit sign of ψ: 2 different bang-bang controls. When λ, 2 structures at the vertices of the simplices. The algorithm converges to λ =, but loses the singular structure. At λ = : Discontinuity of S near the singular arc.

41 Discretized BVP formulation We discretize the equations of the BVP: Euler or Trapezoid formulas ( homotopy is convex valued). (x i, p i ) unknowns, u i obtained by usual NC. Matching conditions at t i : continuity of x, p { xi (x Match i + h x t (t i, x i, p i, ui )) cond p i (p i + h p t (t i, x i, p i, ui )) Note: the discretisation is limited by the problem size. Results Converges without difficulties to λ =. Good approximation of x, p. Detection of the singular structure with ψ. Matching: residual errors related to the u i over the singular arcs.

42 42 Precise resolution Multiple shooting adapted to the singular case Uses the algebraic expression of the singular control u sing. The boundaries of the singular arcs are part of the unknowns. Initialisation with the information provided by the homotopies. 7 x 7 X P U X P.5 U Psi SWITCHING FUNCTION X P2 Psi SWITCHING FUNCTION Accurate solution, quasi instant convergence. We clearly find the suspected singular arcs. Requires a good starting point (correct structure).

43 Quality of the approximate solutions? Comparison of the formulations - Simple shooting (λ =.95 and.925, before the structure loss) - Discretized BVP (5 and 2 points, Trapezoid with refinement) - Reference solution x 7 X P Psi.5..2 CONTROL 5 SWITCHING FUNCTION ψ.3 STRUCTURED SINGLE (λ=.95).4 DISCRETIZED TIME STATE X STATE X COSTATE P COSTATE P2 2 Psi.5.5 CONTROL SWITCHING FUNCTION ψ STRUCTURED SINGLE (λ=.925) DISCRETIZED TIME The discretized solution is quite close (except for singular control...)

44 Discretized control formulation Inspired by direct methods: - the discretized control is part of the unknowns, x, p are integrated. - conditions over the u i : u i Γ(t i, x i, p i ) (needs formalising...). 7 x 7 X P.5 U State x Costate p Control u Psi ψ State x Costate p2 Psi Switching function ψ Discretized control (RK4-5 points) - (P ) and (P 2 ) Good approximation of x, p, ψ. Approximation of the singular control without the algebraic expression. 44

45 Comparison with a direct method Discretization of x and u Optimal control problem non linear optimisation problem Resolution with KNITRO (barrier sub-problems, SQP). 7 x 7 State Control State Control X time U time X X time.5 U.5 5 time time Direct method (RK4 - points) - (P ) and (P 2 ) We find again the singular structure. The solution has some noise over the singular arcs (commutations). 45

46 46 Conclusion The two homotopies (simple shooting and discretized BVP) converge. Simple shooting: guesses the singular structure but loses it at λ =. Discretized BVP formulation: good approximation of x, p and ψ Precise resolution by a multiple shooting method. Discretized control formulation: needs formalising, but interesting first numerical results. Direct method: consistent results for a similar execution time.

47 Conclusions and Prospects

48 48 General conclusions For the problems we studied (orbital transfer and singular arcs), the homotopy allowed us to determine the control structure and obtain a satisfying initial point, with no a priori knowledge. Thanks to this information, we can then solve the problems by the shooting methods. The simplicial algorithm and the shooting methods have been implemented in a single program (Simplicial), used for all numerical experiments (except direct methods). External codes: HYBRD, RKF45 (Shampine/Watts), DOP853 (Hairer/Wanner), ODEX (Hairer/Wanner)

49 49 Prospects Simplicial algorithm: - improve the cost of junctions for adaptative mode and solution refining - introduce speed-up mechanisms (cf Newton / secant) Work on the shooting method itself: - adapt the integration (detection of commutations) - better computation of the derivatives (IND) Further development of the discretized formulations: - more evolved discretisation schemes (symplectic? cf Bonnans/Laurent-Varin) - better formalisation of the discretized control formulation More thorough comparison with direct methods State constraints (formulation, homotopy?)