Value function and optimal trajectories for a control problem with supremum cost function and state constraints

Transcription

1 Value function and optimal trajectories for a control problem with supremum cost function and state constraints Hasnaa Zidani ENSTA ParisTech, University of Paris-Saclay joint work with: A. Assellaou, & O. Bokanowski, & A. Desilles Workshop Numerical methods for Hamilton-Jacobi equations in optimal control and related fields Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

2 Consider the following state constrained control problem: { ϑ(t, y) := inf max θ [0,t] Φ(yu y (θ)) } u U, y u y (s) K, s [0, t] (1) where y u y denotes the solution of the controlled differential system: { ẏ(s) := f (y(s), u(s)), y(0) := y, a.e s [0, t], K is a closed set of R d and U is a set of admissible control inputs. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

3 Hamilton-Jacobi approach How can we handle correctly the state constraints? What boundary conditions should be considered for the HJB equation? Trajectory reconstruction and feedback control law. When Φ 0, we know that ( t 0 ) 1 Φ(yy u (s)) 2p 2p ds max s [0,t] Φ(yu y (s)), as p +. So, it is possible to approximate the maximum running cost problem by a Bolza problem. Does this approximation work well in practice? Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

4 Outline 1 Motivation: Abort landing problem in presence of Wind Shear 2 Optimal control problem with maximum-cost and state constraints 3 Optimal trajectories 4 Numerical simulations Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

5 Outline 1 Motivation: Abort landing problem in presence of Wind Shear 2 Optimal control problem with maximum-cost and state constraints 3 Optimal trajectories 4 Numerical simulations Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

6 Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

7 Abort landing problem in presence of windshear (Miele, Wang and Melvin(1987,1988); Bulirsch, Montrone and Pesch (1991..); Botkin-Turova( )) Consider the flight motion of an aircraft in a vertical plane: where ẋ = V cos γ + w x ḣ = V sin γ + w h V = F T m cos(α + δ) F D m g sin γ (ẇ x cos γ + ẇ h sin γ) γ = 1 V ( F T m sin(α + δ) + F L m g cos γ + (ẇ x sin γ ẇ h cos γ)) ẇ x = wx wx (V cos γ + wx) + x h (V sin γ + w h) ẇ h = w h x (V cos γ + wx) + w h h (V sin γ + w h) and F T := F T (V ) is the thrust force F D := F D (V, α) and F L := F L (V, α) are the drag and lift forces w x := w x (x) and w h := w h (x, h) are the wind components m, g, and δ are constants. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

8 Controlled system Consider the state y(.) = (x(.), h(.), V (.), γ(.), α(.)). The control variable u is the angular speed of the angle of attack α. Let T be a fixed time horizon and let U be the set of admissible controls { } U := u : (0, T ) R, measurable, u(t) U a.e where U is a compact set. The controlled dynamics in this case is: ẋ = V cos γ + w x, ḣ = V sin γ + w h, V = F T m cos(α + δ) F D m g sin γ (ẇ x cos γ + ẇ h sin γ), γ = 1 V ( F T m sin(α + δ) + F L m g cos γ + (ẇ x sin γ ẇ h cos γ)), α = u. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

9 Formulation of the optimal control problem Aim: Maximize the minimal altitude over a time interval: min h(θ) θ [0,t] while the aircraft stays in a given domain K. Consider the following optimal control problem: { (P) : ϑ(t, y) = inf max θ [0,t] Φ(yu y (θ)), } u U, and yy u (s) K, s [0, t] where Φ(y u y (.)) = H r h(.), H r being a reference altitude, and K is a set of state constraints. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

10 Formulation of the optimal control problem Aim: Maximize the minimal altitude over a time interval: min h(θ) θ [0,t] while the aircraft stays in a given domain K. Consider the following optimal control problem: { (P) : ϑ(t, y) = inf max θ [0,t] Φ(yu y (θ)), } u U, and yy u (s) K, s [0, t] where Φ(y u y (.)) = H r h(.), H r being a reference altitude, and K is a set of state constraints. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

11 Outline 1 Motivation: Abort landing problem in presence of Wind Shear 2 Optimal control problem with maximum-cost and state constraints 3 Optimal trajectories 4 Numerical simulations Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

12 A general setting For a given non-empty compact subset U of R k and a finite time T > 0, define the set of admissible control to be, { } U := u : (0, T ) R k, measurable, u(t) U a.e. Consider the following control system: { ẏ(s) := f (y(s), u(s)), a.e s [0, T ], y(0) := y, (2) where u U and the function f is defined and continuous on R d U and that it is Lipschitz continuous w.r.t y, { (i) f : R d U R d is continuous, (ii) L > 0 s.t. (y 1, y 2) R d R d, u U, f (y 1, u) f (y 2, u) L( y 1 y 2 ). The corresponding set of feasible trajectories: S [0,T ] (y) := {y W 1,1 (0, T ; R d ), y satisfies (2) for some u U}, Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

13 A general setting For a given non-empty compact subset U of R k and a finite time T > 0, define the set of admissible control to be, { } U := u : (0, T ) R k, measurable, u(t) U a.e. Consider the following control system: { ẏ(s) := f (y(s), u(s)), a.e s [0, T ], y(0) := y, (2) where u U and the function f is defined and continuous on R d U and that it is Lipschitz continuous w.r.t y, { (i) f : R d U R d is continuous, (ii) L > 0 s.t. (y 1, y 2) R d R d, u U, f (y 1, u) f (y 2, u) L( y 1 y 2 ). The corresponding set of feasible trajectories: S [0,T ] (y) := {y W 1,1 (0, T ; R d ), y satisfies (2) for some u U}, Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

14 State constrained control problem with maximum cost Consider the following state constrained control problem: { ϑ(t, y) := inf max θ [0,t] Φ(yu y (θ)) } u U, y u y (s) K, s [0, t] (3) the cost function Φ( ) is assumed to be Lipschitz continuous and K is a closed set of R d. For every y R d, the set f (y, U) = {f (y, u), u U} is assumed to be convex. The function ϑ is lower semicontinuous (lsc) on K, ϑ + outside K. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

15 State constrained control problem with maximum cost Consider the following state constrained control problem: { ϑ(t, y) := inf max θ [0,t] Φ(yu y (θ)) } u U, y u y (s) K, s [0, t] (3) the cost function Φ( ) is assumed to be Lipschitz continuous and K is a closed set of R d. For every y R d, the set f (y, U) = {f (y, u), u U} is assumed to be convex. The function ϑ is lower semicontinuous (lsc) on K, ϑ + outside K. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

16 State constrained control problem with maximum cost Consider the following state constrained control problem: { ϑ(t, y) := inf max θ [0,t] Φ(yu y (θ)) } u U, y u y (s) K, s [0, t] (3) the cost function Φ( ) is assumed to be Lipschitz continuous and K is a closed set of R d. For every y R d, the set f (y, U) = {f (y, u), u U} is assumed to be convex. The function ϑ is lower semicontinuous (lsc) on K, ϑ + outside K. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

17 Some references Maximum cost problems without state constraints (K = R d ): Barron-Ishii (99) Bolza or Mayer problems with state constraints: Soner (86), Rampazzo-Vinter (89), Frankowska-Vinter (00), Motta (95), Cardaliaguet-Quincampoix-Saint-Pierre (97), Altarovici-Bokanowski-HZ (13), Hermosilla-HZ (15) Maximum cost problems with state constraints: Quincampoix-Serea (02), Bokanowski-Picarelli-HZ (13), Assellaou-Bokanowski-Desilles-HZ (CDC 16), Assellaou-Bokanowski-Desilles-HZ (preprint 16) Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

18 An auxiliary control problem Let g be a Lipschitz continuous function such that: y K, g(y) 0 y K. (4) Consider the following auxiliary control problem : ( w(t, y, z) := inf max Φ(y(θ)) z) ) g(y(θ), y S [0,t] (y) θ [0,t] where a b = max(a, b). Theorem Let (t, y, z) [0, T ] K R. The following assertions hold: (i) ϑ(t, y) z 0 w(t, y, z) 0, { } (ii) ϑ(t, y) = min z R, w(t, y, z) 0. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

19 Define the following Hamiltonian as: ( ) H(y, p) := max f (y, u) p u U y, p R d. Proposition The value function w is the unique Lipschitz continuous viscosity solution of the following Hamilton-Jacobi-Bellman (HJB) equation: ( ) min t w(t, y, z) + H(y, y w), w(t, y, z) Ψ(y, z) = 0 ]0, T ] R d R, w(0, y, z) = Ψ(y, z), R d R, where Ψ(y, z) = (Φ(y) z) g(y). Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

20 A particular choice of function g Let η > 0 and define the following extended set wk: K η : K + B(0, η). g(y) := d K (y) the signed distance to K. Consider the following auxiliary control problem : [ ( w(t, y, z) := inf max Φ(y(θ)) z) ) ] g(y(θ) η, y S [0,t] (y) θ [0,t] where a b = min(a, b). Theorem Let (t, y, z) [0, T ] K R. The following assertions hold: (i) ϑ(t, y) z 0 w(t, y, z) 0, { } (ii) ϑ(t, y) = min z R, w(t, y, z) 0. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

21 A particular choice of function g Theorem The function w is the unique Lipschitz continuous viscosity solution of the following HJB equation: ( ) min t w(t, y, z) + H(y, y w), w(t, y, z) Ψ η (y, z) = 0 ]0, T ] K η R, w(0, y, z) = Ψ η (y, z), K η R, w(t, y, z) = η, y K η, [ where Ψ η (y, z) = (Φ(y) z) g(y)] η. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

22 Link with exit time function Define the exit time function: T (y, z) := sup { t [0, T ] } ϑ(t, y) z = sup { t [0, T ] } w(t, y, z) 0 Link with viability theory: (i) T is the exit time function for Epi(Φ) ( K R d ), (ii) T (y, z) = t w(t, y, z) = 0, (iii) ϑ(t, y) = inf { z T (y, z) t }. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

23 Link with exit time function Define the exit time function: T (y, z) := sup { t [0, T ] } ϑ(t, y) z = sup { t [0, T ] } w(t, y, z) 0 Link with viability theory: (i) T is the exit time function for Epi(Φ) ( K R d ), (ii) T (y, z) = t w(t, y, z) = 0, (iii) ϑ(t, y) = inf { z T (y, z) t }. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

24 Outline 1 Motivation: Abort landing problem in presence of Wind Shear 2 Optimal control problem with maximum-cost and state constraints 3 Optimal trajectories 4 Numerical simulations Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

25 Optimal trajectories Proposition Let y K such that ϑ(t, y) <. Define z := ϑ(t, y). Let ŷ = (y, z ) be the optimal trajectory for the auxiliary control problem associated with the initial point (y, z ) K R. Then, the trajectory y is optimal for the original control problem. Let ŷ = (y, z ) be an optimal trajectory for the exit time problem associated with the initial point (y, z) K R. Then, ŷ is also optimal for the auxiliary control problem. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

26 Reconstruction of optimal trajectories - Algorithm A. For n 1, consider (t 0 = 0, t 1,..., t n 1, t n = T ) a uniform partition of [0, T ] with t = T n. Let {y n ( ), z n ( )} be a trajectory defined recursively on the intervals (t i 1, t i ], with z n ( ) := z = ϑ(0, y) and y n (0) = y. [Step 1] Knowing yk n = yn (t k ), choose the optimal control at t k s.t.: ( uk n arg min w(t k, y n ( k + tf t y n k, u ) ), z) + λ n C(u, t). u U [Step 2] Define u n (t) := u n k, t (t k, t k+1 ] and y n (t) on (t k, t k+1 ] as the solution of ẏ(t) := f (y(t), u n (t)) a.e t (t k, t k+1 ], with initial condition y n (t k ) at t k and z n ( ) := z. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

27 Reconstruction of optimal trajectories - Algorithm A. For n 1, consider (t 0 = 0, t 1,..., t n 1, t n = T ) a uniform partition of [0, T ] with t = T n. Let {y n ( ), z n ( )} be a trajectory defined recursively on the intervals (t i 1, t i ], with z n ( ) := z = ϑ(0, y) and y n (0) = y. [Step 1] Knowing yk n = yn (t k ), choose the optimal control at t k s.t.: ( uk n arg min w(t k, y n ( k + tf t y n k, u ) ), z) + λ n C(u, t). u U [Step 2] Define u n (t) := u n k, t (t k, t k+1 ] and y n (t) on (t k, t k+1 ] as the solution of ẏ(t) := f (y(t), u n (t)) a.e t (t k, t k+1 ], with initial condition y n (t k ) at t k and z n ( ) := z. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

28 Reconstruction of optimal trajectories - Algorithm A. For n 1, consider (t 0 = 0, t 1,..., t n 1, t n = T ) a uniform partition of [0, T ] with t = T n. Let {y n ( ), z n ( )} be a trajectory defined recursively on the intervals (t i 1, t i ], with z n ( ) := z = ϑ(0, y) and y n (0) = y. [Step 1] Knowing yk n = yn (t k ), choose the optimal control at t k s.t.: ( uk n arg min w(t k, y n ( k + tf t y n k, u ) ), z) + λ n C(u, t). u U [Step 2] Define u n (t) := u n k, t (t k, t k+1 ] and y n (t) on (t k, t k+1 ] as the solution of ẏ(t) := f (y(t), u n (t)) a.e t (t k, t k+1 ], with initial condition y n (t k ) at t k and z n ( ) := z. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

29 Theorem Let {y n ( ), z n ( ), u n ( )} be a sequence generated by algorithm A for n 1. Then, the sequence of trajectories {y n ( )} n has cluster points with respect to the uniform convergence topology. For any cluster point ȳ( ) there exists a control law ū( ) such that (ȳ( ), z( ), ū( )) is optimal for the auxiliary control problem. Let w be a numerical approximate solution such that, w (t, y, z) w(t, y, z) E 1 ( t, y), where E 1 ( t, y) 0 as t, y 0. Let {Y n (.), u n (.)} be the sequence generated by the algorithm A with w. Then, (Y n ) n converges to an optimal trajectory for the auxiliary control problem. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

30 Theorem Let {y n ( ), z n ( ), u n ( )} be a sequence generated by algorithm A for n 1. Then, the sequence of trajectories {y n ( )} n has cluster points with respect to the uniform convergence topology. For any cluster point ȳ( ) there exists a control law ū( ) such that (ȳ( ), z( ), ū( )) is optimal for the auxiliary control problem. Let w be a numerical approximate solution such that, w (t, y, z) w(t, y, z) E 1 ( t, y), where E 1 ( t, y) 0 as t, y 0. Let {Y n (.), u n (.)} be the sequence generated by the algorithm A with w. Then, (Y n ) n converges to an optimal trajectory for the auxiliary control problem. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

31 Theorem Let {y n ( ), z n ( ), u n ( )} be a sequence generated by algorithm A for n 1. Then, the sequence of trajectories {y n ( )} n has cluster points with respect to the uniform convergence topology. For any cluster point ȳ( ) there exists a control law ū( ) such that (ȳ( ), z( ), ū( )) is optimal for the auxiliary control problem. Let w be a numerical approximate solution such that, w (t, y, z) w(t, y, z) E 1 ( t, y), where E 1 ( t, y) 0 as t, y 0. Let {Y n (.), u n (.)} be the sequence generated by the algorithm A with w. Then, (Y n ) n converges to an optimal trajectory for the auxiliary control problem. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

32 Theorem Let {y n ( ), z n ( ), u n ( )} be a sequence generated by algorithm A for n 1. Then, the sequence of trajectories {y n ( )} n has cluster points with respect to the uniform convergence topology. For any cluster point ȳ( ) there exists a control law ū( ) such that (ȳ( ), z( ), ū( )) is optimal for the auxiliary control problem. Let w be a numerical approximate solution such that, w (t, y, z) w(t, y, z) E 1 ( t, y), where E 1 ( t, y) 0 as t, y 0. Let {Y n (.), u n (.)} be the sequence generated by the algorithm A with w. Then, (Y n ) n converges to an optimal trajectory for the auxiliary control problem. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

33 Reconstruction of optimal trajectories using the exit time - Algorithm B. For n 1, consider (t 0 = 0, t 1,..., t n 1, t n = T ) a uniform partition of [0, T ] with t = T n. Let {y n ( ), z n ( )} be a trajectory defined recursively on the intervals (t i 1, t i ], with z n ( ) := z = ϑ(t, y) and y n (t 0 ) = y. [Step 1] Knowing yk n = yn (t k ), choose the optimal control at t k s.t.: {( uk n arg max T ( y n ( (t k ) + tf t y n (t k ), u ), z ) ) } + t T, u U [Step 2] Define u n (t) := u n k, t (t k, t k+1 ] and y n (t) on (t k, t k+1 ] as the solution of ẏ(t) := f (y(t), u n (t)) a.e t (t k, t k+1 ], with initial condition y n (t k ) at t k and z n ( ) := z. with initial condition y n (t k ) at t k and z n ( ) := z. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

34 Reconstruction of optimal trajectories using the exit time - Algorithm B. For n 1, consider (t 0 = 0, t 1,..., t n 1, t n = T ) a uniform partition of [0, T ] with t = T n. Let {y n ( ), z n ( )} be a trajectory defined recursively on the intervals (t i 1, t i ], with z n ( ) := z = ϑ(t, y) and y n (t 0 ) = y. [Step 1] Knowing yk n = yn (t k ), choose the optimal control at t k s.t.: {( uk n arg max T ( y n ( (t k ) + tf t y n (t k ), u ), z ) ) } + t T, u U [Step 2] Define u n (t) := u n k, t (t k, t k+1 ] and y n (t) on (t k, t k+1 ] as the solution of ẏ(t) := f (y(t), u n (t)) a.e t (t k, t k+1 ], with initial condition y n (t k ) at t k and z n ( ) := z. with initial condition y n (t k ) at t k and z n ( ) := z. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

35 Reconstruction of optimal trajectories using the exit time - Algorithm B. For n 1, consider (t 0 = 0, t 1,..., t n 1, t n = T ) a uniform partition of [0, T ] with t = T n. Let {y n ( ), z n ( )} be a trajectory defined recursively on the intervals (t i 1, t i ], with z n ( ) := z = ϑ(t, y) and y n (t 0 ) = y. [Step 1] Knowing yk n = yn (t k ), choose the optimal control at t k s.t.: {( uk n arg max T ( y n ( (t k ) + tf t y n (t k ), u ), z ) ) } + t T, u U [Step 2] Define u n (t) := u n k, t (t k, t k+1 ] and y n (t) on (t k, t k+1 ] as the solution of ẏ(t) := f (y(t), u n (t)) a.e t (t k, t k+1 ], with initial condition y n (t k ) at t k and z n ( ) := z. with initial condition y n (t k ) at t k and z n ( ) := z. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

36 Outline 1 Motivation: Abort landing problem in presence of Wind Shear 2 Optimal control problem with maximum-cost and state constraints 3 Optimal trajectories 4 Numerical simulations Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

37 Numerical schemes Finite Difference scheme ( ) W n+1 I,j = max WI n,j + th(y I, D + W n (y I, z j ), D W n (y I, z j )), Ψ I,j WI N,j = Ψ I,j, Semi Lagrangian scheme { W n+1 I,j = min a U (W n( y I + f (y I, a) t, z j ) ) ΨI,j W N I,j = Ψ I,j Same error estimates for both schemes under adequate CFL conditions. Application on the Wind Shear problem (Boeing 727 aircraft model data) Simulations on a grid with N G = nodes (where 30 is the number of points per axis for the first three components, namely, x, h and v, 20 is the number of the points for the angles γ and α an 10 is the number of points for the additional variable z) Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

38 Approximation by Bolza problems ( t 0 ) 1 Φ(yy u (s)) 2p 2p ds max s [0,t] Φ(yu y (s)), as p +. Criterion Optimal cost value Bolza, 2p = Bolza, 2p = Bolza, 2p = Bolza, 2p = Maximum running cost Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

39 Numerical simulations Quite similar optimal trajectories by using either algorithms A and B. However, using the exit time function is much more cheaper (in term of data storage). The control variable enters linearly: f (y, u) = uf 0 (y) + f 1 (y). The Hamiltonian has a simple form (assume u [ 1, 1]): H(x, p) := f 1 (y) p + f 0 (y) p. However, the optimal control strategy may be of Bang-bang type or may include singular arcs when the gradient of the value function is close to 0. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

40 Figure: Reconstruction of the state variables by adding a penalization term λc(u, u n) = u u n, with λ = 0.0, 1.0 and 2.0. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29

41 ... thank you for your attention. Hasnaa Zidani (ENSTA ) Abort landing Problem RICAM, November, / 29