Temporal logic-based decision making and control. Jana Tumova Robotics, Perception, and Learning Department (RPL)

Transcription

1 Temporal logic-based decision making and control Jana Tumova Robotics, Perception, and Learning Department (RPL)

2 DARPA Urban Challenge

3 Formal verification Does a system meet requirements? System = model Finite transition system, Markov Decision Process, Requirements = specification over system state Safety, propositional formula, temporal logic formula Meet = guarantee that all possible behaviors of the system model satisfy the specification Systematic exploration of the state space Model checking, deductive verification, theorem proving, 3

4 Model checking System + controller Model Behavior specification Temporal logic formula Do all the model executions satisfy the formula? Yes No + counterexample 4

5 Model checking vs. control strategy synthesis System + controller Model Behavior specification Temporal logic formula Which of the controllers ensure satisfaction of the formula? Here is one There is none 5

6 Example: Model checking-based robot mission and motion planning System Behavior specification Periodically visit P 1,P 4,P 8 and never enter P 10 [KFP09] Kress-Gazit, Fainekos, Pappas, Temporal-Logic-Based Reactive Mission and Motion Planning, TRO,

7 Example: Model checking-based robot mission and motion planning System Behavior specification Periodically visit P 1,P 4,P 8 and never enter P 10 System model ṗ(t) =u(t) p(t) 2 P R 2 u(t) 2 U R 2 Linear Temporal Logic (LTL) formula GF P 1 ^ GF P 4 ^ GF P 8 ^ G P 10 p(0) = P 1 [KFP09] Kress-Gazit, Fainekos, Pappas, Temporal-Logic-Based Reactive Mission and Motion Planning, TRO,

8 But? System Model ẋ = f(x, u) x(0) = x init? Discrete Model T =(Q, Q 0, Act,!) Behavior specification Temporal logic formula Which of the controllers ensure satisfaction of the formula?? Here is one Q! Act There is none 8

9 Abstractions Abstractions Cell partitioning and a go from A to B control law Concretizations Sampling-based motion planning Kress-Gazit, Fainekos, Pappas. Temporal-Logic-Based Motion Planning Karaman, Frazzoli. RRT* Discrete trace: The sequence of labels of different regions of interest that the the continuous trajectory goes through 9

10 Model checking-based planning Model ẋ = f(x, u) x(0) = x init Temporal logic formula Abstraction Discrete model T =(Q, Q 0, Act,!) Which of the controllers ensure satisfaction of the formula? Projection Here is one Q! Act 10 There is none

11 Control strategy synthesis Which of the executions satisfies the formula? Transition system LTL formula Translation Parallel composition Buchi automaton Product automaton Graph analysis An accepting run Projection An accepting run does not exists Abort A satisfying execution 11

12 Example: Model checking-based robot mission and motion planning System model ṗ(t) =u(t) p(t) 2 P R 2 u(t) 2 U R 2 p(0) = P 1 Behavior specification Periodically visit P 1,P 4,P 8 and never enter P 10 Model Linear Temporal Logic (LTL) formula GF P 1 ^ GF P 4 ^ GF P 8 ^ G P 10 [KFP09] Kress-Gazit, Fainekos, Pappas, Temporal-Logic-Based Reactive Mission and Motion Planning, TRO,

13 Why temporal logic? Autonomous driving: Getting to a goal while obeying road rules Household robotics: Picking up all dirty cups and bringing them to the kitchen. If seeing a fire, putting out the fire. Surveillance robots: Patrolling regions A,B,C in this order while never entering a dangerous region 13

14 Interesting LTL formulas ' ::= ' ^ ' ' ' U ' X' F' G' Reachability F Safety G Surveillance GF Sequencing 1 U ( 2 U 3 ) F( 1 ^ F 2 ) Response G(request ) F response) 14

15 Challenges: LTL vs. user-friendliness Kress Gazit et al. LTLMoP. Fainekos et al. Graphical Language. 15

16 Challenges: Synthesis vs. reactivity Bhatia, Kavraki, Vardi. Synergistic Approach. 16

17 Challenges: A controller vs. the optimal one Smith, Tumova, Belta, Rus Optimal surveillance. Ding, Belta et al. Receding horizon reward collection. 17

18 Many more research challenges Discretizations (approximate) (bi)simulations sampling-based techniques Computational complexity and scalability receding horizon, fragments of logics Dynamic environments and imprecisions of sensors and actuators nondeterministic, probabilistic, partial observable models reactive re-planning Multi-agent systems task decomposition, decentralized planning Optimality weighted models, timed automata Specifications over continuous time Signal Temporal Logic Specification infeasibility least-violating planning, model repair, analysis of reasons 18

19 Challenges: Specification infeasibility A B Go from A to B and do not hit any obstacles Follow the road rules Do not go in the left lane Do not enter sidewalk 19

20 Dealing with infeasible specifications Reveal the reason why the specification cannot be accomplished (unsatisfiability vs. unrealizability) [RK11, RK12] Repair the model to allow for the specification satisfaction while being close enough to the original one [BGK+11, CHH+13] Repair the specification so that it becomes feasible and close enough to the original specification [F11, KFS12, GD15] Remove some obstacles in the environment [H12] [RK11] Raman, Kress-Gazit, Analyzing unsythesizable specifications for high-level robot behaviors, CAV, [RK12] Raman, Kress-Gazit, Automatic feedback for unachievable high-level robot behaviors, ICRA, [H12] Hauser, The minimum constraint removal problem with three robotics applications, WAFR, [F11] Fainekos, Revising temporal logic specifications for motion planning, ICRA, [KFS12] Kim, Fainekos, Sankaranarayanan, On the revision problem of specification automata, ICRA, [GD15] Guo, Dimarogonas, Multi-agent plan reconfiguration under local LTL specifications, IJRR, [CHH+13] Chen, Hahn, Han, Kwiatkowska, Qu, Zhang, Model Repair for Markov Decision Processes, TASE, [BGK+11] Bartocci, Grosu, Katsaros, Ramakrishnan, Smolka, Model Repair for Probabilistic Systems, TACAS,

21 Dealing with infeasible specifications: Our approach Find the least violating controller [CY98, DF12] Measure the level of violation Find an optimal solution [CY98] Courcoubetis, Yannakakis, Markov decision processes and regular events, IEEE TAC, [DF12] Damm, Finkbeiner, Does it pay to extend the perimeter of a world model?, FM,

22 Challenges: Specification infeasibility A B Go from A to B and do not hit any obstacles Follow the road rules Do not go in the left lane Do not enter sidewalk 22

23 Least-violating robot motion planning System Abstraction Model Behavior specification Temporal logic formula Which of the controllers violates the formula the least? Projection Here is one 23

24 Infeasibility due to environmental constraints Mission: Go from A to B while avoiding obstacles Road rules: LTL formulas over finite runs Penalty for violation per time step 24

25 Infeasibility due to environmental constraints Level of unsafety: the number of steps that need to be ignored to make the run satisfy the formula weighted by the penalties Goal: Find a trace that Leads the robot from A to B while avoiding obstacles Minimizes the level of unsafety among the traces satisfying (1) Is the shortest among the traces satisfying (1) and (2) 25

26 Control strategy synthesis solution Which of the executions violate the formula the least? Model 1 Translation N Translation Parallel composition FA Enhancement Weighted FA FA Enhancement Weighted FA Weighted product automaton Graph analysis The shortest accepting run Projection The least violating execution 26 The runs of the product automaton map to the traces of the transition system The weights along the run of the product automaton determine the level of unsafety The shortest run of the product maps onto the best trace of the model

27 Least-violating robot motion planning System ẋ = f(x, u) x(0) = x init Abstraction Model? Behavior specification Temporal logic formula Which of the executions violates the formula the least? Projection? Here is one 27

28 Least Violating RRT* Incrementally built weighted tree Incremental update of the shortest path Distance is the optimality criterion Incrementally built weighted product automaton Incremental update of the least-violating path Level of violation is the optimality criterion 28

29 Experiments 29

30 References Least-violating Control Strategy Synthesis with Safety Rules in HSCC 2013, with Gavin Hall, Sertac Karaman, Emilio Frazzoli, Daniela Rus Incremental Sampling-based Algorithm for Minimum-violation Motion Planning in CDC 2013, with Luis Reyes-Castro, Pratik Chaudhari, Sertac Karaman, Emilio Frazzoli, Daniela Rus 30

31 Detail of modeling Intricacy of the goal Length of the horizon Task/symbolic planning Motion planning Control 31

32 Detail of modeling Intricacy of the goal Length of the horizon Task/symbolic planning Motion planning Control 32

33 Detail of modeling Intricacy of the goal Length of the horizon Task/symbolic planning Motion planning Control 33

34 Bridging the gap High-level planning Simple models and complex goals Classical AI task planning, PDDL, STRIPS, logics Low-level control Complex systems and simple goals Reachability, stability, What we want: controllers for complex systems with complex specifications 34

35 Signal temporal logic (STL) ' ::= µ ' ' ^ ' ' U [a,b] ' F [a,b] ' G [a,b] ' Interpreted over continuous signals µ = f(x 1 [t],...,x n [t]) > 0 35

36 Example: Signal temporal logic ' = G [2,6] ( x[t] < 2) Thanks to A. Donze 36

37 Signal temporal logic: remarks Even model checking is intractable Some interesting results for control under fragments of STL [LD15] Quantitative semantics: robustness measure [DM13] [LVD15] Lindemann, Verginis, Dimarogonas, Prescribed Performance Control for Signal Temporal Logic Specifications, CDC, [DM13] Donze, Maler, Efficient Robust Monitoring for STL, LNCS,

38 Conclusions Model checking vs. control synthesis Temporal logics Expressive Rich, somewhat user-friendly, rigorous Framework for provable guarantees Quantitative semantics Least-violating motion planning Complex model, complex specification Signal temporal logic Many challenges 38

39 Thank you! 39