Other course stuff. CSCI 599: Coordinated Mobile Robotics Prof. Nora Ayanian

Transcription

1 Other course stuff CSCI 599: Coordinated Mobile Robotics Prof. Nora Ayanian

2 Syllabus Change $ Grading:$$ $ %$of$final$grade$ Homework$ 0%$ Exercise$$Evaluation$ 5%$ Proposal$Presentation$evaluations$you$write$ 5%$ Evaluations$by$your$classmates$of$your$project$presentations$ 5%$ Online Participation Paper$reviews$ 0%$ Class$Participation$ 5%$ Class$Project$ 40%$ 00%$ Syllabus and reading list to be posted by the evening Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics

3 Exercise : Paper Assignment Preliminary details posted by this evening Make sure you evaluate reading list Which papers are you most interested in? Being assigned these readings is your own responsibility Papers MUST be assigned by the end of class! Your evaluation must include: Which papers are your responsibility? How were the papers assigned? How did the class organize? Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 4

4 Lecture : Discrete Planning CSCI 599: Coordinated Mobile Robotics Prof. Nora Ayanian

5 Planning Formulation State x in state space X X is countable, but mostly finite Actions u applied at x produce x according to f State transition equation: x =f (x,u) Action space U(x), all actions that can be applied from x U = [ U(x) Set of goal states Initial state x xx X G X Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 6

6 State transition graph The set of vertices à state space X. Directed edge from x to x exists in the graph if and only if (iff) there exists (E) 9 an action u U(x) such that (s.t.) x =f (x,u). Initial state and goal set are special vertices in the graph. Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 7

7 Autonomous Recycling Plant Transition Graph / Finite State Machine [Kress-Gazit, Ayanian, Pappas, Kumar, CASE, 008]! Sort recyclables into rooms by composition.! 8

8 Transition Graph Finite State Machine Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 9

9 Forward Search Unvisited, Dead, or Alive States Initially only x is alive How do we sort Q? How does Q terminate? FORWARD SEARCH Q.Insert(x I )andmarkx I as visited while Q not empty do 3 x Q.GetF irst() 4 if x X G 5 return SUCCESS 6 forall u U(x) 7 x f(x, u) 8 if x not visited 9 Mark x as visited 0 Q.Insert(x ) else Resolve duplicate x 3 return FAILURE How can we produce a plan and not just report that a solution exists? Some searches require a cost for each state, which may need to be updated if the state is revisited Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 0

10 Search Types Breadth First: (e.g. FIFO) Frontier grows uniformly, all k-length plans exhausted before k+ plans are considered Running time O( V + E ) V: vertices, E: edges Depth First: (e.g. LIFO) Search dives quickly into the graph, investigate longer plans early Actions applied how they are defined; Risk wasting time Not systematic for infinite X Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics

11 Dijkstra s Algorithm Single-source shortest paths in a graph Edges associated with nonnegative action cost l(e); total cost is sum of edge costs Q sorted according to cost-to-come C : X! [0, ] C * (x) is the optimal cost-to-come from x, from summing edge costs over all possible paths from x to x Cost is computed incrementally: each time x is generated a cost is computed C (x )=C * (x)+l(e) Once x is dead we know it can t be reached with lower cost Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics

12 A* Algorithm Heuristic-based search from a source to a goal C (x) : cost-to-come from x to x G (x) : cost-to-go from x to a state in X G Can t know G* in advance, so underestimate For (i,j) to (i,j ) then i i + j j works The better the estimate, the faster the result MUST be an underestimate Same as Dijkstra but Q is sorted by C (x 0 )+Ĝ (x 0 ) Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 3

13 Best first search & Iterative deepening Best First Search: Q sorted by estimate of optimal cost-to-go Not always optimal In general: Fewest vertices are explored, faster run time Worst case results are worse than A * ID: Converts DFS into a systematic approach Find all states with distance k, if you don t reach the goal, discard and find all states with distance k+; repeat Used for large branching factor Combines DFS space efficiency with BFS completeness Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 4

14 Anytime Heuristic Searches Main Idea: Find a solution quickly, and incrementally improve it as you get more time Finding an optimal path may take a long time, but a suboptimal solution can be found quickly Useful for problem-solving under varying or uncertain time constraints because they have a solution ready whenever they are stopped Quality of solution improves with additional computation time Method: Weight the heuristic by ε. The first solution won t exceed the optimal by a factor greater than +ε E. A. Hansen and R. Zhou (007) "Anytime Heuristic Search", Journal of Artificial Intelligence Research, volume 8, pages Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 5

15 Anytime Search Example Example: Newton-Raphson Method: f 0 (x) =x x = 43 f(x) =x 43 f(x 0 ) x = x 0 f 0 (x 0 ) = f(x ) x = x f 0 (x ) = = 6.6 =.40 f(x ) x 3 = x f 0 (x ) = = f(x 3 ) x 4 = x 3 f 0 (x 3 ) = = You d keep more than 3 decimal places though Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 6

16 Discrete Optimal Planning K-step plan: sequence of K actions Given K,x I, we can derive a sequence of states with f : (x,x,...,x K+ ) Initially x = x I Subsequent states X is finite x k+ = f(x k,u k ) K stages, the exact length of a plan K =(u,u,...,u K ) L stage-additive cost functional, final stage F =K+ L( K )= KX 0, xf X l(x k,u k )+l F (x K ) l F (x K )= G, otherwise k= Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 7

17 Optimal Fixed-Length Plans Principle of optimality: Portions of optimal plans are themselves optimal Iteratively compute optimal cost-to-go or cost-tocome functions over the state space Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 8

18 Backward value iteration Cost that accumulates from stage k to F under optimal plan: Optimal cost-to-go for boundary condition k=f Consider an algorithm that makes K passes over X, each time computing G k from G k +, as k ranges from F to. st iteration: G k(x k )= min u k,...,u K G F (x F )=l F (x F ) G F (x F )=l F (x F ) ( X K ) l(x i,u i )+l F (x F ) i=k nd iteration, compute G K * for each x K in X: G K(x K )=min u K {l(x K,u K )+l F (x F )} G K(x K )=min u K {l(x K,u K )+G F (f(x K,u K ))} Last F - k terms of cost functional F =K+ state transition function: f (x,u) computes costs of all optimal one-step plans from stage K to stage F = K + ( ( )) KX G k(x k )=min min l(x k,u k )+ l(x i,u i )+l F (x F ) u k u k+,...,u K Lecture, /6/4 i=k+ CSCI 599: Coordinated Mobile Robotics 9

19 G k(x k )=min u k G k(x k )=min u k Backward value iteration ( ( min u k+,...,u K ( l(x k,u k )+ l(x k,u k )+ min u k+,...,u K KX i=k+ ( K X i=k+ l(x i,u i )+l F (x F ) l(x i,u i )+l F (x F ) )) )) G k(x k )=min u k l(x k,u k )+G k+(x k+ ) Recurrence for calculating G k * from G * k+ G F! G K! G K G k! G k G! G Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 0

20 e e e Backward value iteration: ex. Figure.: By turning Figure.0 sideway 4 a b c d e X = {a, c, b, d, e}. Suppose K = 4, x = a, and X G = {d}. Thus: Four value iterations, construct G 4*, G 3*, G *, G * once G * 5 is given a b c d e 4 a b c d e All possibilities for advancing forward one stage. Figure.0: The possibilities for advancing forward one stage. This is obtained by making two copies of the states from Figure.8, one copy for thecurrentstate and one for the potential next state. that K =4,xI = a, andxg = {d}. There will hence be four value iterations, which Figure construct.0: The G possibilities 4, G 3, G,andG for advancing,oncethefinal-stagecost-to-go,g 5,isgiven. The cost-to-go 4 forward one stage. This obtained by making two copies functions of the states are shown from Figure in Figure.8,.9. one copy Figures for.0 thecurrentstate and. il- and one for the potential next state. a b c d e Figure.0: The possibilities for advancing 4 forward one stage. This is obtained by making two copies of the states from Figure.8, one copy for thecurrentstate and one for the potential next state. a b c d e Example.3 (A Five-State Optimal Planning Problem) Figure.8 shows agraphrepresentationofaplanningprobleminwhichx a b c d = {a, c, b, d, e}. e Suppose Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics can be drawn that easily shows all of the way initial state by flowing from left to right. The the optimal route. lustrate the computations. For computing G 4 because only they can reach d in one stage. G 4(b) =4andG 4(c) =areimportant. Only lead to d in stage k = 5. Note that the minim action that produces the lowest total cost wh stage..3.. Forward value iteration The ideas from Section.3.. may be recycle lent method that computes optimal cost-to-co Whereas backward value iterations were able t states simultaneously, forward value iterations to all states in X. In the backward case, X G a b c d e Figure.8: A five-state example. Each vertex 4 represents a state, and each edge represents an input that can G be 5 applied to the state 0 transition equation to change the state. The a weights on the b G 4 edges represent 4 c l(xk,uk) d e (xk is the originating vertex of.3. the DISCRETE edge). G 3 6 OPTIMAL PLANNING 47 G G Figure.8: A five-state example. a 6 4 b 5 c d 4 e Each vertex 4 represents a state, and each edge represents an input that can G be 5 applied to the state 0 transition equation to change a b c d the Figure state..9: The The weights optimal on cost-to-go the G 4 edges represent functions 4 l(xk,uk) computed e (xk by is backward the originating value vertex iteration. the edge). G 3 6 of G G Figure.8: A five-state example. a 6 4 b 5 c d 4 e It seems convenient that the cost Each of the vertex optimal represents plan can a be state, computed and each so easily, edge but represents how is an the input actual that plan can G extracted? be 5 applied One to the possibility state 0 transition is to store equation the action to change that satisfied the Figure state..9: the The The min weights optimal in (.) on cost-to-go the from G 4 edges every represent functions 4 state, l(xk,uk) and computed at every (xk by is stage. backward the originating Unfortunately, value vertex iteration. the requires edge). O(K X ) storage, G 3 but 6 it can be reduced to O( X ) using the tricks to this of come in Section.3. for the G more 4 general 6 3 case of variable-length plans. G a 6 4 b 5 c d 4 e It seems convenient that the cost of the optimal plan can be computed so easily, but Example how is.3 the (A actual Five-State plan G extracted? 5 Optimal One Planning possibility 0 Problem) is to store Figure the action.8 shows that satisfied agraphrepresentationofaplanningprobleminwhichx Figure.9: the The min optimal in (.) cost-to-go from G 4 every functions 4 state, and computed at every by = stage. {a, backward c, b, Unfortunately, d, e}. value Suppose iteration. requires that this K =4,xI O(K X ) = a, andxg storage, G = 3 but 6 {d}. it can There be reduced will hence to O( X ) be four using value the iterations, tricks to come which in construct Section.3. G 4, G for 3, G the,andg G more 4 general,oncethefinal-stagecost-to-go,g 6 3 case of variable-length plans. 5,isgiven. The cost-to-go functions G are 6 shown 4 in 5 Figure 4.9. Figures.0 and. ilb from every state, c and at every d = It seems convenient that the cost of the optimal plan can be computed so easily, but Example how is.3 the (A actual Five-State plan extracted? Optimal One Planning possibility Problem) is to store Figure the action.8 shows that satisfied agraphrepresentationofaplanningprobleminwhichx Figure.9: the a The min optimal in (.) b cost-to-go from every functions state, c and computed at every d by = stage. {a, backward c, b, Unfortunately, d, e}. e value Suppose iteration. requires that this K =4,xI O(K X ) = a, andxg storage, = but {d}. it can There be reduced will hence to O( X ) be four using value the iterations, tricks to come which in construct Section.3. G 4, G for 3, G the,andg more general,oncethefinal-stagecost-to-go,g case of variable-length plans. 5,isgiven. The cost-to-go 4 functions are shown in Figure.9. Figures.0 and. il- It seems convenient that the cost of the optimal plan can be computed so easily, but Example how is.3 the (A actual Five-State plan extracted? Optimal One Planning possibility Problem) is to store Figure the action.8 shows that satisfied agraphrepresentationofaplanningprobleminwhichx the a min in (.) stage. {a, c, b, Unfortunately, d, e}. e Suppose that this requires K =4,xI O(K X ) = a, andxg storage, = but {d}. it can There be reduced will hence to O( X ) be four using value the iterations, tricks to come which Figure in construct.0: Section The.3. G possibilities 4, G for 3, G the,andg more for advancing general,oncethefinal-stagecost-to-go,g case of variable-length plans. 5,isgiven. The cost-to-go 4 forward one stage. This obtained by making two copies functions of the states are shown from Figure in Figure.8,.9. one copy Figures for.0 thecurrentstate and. il- and one for the potential next state. the state. The weights on the edges represent l(xk,uk) (xk is the originating vertex of the edge)..3. DISCRETE OPTIMAL PLANNING 47 Figure.8: A five-state example. Each vertex 4 represents a state, and each edge represents an input that can be applied to the state transition equation to change.3. DISCRETE OPTIMAL PLANNING 47 case, x I must be fixed. The issue of maintaining feasible solutions a b c d e a b c d e Figure can be initial the op Figure lustrat can becaus initial G 4(b) the op lead to action stage. Figure lustrat can becaus initial G 4(b) lead the op to action.3.. Figure stage. The lustrat id lent can becaus m initial G Where 4(b) lead the states op to action.3.. to all case, stage. x The lustrat id lent Th becaus m G Where 4(b) lead states to action.3.. to all case, stage. x The id Th lent m Where states.3.. to all case, x The id Th lent m Where states to all case, x Th

21 Backward value iteration: ex. 4 a b c d e.3. DISCRETE OPTIMAL PLANNING 47 X = {a, c, b, d, e}. Suppose K = 4, x = a, and X G = {d}. 4 Thus: Four value iterations, construct G 4*, G 3*, G *, G * once G * 5 is given a b c d e a a a a a Figure.8: A five-state example. Each vertex represents a state, and each edge represents an input that can be applied to the state transition equation to change b b b b the state. The weights on the edges represent l(x k,u k )(x k is the originating vertex of the b edge). 48 c 4 c 4 c 4 c 4 c a b c d e G 5 0 G 4 4 G 3 6 G G d d d d d Figure.9: The optimal cost-to-go functions computed by backward value iteration. It seems convenient that the cost of the optimal plan can be computed so easily, e e e e but how e is the actual plan extracted? One possibility is to store the action that satisfied the min in (.) from every state, and at every stage. Unfortunately, Figure.: GBy * turning Figure G *.0 sideways G * and 3 Gcopying * it this K 4 Grequires times, * ao(k X ) graph storage, butitcanbereducedtoo( X ) using the tricks to can be drawn that easily shows all of the ways to arrive at acome finalin 5 statefroman Section.3. for the more general case of variable-length plans. initial state by flowing from left to right. The computations automatically select the optimal route. Example.3 (A Five-State Optimal Planning Problem) Figure.8 shows Principle agraphrepresentationofaplanningprobleminwhichx of optimality: Portions of = {a, optimal c, b, d, e}. Suppose lustrate the computations. For computing G 4, only b and cthat receive K =4,x finite values I = a, andx G = {d}. There will hence be four value iterations, plans because only they can reach d in one stage. For computingwhich are G 3, only construct themselves the values G 4, G 3, G optimal,andg,oncethefinal-stagecost-to-go,g 5,isgiven. G 4(b) =4andG 4(c) =areimportant. OnlypathsthatreachbThe or c can cost-to-go possiblyfunctions are shown in Figure.9. Figures.0 and. il- Lecture, /6/4 lead to d in stage k = 5. Note that CSCI the minimization 599: Coordinated (.) always Mobile chooses Robotics the action that produces the lowest total cost when arriving at a vertex in a the next b c d e stage. 4 Fig can ini the lus be G 4 lea act sta.3 Th

22 A few project ideas Relative localization for a team of robots using RF signals in a cluttered environment Implement navigation in known cluttered environments without external localization Polygonal object caging and transport without force closure and without updated knowledge of the object s position Implement mobile ad-hoc networks between robots Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 4

23 A few project ideas Lecture, /6/4 CSCI 599: Coordinated Mobile Robotics 5