CS 387/680: GAME AI PATHFINDING

Transcription

1 CS 387/680: GAME AI PATHFINDING 4/14/2014 Instructor: Santiago Ontañón TA: Alberto Uriarte office hours: Tuesday 4-6pm, Cyber Learning Center Class website:

2 Reminders Check BBVista site for the course regularly Also: Next week is the deadline for people doing Project 1. Submission will be via learn.drexel.edu Submission page will be available shortly Any questions about Project 1?

3 Outline Student Presentations Pathfinding Basics: Breadth First Search, A* Pathfinding under Time Constraints: TBA*, LRTA* Pathfinding in Dynamic Domains: D*, D* Lite, AD* From Level Geometry to Pathfinding Graphs Pathfinding and Movement

4 Outline Student Presentations Pathfinding Basics: Breadth First Search, A* Pathfinding under Time Constraints: TBA*, LRTA* Pathfinding in Dynamic Domains: D*, D* Lite, AD* From Level Geometry to Pathfinding Graphs Pathfinding and Movement

5 Pathfinding Problem: Finding a path for a character/unit to move from point A to point B A B One of the most common AI requirements in games Specially critical in RTS games since there are lots of units

6 Pathfinding Problem: Finding a path for a character/unit to move from point A to point B A B One of the most common AI requirements in games Specially critical in RTS games since there are lots of units

7 Pathfinding Simplest scenario: Single character Non-real time Grid Static world Solution: A* Complex scenario: Multiple characters (overlapping paths) Real time Continuous map Dynamic world

8 Pathfinding is a Problem! Even in modern commercial games: v=lw9g-8glo0&feature=player_embedded

9 Example Wall Reach from Unit can move: up, down, left, right

10 Example: Breadth First Search OPEN = [] CLOSED = [] A search algorithm explores nodes in the search space until the goal is found Different algorithms explore nodes in different orders At each point in time, two lists of nodes: Explored nodes (CLOSED) Candidates to be explored (OPEN) OPEN CLOSED

11 Example: Breadth First Search OPEN = [A,B,C] CLOSED = [] A B C A B When a node is explored (expanded), all its successors (children) are added to the OPEN list C

12 Example: Breadth First Search OPEN = [B,C,D] CLOSED = [,A] A B C D A D B C

13 Example: Breadth First Search A B C I draw this tree here just for illustration purposes. In a real implementation, no such tree is kept in memory D A D B C Open: B C D In a real implementation, there are just the two lists of OPEN and CLOSED nodes. Each node stores who was its parent Closed: A

14 Example: Breadth First Search OPEN = [C,D,E] CLOSED = [,A,B] A B C D A D E B Breadth First Search explores nodes in the same order they were added to the OPEN list: Nodes that are closer to are always explored before nodes that are further. E C

15 Example: Breadth First Search OPEN = [D,E,F,G] CLOSED = [,A,B,C] A B C D A D E F G B E C F G

16 Example: Breadth First Search OPEN = [E,F,G] CLOSED = [,A,B,C,D] A B C D A D E F G B E C F G

17 Example: Breadth First Search OPEN = [F,G,H] CLOSED = [,A,B,C,D,E] A B C D A D E F G B H E C F H G

18 Example: Breadth First Search OPEN = [G,H,I,J] CLOSED = [,A,B,C,D,E,F] A B C D A D E F G B H I J E C F J H G I

19 Example: Breadth First Search OPEN = [H,I,J] CLOSED = [,A,B,C,D,E,F,G] A B C D A D E F G B H I J E C F J H G I

20 Example: Breadth First Search OPEN = [I,J] CLOSED = [,A,B,C,D,E,F,G,H] A B C D A D E F G B H I J E C F J H G I

21 Example: Breadth First Search OPEN = [J,K] CLOSED = [,A,B,C,D,E,F,G,H,I] A B C D A D E F G B H I J E C F J K H G I K

22 Example: Breadth First Search OPEN = [K,L] CLOSED = [,A,B,C,D,E,F,G,H,I,J] A B C D A D E F G B L H I J E C F J K L H G I K

23 Example: Breadth First Search OPEN = [L] CLOSED = [,A,B,C,D,E,F,G,H,I,J,K] A B C D A D E F G B L H I J E C F J K L H G I K

24 Example: Breadth First Search OPEN = [] CLOSED = [,A,B,C,D,E,F,G,H,I,J,K,L] A B C D A D E F G B L H I J E C F J K L H G I K

25 Example: Breadth First Search OPEN = [] CLOSED = [,A,B,C,D,E,F,G,H,I,J,K,L,] A B C D A D E F G B L H I J E C F J K L H G I K

26 Example: Breadth First Search OPEN = [] CLOSED = [,A,B,C,D,E,F,G,H,I,J,K,L,] A B C D A D E F G B L H I J E C F J K L H G I K

27 Example: Breadth First Search OPEN = [] CLOSED = [,A,B,C,D,E,F,G,H,I,J,K,L,] A B C D A D E F G B L H I J E C F J K L H G I K

28 Breadth First Search OPEN = [] CLOSED = [] WHILE OPEN is not empty N = OPEN.removeFirst() IF goal(n) THEN RETURN path to N CLOSED.add(N) FOR all children C of N that are not in CLOSED nor OPEN: C.parent = N OPEN.add(C) ENDFOR ENDWHILE

29 Breadth First Search It will find the shortest paths for pathfinding But High computational cost: will explore too many cells of the map Time Cost: Number of expanded nodes Memory Cost: Number of expanded nodes

30 A* Heuristic search algorithm: Finds the shortest path between a given start node and a goal Uses a heuristic to guide the search Heuristic: h(n) -> estimation of the distance from n to the goal Complete: if there is a solution, A* finds one Optimal: if heuristic admissible, A* finds the optimal path A heuristic is admissible if it never overestimates the distance to the goal In Pathfinding: Euclidean or Hamming distances are admissible heuristics

31 A* Heuristic search algorithm: Finds the shortest path between a given start node and a goal Uses a heuristic to guide the search Heuristic: h(n) -> estimation of the distance from n to the goal For pathfinding, the two most typical heuristics are the Euclidean distance and the Manhattan distance. Complete: if there is a solution, A* finds one Optimal: if heuristic admissible, A* finds the optimal path A heuristic is admissible if it never overestimates the distance to the goal In Pathfinding: Euclidean or Hamming distances are admissible heuristics

32 Example: A* OPEN = [] CLOSED = [] Heuristic used: Manhattan Distance

33 Example: A* f? OPEN = [] CLOSED = [] Assigns an estimated cost, f, to each node: f(n) = g(n) + h(n) g =? h =? Heuristic Real cost from to n Heuristic used: Manhattan Distance

34 Example: A* 3 OPEN = [] CLOSED = [] Assigns an estimated cost, f, to each node: f(n) = g(n) + h(n) g = 0 Heuristic Real cost from to n Heuristic used: Manhattan Distance

35 Example: A* 3 OPEN = [A,B,C] CLOSED = [] A f? B f? C f? B g =? h =? A g =? h =? g = 0 C g =? h =?

36 Example: A* 3 OPEN = [A,B,C] CLOSED = [] A 3 B C B h = 4 A h = 2 g = 0 Expands the node with the lowest Estimated cost first C h = 4

37 Example: A* 3 OPEN = [B, C, D] CLOSED = [, A] A 3 D B C D B h = 4 A h = 2 g = 0 C h = 4

38 Example: A* 3 OPEN = [C, D, E] CLOSED = [, A, B] A 3 D B E 7 C D B h = 4 E h = A h = 2 g = 0 C h = 4

39 Example: A* 3 OPEN = [D, E, F, G] CLOSED = [, A, B, C] A 3 D B E 7 C F G 7 D B h = 4 E h = A h = 2 g = 0 C h = 4 F G h =

40 Example: A* 3 OPEN = [E, F, G] CLOSED = [, A, B, C, D] A 3 D B E 7 C F G 7 D B h = 4 E h = A h = 2 g = 0 C h = 4 F G h =

41 Example: A* 3 OPEN = [E, G, I, J] CLOSED = [, A, B, C, D, F] A 3 D B E 7 C F I 7 G 7 J D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 J g = 3 h = 2

42 Example: A* 3 OPEN = [E, G, I, K, L] CLOSED = [, A, B, C, D, F, J] A 3 D B E 7 C F I 7 K 7 G 7 J L D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 L g = 4 h = 1 J g = 3 h = 2 K g = 4

43 Example: A* 3 OPEN = [E, G, I, K, ] CLOSED = [, A, B, C, D, F, J, L] A 3 D B E 7 C F I 7 K 7 G 7 J L D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 g = h = 0 L g = 4 h = 1 J g = 3 h = 2 K g = 4

44 Example: A* 3 OPEN = [E, G, I, K] CLOSED = [, A, B, C, D, F, J, L, ] A 3 D B E 7 C F I 7 K 7 G 7 J L D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 g = h = 0 L g = 4 h = 1 J g = 3 h = 2 K g = 4

45 Example: A* 3 OPEN = [E, G, I, K] CLOSED = [, A, B, C, D, F, J, L, ] A 3 D B E 7 C F I 7 K 7 G 7 J L D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 g = h = 0 L g = 4 h = 1 J g = 3 h = 2 K g = 4

46 A*.g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED nor OPEN: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE

47 A*.g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() Differences wrt Breadth First Search IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED nor OPEN: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE

48 A*.g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE Nodes in red are in CLOSED Nodes in grey are in OPEN

49 A*.g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE g = 0 Nodes in red are in CLOSED Nodes in grey are in OPEN

50 A*.g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE g = 0 Nodes in red are in CLOSED Nodes in grey are in OPEN

51 A*.g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE g = 0 OPEN = [] Nodes in red are in CLOSED Nodes in grey are in OPEN

52 A*.g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE g = 0 OPEN = [] CLOSED = [] Nodes in red are in CLOSED Nodes in grey are in OPEN

53 A*.g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE g = 0 OPEN = [] CLOSED = [] Nodes in red are in CLOSED Nodes in grey are in OPEN

54 A* N OPEN = [] CLOSED = [].g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE g = 0 Nodes in red are in CLOSED Nodes in grey are in OPEN

55 A* N OPEN = [] CLOSED = [].g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE g = 0 Nodes in red are in CLOSED Nodes in grey are in OPEN

56 A* N OPEN = [] CLOSED = [].g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE g = 0 Nodes in red are in CLOSED Nodes in grey are in OPEN

57 A* children(n) N OPEN = [] CLOSED = [].g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE g = 0 Nodes in red are in CLOSED Nodes in grey are in OPEN

58 A* children(n) N OPEN = [] CLOSED = [] M.g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE B A g = 0 C Nodes in red are in CLOSED Nodes in grey are in OPEN

59 A* children(n) N OPEN = [] CLOSED = [] M.g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE B A g = 0 C Nodes in red are in CLOSED Nodes in grey are in OPEN

60 A* children(n) N OPEN = [] CLOSED = [] M.g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE B A h = 2 g = 0 C Nodes in red are in CLOSED Nodes in grey are in OPEN

61 A* children(n) N OPEN = [A] CLOSED = [] M.g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE B A h = 2 g = 0 C Nodes in red are in CLOSED Nodes in grey are in OPEN

62 A* children(n) N OPEN = [A,B] CLOSED = [].g = 0;.h = heuristic() OPEN = []; CLOSED = [] M WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE B h = 4 A h = 2 g = 0 C Nodes in red are in CLOSED Nodes in grey are in OPEN

63 A* children(n) N OPEN = [A,B,C] CLOSED = [].g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N M CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE B h = 4 A h = 2 g = 0 C h = 4 Nodes in red are in CLOSED Nodes in grey are in OPEN

64 A* N OPEN = [A,B,C] CLOSED = [].g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE B h = 4 A h = 2 g = 0 C h = 4 Nodes in red are in CLOSED Nodes in grey are in OPEN

65 A* N OPEN = [A,B,C] CLOSED = [].g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE B h = 4 A h = 2 g = 0 C h = 4 Nodes in red are in CLOSED Nodes in grey are in OPEN

66 A*.g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE B h = 4 A h = 2 g = 0 C h = 4 OPEN = [A,B,C] CLOSED = [] Nodes in red are in CLOSED Nodes in grey are in OPEN

67 A* N OPEN = [B,C] CLOSED = [].g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE B h = 4 A h = 2 g = 0 C h = 4 Nodes in red are in CLOSED Nodes in grey are in OPEN

68 A* N OPEN = [B,C] CLOSED = [].g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE B h = 4 A h = 2 g = 0 C h = 4 Nodes in red are in CLOSED Nodes in grey are in OPEN

69 A* N OPEN = [B,C] CLOSED = [,A].g = 0;.h = heuristic() OPEN = []; CLOSED = [] WHILE OPEN is not empty N = OPEN.removeLowestF() IF goal(n) RETURN path to N CLOSED.add(N) FOR all children M of N not in CLOSED: M.parent = N M.g = N.g + 1; M.h = heuristic(m) OPEN.add(M) ENDFOR ENDWHILE B h = 4 A h = 2 g = 0 C h = 4 Nodes in red are in CLOSED Nodes in grey are in OPEN

70 Implementation Notes Remember the distinction between state and node State: The configuration of the problem (e.g. coordinates of a character) Node: A state plus: current cost (g), current heuristic (h), parent node, action that got us here form the parent node It is important to remember who was the parent, and which action, so that once the solution is found, we can reconstruct the path

71 A* Optimal: finds the shortest paths for pathfinding With an admissible heuristic Faster than Breath First Search: Explores less cells of the map (same worst-case scenario though) Time Cost: Number of expanded nodes Memory Cost: Number of expanded nodes

72 A* The heuristic biases the search of the algorithm towards the goal: Breadth First Search No bias

73 A* The heuristic biases the search of the algorithm towards the goal: A* Breadth First Search No bias Biased towards the goal

74 Outline Student Presentations Pathfinding Basics: Breadth First Search, A* Pathfinding under Time Constraints: TBA*, LRTA* Pathfinding in Dynamic Domains: D*, D* Lite, AD* From Level Geometry to Pathfinding Graphs Pathfinding and Movement

75 TBA* (Time-Bounded A*) Main problem of A* for games is that it is not real-time: What happens if we stop A* in mid execution? Can we get any path?

76 TBA* (Time-Bounded A*) Main problem of A* for games is that it is not real-time: A* needs to run to completion before returning a solution In games, we have a limited amount of time for AI computations per frame, before the unit we are controlling needs to move TBA* is the simplest real-time variant of A*: Given a finite amount of time, returns the best path found so far During the subsequent frames, TBA* continues refining the path The unit can start moving right away, without waiting for A* to complete

77 TBA* Uses A* as a subroutine (executing only NS steps of A* at a time) NS,NT 1 A* Current Path 3 OPEN List CLOSED List Loc Unit Action 2 Path Tracer New Path

78 TBA* Uses A* as a subroutine (executing only NS steps of A* at a time) NS,NT 1 A* 1) Improve the search results 3 Current Path 3) Follow the path OPEN List CLOSED List 2 Loc 2) Compute a new path Unit Action Path Tracer New Path

79 TBA* OPEN = [], CLOSED = [] CurrentPath = [] Loc = WHILE Loc!= DO A*(OPEN,CLOSED,,NS) NewPath = PathTracer(OPEN,CLOSED,Loc, NT) IF Loc in NewPath CurrentPath = NewPath stepforward(currentpath) ELSE stepback(currentpath) IF ENDWHILE

80 TBA* OPEN = [], CLOSED = [] CurrentPath = [] Loc = WHILE Loc!= DO A*(OPEN,CLOSED,,NS) Takes the node in CLOSED with the minimum h value so far and returns the current path to that node NewPath = PathTracer(OPEN,CLOSED,Loc, NT) IF Loc in NewPath CurrentPath = NewPath stepforward(currentpath) ELSE stepback(currentpath) IF ENDWHILE

81 Example: TBA* (NS = 3, NT = 3) 3 CurrentPath = [] NewPath= [] Loc g = 0 OPEN = [] CLOSED = []

82 Example: TBA* (NS = 3, NT = 3) 3 CurrentPath = [] NewPath= [] A 3 B C D A h = 2 D E 7 B h = 4 E h = g = 0 C h = 4 OPEN = [C,D,E] CLOSED = [,A,B] A* is executed for NS = 3 cycles

83 Example: TBA* (NS = 3, NT = 3) 3 CurrentPath = [] NewPath= [,A,D] A 3 B C D A h = 2 D E 7 B h = 4 E h = g = 0 C h = 4 OPEN = [C,D,E] CLOSED = [,A,B] Path Tracer is executed for NT = 3 cycles

84 Example: TBA* (NS = 3, NT = 3) 3 CurrentPath = [,A,D] NewPath= [,A,D] A 3 B C D A h = 2 D E 7 B h = 4 E h = g = 0 C h = 4 OPEN = [C,D,E] CLOSED = [,A,B] Unit moves along the path

85 Example: TBA* (NS = 3, NT = 3) 3 CurrentPath = [,A,D] NewPath= [,A,D] A 3 B C D A h = 2 D E 7 F I 7 OPEN = [E,G,I,J] CLOSED = [,A,B,C,D,F] G 7 J B h = 4 E h = g = 0 C h = 4 G h = F I g = 3 h = 4 J g = 3 h = 2 A* is executed for NS = 3 cycles

86 Example: TBA* (NS = 3, NT = 3) 3 CurrentPath = [,A,D] NewPath= [,A,D] A 3 B C D A h = 2 D E 7 F I 7 OPEN = [E,G,I,J] CLOSED = [,A,B,C,D,F] G 7 J B h = 4 E h = g = 0 C h = 4 G h = F I g = 3 h = 4 J g = 3 h = 2 Path Tracer is executed for NT = 3 cycles (not enough to trace completely)

87 Example: TBA* (NS = 3, NT = 3) 3 CurrentPath = [,A,D] NewPath= [,A,D] A 3 B C D A h = 2 D E 7 F I 7 OPEN = [E,G,I,J] CLOSED = [,A,B,C,D,F] G 7 J B h = 4 E h = g = 0 C h = 4 G h = F I g = 3 h = 4 J g = 3 h = 2 Unit moves along the path

88 Example: TBA* (NS = 3, NT = 3) A 3 D 3 B E 7 OPEN = [E,G,I,K] CLOSED = [,A,B,C,D, F,L,] C F I 7 K 7 G 7 J L CurrentPath = [,A,D] NewPath= [,A,D] D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 g = h = 0 L g = 4 h = 1 J g = 3 h = 2 K g = 4 A* is executed for NS = 3 cycles

89 Example: TBA* (NS = 3, NT = 3) A 3 D 3 B E 7 OPEN = [E,G,I,K] CLOSED = [,A,B,C,D, F,L,] C F I 7 K 7 G 7 J L CurrentPath = [,A,D] NewPath= [,C,F,J] D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 g = h = 0 L g = 4 h = 1 J g = 3 h = 2 K g = 4 Path Tracer is executed for NT = 3 cycles

90 Example: TBA* (NS = 3, NT = 3) A 3 D 3 B E 7 OPEN = [E,G,I,K] CLOSED = [,A,B,C,D, F,L,] C F I 7 K 7 G 7 J L CurrentPath = [,A,D] NewPath= [,C,F,J] D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 Unit moves along the path g = h = 0 L g = 4 h = 1 J g = 3 h = 2 K g = 4

91 Example: TBA* (NS = 3, NT = 3) A 3 D 3 B E 7 OPEN = [E,G,I,K] CLOSED = [,A,B,C,D, F,L,] C F I 7 K 7 G 7 J L CurrentPath = [,A,D] NewPath= [,C,F,J] D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 g = h = 0 L g = 4 h = 1 J g = 3 h = 2 K g = 4 Path Tracer is executed for NT = 3 cycles

92 Example: TBA* (NS = 3, NT = 3) 3 CurrentPath = [,A,D] NewPath= [,C,F,J] A 3 B C D A h = 2 g = h = 0 D E 7 F I 7 K 7 G 7 J L B h = 4 E h = g = 0 C h = 4 G h = F I g = 3 h = 4 L g = 4 h = 1 J g = 3 h = 2 K g = 4 OPEN = [E,G,I,K] CLOSED = [,A,B,C,D, F,L,] Unit moves along the path

93 Example: TBA* (NS = 3, NT = 3) A 3 D 3 B E 7 OPEN = [E,G,I,K] CLOSED = [,A,B,C,D, F,L,] C F I 7 K 7 G 7 J L CurrentPath = [,A,D] NewPath= [,C,F,J,L,] D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 g = h = 0 L g = 4 h = 1 J g = 3 h = 2 K g = 4 Path Tracer is executed for NT = 3 cycles

94 Example: TBA* (NS = 3, NT = 3) A 3 D 3 B E 7 OPEN = [E,G,I,K] CLOSED = [,A,B,C,D, F,L,] C F I 7 K 7 G 7 J L CurrentPath = [,C,F,J,L,] NewPath= [,C,F,J,L,] D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 Unit moves along the path g = h = 0 L g = 4 h = 1 J g = 3 h = 2 K g = 4

95 Example: TBA* (NS = 3, NT = 3) A 3 D 3 B E 7 OPEN = [E,G,I,K] CLOSED = [,A,B,C,D, F,L,] C F I 7 K 7 G 7 J L CurrentPath = [,C,F,J,L,] NewPath= [,C,F,J,L,] D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 Unit moves along the path g = h = 0 L g = 4 h = 1 J g = 3 h = 2 K g = 4

96 Example: TBA* (NS = 3, NT = 3) A 3 D 3 B E 7 OPEN = [E,G,I,K] CLOSED = [,A,B,C,D, F,L,] C F I 7 K 7 G 7 J L CurrentPath = [,C,F,J,L,] NewPath= [,C,F,J,L,] D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 Unit moves along the path g = h = 0 L g = 4 h = 1 J g = 3 h = 2 K g = 4

97 Example: TBA* (NS = 3, NT = 3) A 3 D 3 B E 7 OPEN = [E,G,I,K] CLOSED = [,A,B,C,D, F,L,] C F I 7 K 7 G 7 J L CurrentPath = [,C,F,J,L,] NewPath= [,C,F,J,L,] D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 Unit moves along the path g = h = 0 L g = 4 h = 1 J g = 3 h = 2 K g = 4

98 Example: TBA* (NS = 3, NT = 3) A 3 D 3 B E 7 OPEN = [E,G,I,K] CLOSED = [,A,B,C,D, F,L,] C F I 7 K 7 G 7 J L CurrentPath = [,C,F,J,L,] NewPath= [,C,F,J,L,] D B h = 4 E h = A h = 2 g = 0 C h = 4 G h = F I g = 3 h = 4 Unit moves along the path g = h = 0 L g = 4 h = 1 J g = 3 h = 2 K g = 4

99 TBA* Real-time: Each execution cycle takes at most a constant time function of NS and NT Optimal: Will eventually find the same optimal path found by A* Although the unit might perform some additional movements at the beginning until the algorithm converges to a final path

100 LRTA* (Learning Real-Time A*) Real-Time A* variant that converges to optimal path: Unit can start moving from step 1, but might waste moves Eventually, algorithm converges to the optimal path, and character reaches destination Idea: Learn a better heuristic After each iteration of the algorithm, the heuristic better approximates the real distances to the goal Eventually, it converges to the real distance to the goal Unit just moves to the cell with lowest heuristic around until reaching the goal

101 LRTA* Initializes the heuristic in each cell to h For this example, we will use Manhattan distance

102 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 3 4 f = f = f =

103 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 3 4 f = f = f =

104 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around

105 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 3 f = f =

106 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 3 f = f =

107 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around

108 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 3 4 f = f =

109 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 4 f = f =

110 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around

111 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around f = 1+ 4 f = f =

112 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around f = 1+ 4 f = f =

113 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around

114 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 4 f = f = f =

115 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 4 f = f = f =

116 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around

117 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around f = f =

118 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around f = f =

119 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around

120 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 4 f = f =

121 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 4 f = f =

122 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around

123 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around f = 1+ 4 f = f =

124 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around f = 1+ 6 f = f =

125 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around

126 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 6 f =1+6 6 f =

127 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 7 6 f =1+6 6 f =

128 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around

129 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 7 f = f =

130 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 7 f = f =

131 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around

132 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 7 6 f = f = f =

133 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around 7 6 f = f = f =

134 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around

135 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around f = 1+ 1 f=1+ 4 f= f=1+ 4 3

136 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around f = 1+ 1 f=1+ 4 f= f=1+ 4 3

137 LRTA* In each step: Breadth-first search with bounded depth (e.g. 1) Updates the heuristic of the current node to the best f found Move to the lowest heuristic cell around

138 Other Algorithms: IDA* (Iterative Deepening A*): Like A*, but trades memory by time (only needs memory to store the path, NO open nor closed lists) RIBS (Real-Time Iterative Deepening Best-First Search): Improves on LRTA* by updating several cells at a time, rather than one knn LRTA* (k-nearest Neighbor LRTA*): Useful when pathfinding is executed multiple times in the same map It remembers previous paths, and uses them to better estimate newer paths

139 Outline Student Presentations Pathfinding Basics: Breadth First Search, A* Pathfinding under Time Constraints: TBA*, LRTA* Pathfinding in Dynamic Domains: D*, D* Lite, AD* From Level Geometry to Pathfinding Graphs Pathfinding and Movement

140 Pathfinding in Dynamic Environments A* variants (TBA*, LRTA*, etc.) assume environment doesn t change In a RTS game, units move, new buildings are constructed and destroyed: environment is dynamic What happens if after the game has a path using A*/TBA*/ LRTA*, the map changes?

141 Pathfinding in Dynamic Environments A* variants (TBA*, LRTA*, etc.) assume environment doesn t change In a RTS game, units move, new buildings are constructed and destroyed: environment is dynamic Simple solution (used in many games!): Recompute the path each time something changes (always!) (That s what you are doing for Project 2) Better solution: Specialized algorithms that can take this into account

142 D* Lite Same behavior as D* (which we will NOT cover in class), but simpler Works on dynamic environments: Modeled by changing the cost of moving from one cell to another If one cell n is not reachable from another n, the cost from n to n is infinity Searches from goal to start Uses a consistency test on nodes to determine when more search is needed: At the start When the environment changes

143 D* Lite Maintains an estimate (rhs) of the distance to the goal: A* maintains the g-estimates: actual distance from start rhs-estimate: 1-step look ahead g-estimate rhs(n) =min n0 2Succ(n)(c(n, n 0 )+g(n 0 )) If g(n) = rhs(n), n is considered consistent If all nodes are consistent, finding the optimal path is just moving to the best neighbor Search is only carried out on inconsistent nodes. When the environment changes, nodes become inconsistent

144 D* Lite (Simplified Algorithm) Initialize g and rhs of all nodes to infinity g() = rhs() = 0 U = [] Loc = WHILE Loc!= ComputeShortestPath() Loc = bestneighbor(loc) FOR all cells N with cost changed Update rhs(n) Add to U any inconsistent neighbor of N ENDIF ENDWHILE

145 D* Lite (Simplified Algorithm) Initialize g and rhs of all nodes to infinity g() = rhs() = 0 U = [] Loc = WHILE Loc!= ComputeShortestPath() Loc = bestneighbor(loc) FOR all cells N with cost changed Update rhs(n) U plays the role of the OPEN list in A*. Node consistency plays the role of the CLOSED list in A* Add to U any inconsistent neighbor of N ENDIF ENDWHILE Equivalent to A*, but using U and consistency

146 D* Lite (Full Algorithm) procedure CalculateKey 01 return ; procedure Initialize 02 ; for all ; 0 ; 06 U.Insert CalculateKey ; procedure UpdateVertex 07 if ; 08 if U.Remove ; 09 if U.Insert CalculateKey ; procedure ComputeShortestPath 10 while U.TopKey CalculateKey OR 11 U.TopKey 12 U.Pop ; 13 if CalculateKey 14 U.Insert CalculateKey 1 else if 16 ; 17 for all UpdateVertex ; 18 else 19 ; 20 for all UpdateVertex ; procedure Main Initialize ; 23 ComputeShortestPath ; 24 while 2 /* if then there is no known path */ 26 ; 27 Move to ; 28 Scan graph for changed edge costs; 29 if any edge costs changed for all directed edges with changed edge costs 33 Update the edge cost ; 34 UpdateVertex ; 3 ComputeShortestPath ; This method is equivalent to A*, but using g and rhs (ComputeShortestPath from previous slide) Part shown in the previous slide

147 Example: D* Lite U = [] All g and rhs are initialized to infinity g = rhs = g = rhs = g = 0 rhs = 0 g and rhs initialized to 0 The only inconsistent node is g = rhs = g = rhs = g = rhs = g = rhs = g = rhs = g = rhs = g = rhs = g = rhs = g = rhs = g = rhs =

148 Example: D* Lite U = [] After running ComputeShortestPath, the nodes relevant for the path from to will be consistent g = 9 rhs = 9 g = 8 rhs = 8 g = 8 rhs = 8 g = 7 rhs = 7 g = 0 rhs = 0 rhs = 1 g = 7 rhs = 7 g = 6 rhs = 6 rhs = 2 g = 6 rhs = 6 g = rhs = g = 4 rhs = 4 g = 3 rhs = 3

149 Example: D* Lite U = [] The Unit can now move by choosing the minimum neighbor around g = 9 rhs = 9 g = 8 rhs = 8 g = 8 rhs = 8 g = 7 rhs = 7 g = 0 rhs = 0 rhs = 1 g = 7 rhs = 7 g = 6 rhs = 6 rhs = 2 g = 6 rhs = 6 g = rhs = g = 4 rhs = 4 g = 3 rhs = 3

150 Example: D* Lite U = [A,B,] If something changes, nodes become inconsistent, and are pushed back into U g = 9 rhs = 9 g = 8 rhs = 8 A g = 8 rhs = 8 g = 7 rhs = 7 B g = rhs = g = 0 rhs = 0 rhs = 1 g = 7 rhs = 7 g = 6 rhs = 6 rhs = 2 g = 6 rhs = 6 g = rhs = g = 4 rhs = 4 g = 3 rhs = 3

151 Example: D* Lite U = [] After running ComputeShortestPath, again, nodes are made consistent again. g = 3 rhs = 3 g = 4 rhs = 4 A rhs = 2 g = 3 rhs = 3 B rhs = 1 g = 0 rhs = 0 rhs = 1 No need to replan from scratch, only inconsistent nodes are updated, and their changes propagated g = rhs = g = 6 rhs = 6 g = 4 rhs = 4 g = rhs = g = 4 rhs = 4 rhs = 2 g = 3 rhs = 3

152 Example: D* Lite U = [] The Unit can again move by choosing the minimum neighbor around g = 3 rhs = 3 g = 4 rhs = 4 A rhs = 2 g = 3 rhs = 3 B rhs = 1 g = 0 rhs = 0 rhs = 1 g = rhs = g = 4 rhs = 4 rhs = 2 g = 6 rhs = 6 g = rhs = g = 4 rhs = 4 g = 3 rhs = 3

153 D* Lite D* Lite can handle dynamic environments Doesn t need to replan from scratch As presented it is not real-time (ComputeShortestPath might have to update an arbitrary number of nodes): It can be made real-time by limiting the number of updates ComputeShortestPath performs per cycle

154 Other Algorithms D* Previous to D* Lite Same behavior, but more complex AD* (Anytime Dynamic A*) Tunes quality of paths depending on available time Reuses past paths in new searches to improve their quality Uses a bloating coefficient to find paths faster when little time available (a multiplier of the heuristic)

155 Outline Student Presentations Pathfinding Basics: Breadth First Search, A* Pathfinding under Time Constraints: TBA*, LRTA* Pathfinding in Dynamic Domains: D*, D* Lite, AD* From Level Geometry to Pathfinding Graphs Pathfinding and Movement

156 Waypoints Predefine a collection of waypoints, and do pathfinding in the waypoint graph:

157 Precomputation If environment is largely static paths can be precomputed To reduce the amount of precomputation: Precomputation only in the waypoint graph Precomputation of paths between each possible map location is not feasible At runtime, the precomputed path can be used as a starting point: If environment changes a bit, the path can be modified locally using obstacle avoidance techniques (faster than pathfinding)

158 Quantization and Localization Pathfinding computations are performed with an abstraction over the game map (a graph, composed of nodes and links) Quantization: Game map coordinates -> graph node Localization: Graph node -> Game map coordinates

159 Good and Bad Quantizations The pathfinding graph has to be created carefully, or weird pathfinding problems might occur:

160 Quantization approaches Tile graphs (simplest) Dirichlet Domains (aka Voronoi polygon) Points of visibility Navigation Meshes

161 Tile Graphs Divide the game map in equal tiles (squares, hexagons, etc.) Typical in RTS games

162 Dirichlet Domains Define a set of characteristic points in the game map (typically defined by the level designer) Quantization is just computing the closest characteristic point

163 Dirichlet Domains Define a set of characteristic points in the game map (typically defined by the level designer) Pathfinding graph, corresponding to a Dirichlet domain

164 Points of Visibility The shortest path between two points is a straight line Generate a collection of characteristic points in the map Connect two points if the character can move in a straight line between them Characteristic points can be generated automatically (for example at the corners of the map)

165 Points of Visibility Joint with Dirichlet domains, this method is widely used. However, the resulting graph can be huge!

166 Navigation Meshes (Navmesh) By far the most widely used Use the level geometry as the pathfinding graph Floor is made out of triangles: Use the center of each floor triangle as a charcteristic point

167 Navigation Meshes (Navmesh) One characteristic point is only connected to neighboring ones

168 Navigation Meshes (Navmesh) The validity of the graph depends on the level designer. It is her responsibility to author proper triangles for navigation:

169 Hierarchical Pathfinding If the path that needs to be planned is long, it doesn t make sense to use a standard A* graph Hierarchical graphs

170 Pathfinding Simplest scenario: Single character Non-real time Grid Static world Solution: A* Complex scenario: Multiple characters/dynamic world: D* / D* Lite Real time: TBA* / LRTA* Continuous map: Quantization

171 Outline Student Presentations Pathfinding Basics: Breadth First Search, A* Pathfinding under Time Constraints: TBA*, LRTA* Pathfinding in Dynamic Domains: D*, D* Lite, AD* From Level Geometry to Pathfinding Graphs Pathfinding and Movement

172 Game AI Architecture AI Strategy Decision Making World Interface (perception) Movement

173 Game AI Architecture AI Strategy Decision Making Movement World Interface (perception) Steering behaviors / Movement executed in this module

174 Game AI Architecture AI Strategy Decision Making Movement World Interface (perception) Pathplaning sits in the middle. Sometimes in Movement, sometimes in Decision Making

175 Example Grand Theft Auto

176 Example Grand Theft Auto: Pathfinding graph contains roads and intersections as node graphs Pathfinding plans the roads to take Movement controls the car to actually follow the path (same as in the PTSP project 1) In games where the only decision making is path planning, then pathfinding IS the decision making module In games where decision making involves other things (e.g. when to attack, when to retreat, etc.), pathfinding is just a layer in between

177 Project 2: Pathfinding Implement A* in a RTS Game Game Engine: S3 (Java)

178 Project 1: Steering Behaviors Implement steering behaviors (explained today) Game Engine: PTSP (Java) Competition: (if you are interested)