Robotic Perception and Action - Project. State Machine Part 2. M. De Cecco - Robotic Perception and Action

Size: px

Start display at page:

Download "Robotic Perception and Action - Project. State Machine Part 2. M. De Cecco - Robotic Perception and Action"

Delphia Fields
5 years ago
Views:

1 Robotic Perception and Action - Project State Machine Part 2

2 Project Modules Description Software to use Groups State Machine > Main software that manages the HMI interaction. > Main projected interface. HW: virtual buttons with retroreflective material C# - UNITY Objects Localization with AR Toolkit Skeleton acquisition > Markers placed on the objects extraction (position and location on the projector space). > Calibration of camera/projector/furnitures HW: kitchen simulated furnitures C# - UNITY > Skeleton acquisition with ToF (kinect) > Control of task execution UNITY Animations Objects Localization with ToF camera Sensor Fusion Development of proper anmations (for example with Blender) > Point cloud extraction and its use for objects recognition and localization > Sensor Fusion of Objects locations estimated with Ar Toolkit and the ToF camera UNITY - Blender C++ (PCL) UNITY Project modules to develop

3 We will create four State classes identical in structure as far as they must obey to the IStateBase interface that makes it a requirement that all the required methods are included in each State! Now let s modify BeginState and then create three more states: PlayState, WonState, LostState Creating State Classes

4 BeginState till now

5 BeginState modified!

6 Lets create the other States!

7 This for you shall be the directory State Lets create the other States!

8 In each of the three new files, make them almost identical to the code of BeginState except for the class name, the constructor method name, and the text in the Debug.Log Lets create the other States!

9 class name, the constructor method name, and the text in the Debug.Log Lets create the other States!

10 The same for WonState and LostState! PlayState

11 In order to test the State Machine we will manually switch in Unity when we press the space bar key States

12 States

13 LostState and all the others!

14 but the activestate variable needs to be able to store all of the State types. Right now it can only store a reference to a BeginState type of object!! StateManager

15 Our dilemma right now..

16 Let s see how to solve by an example Let s say that we have a potato bag. This in code would declare: Public Potato bag; So we have a variable named bag that can only store a Potato!!! The issue is shown in the following picture Inheritance

17 Inheritance

18 this is exactly the same issue we have with our activestate variable that is declared to only store a BeginState object!!! So we cannot put a PlayState, a WonState, or a LostState object in activestate How do we solve this issue? Instead of using a very specific Potato bag, what about making the bag a bit more general, like a Food bag??? Inheritance

19 Inheritance

20 Look at each class declaration right below each Food object. Each class inherits properties from the Food class. So, now that we re classifying each object in the more general class of Food, we can now declare that the variable bag can hold a type of Food! And that s done!!!!! Inheritance

21 That great!!! So we can do something similar with our classes such as create a more general StateBase class and then modify each State class to inherit from StateBase! Yes, we could do that. However inheritance does have some limitations: If we use inheritance every method that s inherited will be identical which is not what we want! Using an inherited method is optional If we created a State system using inheritance and lated wished to inherit from another class we cannot! Inheritance

22 we do not have to create another class for the State classes to inherit Each of our States are already implementing the IStateBase interface to guarantee the methods are included Now, since IStateBase acts like it s being inherited, it means each of our States can be treated as if they re an IStateBase type of object What we will have is shown in the following: Inheritance via C# interface!

24 Now, the IStateBase interface is providing a double guarantee that: The State objects have the required methods that StateManager needs The activestate variable, of type IStateBase, requires all States to implement IStateBase (allows to implement all States)

25 StateManager

26 StateManager

The following is the output to the Console as you repeatedly press the Space bar key: The machine starts with BeginState then, pressing the spacebar key makes PlayState the

27 The following is the output to the Console as you repeatedly press the Space bar key: The machine starts with BeginState then, pressing the spacebar key makes PlayState the active State Pressing the spacebar key again makes WonState the active State Pressing the spacebar key again makes BeginState the active State never loses. Output to the Unity console

28 Let us follow the code flow!

29 1 2 On StateManager

30 3 On StateManager

31 1 2 3 On StateManager

32 4 5 6 On BeginState

33 7 On BeginState

34 On BeginState

35 3 On StateManager

36 On StateManager

37 On StateManager

38 On BeginState

39 The PlayState object is instantiated to switch to this new state On BeginState

40 PlayState BeginState StateManager PlayState BeginState - StateManager

41 PlayState

42 14 15 On StateManager

43 On StateManager

44 Summarizing the Code Flow

45 16 Adding LostState into the Game loop..

46 16

47 The IStateBase interface requires that each State class have the ShowIt() method that will also be called from the StateManager Now it s time to connect this method with the StateManager code The StateManager class needs to have Unity s OnGUI() method just the way the UpDate() method transfers control to the StateUpdate() method on the active State, the OnGUI() method transfers control to the ShowIt() method on the active State.. Adding OnGUI to the StateManager

48 17 Adding OnGUI to the StateManager

49 Unity calls the OnGUI() method at least once per frame, but it could also be several times per frame. This means that the ShowIt() method on the active State will be called to display graphics, text and buttons! nobehaviour.ongui.html Adding OnGUI to the StateManager

51 Our State Machine is presently changing States in the only Scene we ve created, which is called for example Scene1 However there is a potential issue that is behind the graphics of Scene1 your game will be active and you won t be able to see the action because of the fullscreen graphic blocking your view. This is not an ideal situation!

52 In Unity

53 In BeginState

54 What happens if you click on Play in Unity? The following screen shot appears with a cube that then falls down: Click Play in Unity

55 suppose your game had many things moving around, and instead of a button, you had a set of many buttons to select various games options You would see nothing behind the set of buttons Nor will you really want all that action to take place at the beginning! i.e. before you switch to the PlayState class! Click Play in Unity

56 But there is a little trick you can use to stop all the game activity while in BeginState Unity has a Time class and a timescale variable which you can set to slow down your game or stop it completely it s similar to a pause feature!

57 This makes the time in the game to Stop!!! 18 This makes the time in the game to Start!!! 19 In BeginState

58 Now, when you Play, everything in the Game is stopped When you press on the button of the interface the button disappears and the game starts!!! This without using many if statements to make things appear or disappear: each State determines the control you want and what to show on the screen!!!

59 Using a Timer to start automatically after some seconds..

60 In BeginState

61 complete with the following code: In BeginState

63 Code flow

64 Code flow

65 Code flow

66 Code flow

67 Code flow

68 Code flow

69 Code flow

70 Till now BeginState appears when the game starts and again after you lose Try to change WonState to switch to PlayState instead of starting with BeginState! A little exercise

71 So far the State Machine is working with one Scene to setup game control Suppose we want to use another scene or simply do not want to use the timer.. Adding several Unity Scenes!

72 Adding several Unity Scenes!

73 ??? Cosa fa il build come si usa?

74 In BeginState

75 We have still two issues: 1. Every GameObject in the current Scene gets destroyed when another Scene is loaded. 1. When the game starts in BeginningScene, the GameManager is created with the StateManager Component and BeginState being active. 2. When BeginState will switch to PlayState, the next scene (Scene1 or whatever you called it) will be loaded and our GameManager will be destroyed 2. Every time BegininngScene is reloaded, another GameManager with another StateManager Component will be created! Managing GameManager through scenes

76 This is simply what we do not want to behave! Throughout the game there needs to be only one GameManager component with one StateManager class!!! So we have to solve the two issues: 1. To prevent GamaManager from being distroyed when switching from BeginningScene to Scene1, Unity provides a method called DontDestroyOnLoad()! 2. To pevent to create GameManager a second time it is enough to detect that already exist and immediately destroying any new ones that are created! Managing GameManager through scenes

77 Both the two issues will be fixed in Unity s Awake() method Awake() is called on any component that invokes Awake() right after a Scene is loaded ss Managing GameManager through scenes

Robotic Perception and Action - Project. Augmented Reality for the Man in the Loop. M. De Cecco - Robotic Perception and Action

Robotic Perception and Action - Project Augmented Reality for the Man in the Loop Jaakko Hyry, Max Krichenbauer, Goshiro Yamamoto, Takafumi Taketomi, Christian Sandor, Hirokazu Kato, Petri Pulli, Design