OculusUnityMatlab Documentation

Transcription

1 OculusUnityMatlab Documentation Release Luca Donini December 31, 2016

2

3 Contents 1 Another Simple Header Table of contents: Installation Installing Unity Installing MATLAB Getting Started Getting Started: Unity Getting Started: MATLAB Downloads Unity Reference Recorder Save data to file CommunicationWIF Send data to MATLAB Timer - A countdown timer Help Need further help Indices and tables 11 i

4 ii

5 CHAPTER 1 Another Simple Header Here is some text.: using UnityEngine 1.1 Table of contents: Installation Installing Unity Download and install Unity from here. Required Packages The following packages are required: Oculus Utilities for Unity 5: Download from here. Our own data collection package: Refer to the Getting Started section to decide which flavour you need. Installing MATLAB Download and install MATLAB from here. Required Packages Install following packages: Psychtoolbox-3: Install Psychtoolbox-3 by Mario Kleiner from the MATLAB Add-On Explorer or download zip archive from here. Recursive directory listing - Enhanced RDIR, by Thomas Vanaret: Install this package from the MATLAB Add-On Explorer or download zip archive from here. Controllable tight subplot, by Nikolay S.: Install this package from the MATLAB Add-On Explorer or download zip archive from here. 1

6 1.1.2 Getting Started These pages show how to set up a simple project with all the implemented functionalities. Getting Started: Unity The following pages will explain how to configure your Unity project to save and transmit data. Importing the required packages To enable data collection in Unity, you will need to import the following packages. If you already have the OculusUtilities for Unity installed (these include OVRPlayerController, for instance): Download the DATA COLLECTION PACKAGE from HERE and import it into Unity If you haven t installed the OculusUtilities for Unity, you have two options: If you intend to use the latest version of the OculusUtilities for Unity Download and extract Oculus Utilities for Unity 5 (this is in the Engine Integration section of the linked webpage) and then import OculusUtilities.unitypackage into Unity. Download the DATA COLLECTION PACKAGE from HERE and import it into Unity OR, if you intend to use the version of the OculusUtilities that was originally used when implementing the platform (for example if the above doesn t work) Download the DATA COLLECTION PACKAGE WITH OCULUSUTILITIES (SHOULD BE 1.3.2) from HERE and import it into Unity. After having imported either data collection package, you will find several new folders in your Project Explorer, including: CommunicationWIF OVR Plugins Recorder Sample Scene Time Sample scene Before implementing data collection in your project, you might want to become more familiar with it by looking at a sample scene that includes all the existing functionalities. To open the sample scene, navigate to the Sample Scene folder and double click the samplescene file with a Unity icon. 2 Chapter 1. Another Simple Header

7 The Hierarchy view contains several game objects. The MainRecorder GameObject manages the data output to file. The scripts attached to it save general game properties to file (i.e. properties that are not dependent on a single GameObject), such as audio sources, controller input, game information, etc. The VIESYS GameObject manages the communication with MATLAB. The OVRPlayerController > OVRCameraRig > TrackingSpace > CenterEyeAnchor GameObject has a RecordPositionAttach script attached to itself, which saves position and rotation data for this GameObject to file. The OVRPlayerController > OVRCameraRig > TrackingSpace > CenterEyeAnchor GameObject also has a WorldObject script attached to itself, which allows communication with MATLAB. The Sample Audio Source GameObject contains an Audio Source which will generate a sound as the game is started. The Timer GameObject contains a countdown timer. For further information, please visit DOCUMENTATION!!!! 1.1. Table of contents: 3

8 Adding data collection to your own scene Now that you have familiarized yourself with the sample scene, you can implement data collection into your own scene. Save data to file The section below describes how to save data to file. For output data format, please refer to DOCUMENTATION. Main Scripts (other experimental scripts are listed in DOCUMENTATION): MainRecorder RecordAudioSources RecordInputJoystick RecordGameInfo AudioWAVRecorder RecordPositionAttach MainRecorder GameObject The first step is to create the MainRecorder GameObject. This can be conveniently done by dragging the MainRecorder prefab (blue box icon) from the Recorder assets folder into the Hierarchy. The MainRecorder comes with several script components attached. 4 Chapter 1. Another Simple Header

9 MainRecorder The MainRecorder script handles the save directory. Its parameters are: Save Directory: The path of the directory where the data will be output. This can either be absolute or relative to the project root folder. Use.. to navigate up in the directory tree. Local Path: Specifies whether the path in Save Directory is local (relative to the project root folder) or absolute. Additional Name: An additional string that will be added to the data output folders. For example, this can be set to the subject s name or ID. In the most typical implementation, however, this value will be set and overwritten by MATLAB when the game is started Table of contents: 5

10 RecordAudioSources The RecordAudioSources script creates two separate files: one file saving every time an audio clip starts playing, and one file saving every time the volume of an audio clip perceived by the player changes (for example if the distance between the player and the source changes). The variable LogNumberPlaying, if set to true (checked), will write to console the number of audio sources playing every frame. This should only be used for debugging purposes, as it might slow down the simulation. RecordInputJoystick The RecordInputJoystick script records input from the controller. It creates one file for the recorded axes and one for the recorded buttons. The Buttons To Record menu contains the list of the buttons that should be recorded. By default, all buttons are recorded, but it is possible to remove any of them. The Axes To Record menu contains the list of the axes that should be recorded. By default, all axes are recorded, but it is possible to remove any of them. The Save Axis Change Only checkbox specifies whether axes values should be recorded only when they change or every frame. RecordGameInfo The RecordInputJoystick script records general game information. Please refer to DOCUMENTATION! to learn how to implement recording of features. AudioWAVRecorder The AudioWAVRecorder script records the audio perceived by the player (AudioListener). If the Record WAV checkbox is enabled, the audio will be saved to a.wav file. If the Record TXT checkbox is enabled, the audio data will be saved to a.txt file. DOCUMENTATION!!! for more information. Please refer to NOTE: This script has known issues and does not record all sounds being played. Please test the script on your project and see if it works for you. Record Object Position: RecordPositionAttach To record the position and rotation of a GameObject, add the RecordPositionAttach script (found in the Recorder assets folder) to its components. The Save Object Scale checkbox specifies whether the object s scale should be recorded. The Only Save Change checkbox specifies whether the position and rotation data should only be saved when any one of the (position or rotation) coordinates changes. The Record Local checkbox specified whether the position and rotation of the object should also be recorded respective to its parent GameObject. For example, this might be required for CenterEyeAnchor, as local position and rotation give the player s head position and rotation in the real world. Send data over network 6 Chapter 1. Another Simple Header

11 VIESYS GameObject The first step is to create the VIESYS GameObject. This can be conveniently done by dragging the VIESYS prefab (blue box icon) from the CommunicationWIF assets folder into the Hierarchy. The VIESYS GameObject comes with various script components attached. The WorldInterfaceWrapper script is the backbone of the communication with MATLAB. The Start 1.1. Table of contents: 7

12 Paused checkbox specifies whether the game should start paused. The MATLABListenerComponent script exposes some functionalities primarily intended for debugging. Refer to DOCUMENTATION for more information. The RemoveDuplicateSubscribedObjects button removes duplicate subscribed objects attached to the same GameObject (as they are not required). This is only intended for debugging purposes. Track object position: WorldObject To track the position and rotation of a GameObject and send it to MATLAB, add the WorldObject script (found in the CommunicationWIF assets folder) to its components. If the object has been SUBSCRIBED!!! (link) from MATLAB, a message will be sent every time the position or rotation of the object has changed. Use the Id field to specify the name by which MATLAB will refer and subscribe to the GameObject that has this script attached. Use the Property1 field to send a generic message to MATLAB. The field will be cleared once the message has been sent. You will need to implement your own protocol to send messages through the Property1 field and decode them in MATLAB. Getting Started: MATLAB The following pages will explain how to use MATLAB to control your Unity project and collect data. Importing the MATLAB classes Sample script This sample script creates an instance of the BaseGame class and then runs the Unity game.: g = BaseGame() g.run() When the BaseGame class is instantiated without arguments, it assumes Unity is running on the same machine (localhost). Alternatively, it is possible to specify a different IP address passing it as a string, for example g = BaseGame( );. The command g.run() starts the game, loading a MATLAB real-time plot of the position of the player in the game. How to plot other objects will be explained in the Custom script section. The game can be ended anytime by pressing the Esc key and paused/unpaused by pressing the P key. NOTE: Once the game is started from MATLAB, no further action is required. Unity will automatically become active and the player controller will work in the game. Clicking anything other than the Unity game screen will in fact deactivate the controller. 8 Chapter 1. Another Simple Header

13 Creating your own custom MATLAB game script Downloads Unity Reference The following pages serve as reference for the Unity scripts. Recorder Save data to file saudiowavrecorder.. - AudioWAVRecorderListenerClient.. MainRecorder MainRecorderDefaultSaveDir MainRecorderEditor RecordAudioSources RecordGameInfo RecordInput RecordInputJoystick RecordPositionAttach RecordScreenshots RecordTimeScale SavWav AudioWAVRecorder The AudioWAVRecorder script records the audio perceived by the player (AudioListener). If the Record WAV checkbox is enabled, the audio will be saved to a.wav file. If the Record TXT checkbox is enabled, the audio data will be saved to a.txt file. DOCUMENTATION!!! for more information. Please refer to NOTE: This script has known issues and does not record all sounds being played. Please test the script on your project and see if it works for you. CommunicationWIF Send data to MATLAB Timer - A countdown timer Help Here is some help. Need further help Further help search 1.1. Table of contents: 9

14 10 Chapter 1. Another Simple Header

15 CHAPTER 2 Indices and tables search 11