8iUnityPlugin Documentation

Transcription

1 8iUnityPlugin Documentation Release i Jul 18, 2017

2

3 Contents 1 What is the 8i Plugin? 3 2 Why are we doing it? 5 3 Supported Unity Versions and Platforms 7 4 Supported Unity Versions and Platforms 9 i

4 ii

5 Welcome to the 8i Unity Alpha programme! We re looking forward to finding out how you use our custom plugin and getting your feedback. There are a few things to cover before we let you loose, so have a read. Contents 1

6 2 Contents

7 CHAPTER 1 What is the 8i Plugin? The 8i Unity Plugin allows developers to add 8i s fully volumetric video to any Unity experience. It s like embedding traditional 2D video content, except you get to walk around ours as it plays! 3

8 4 Chapter 1. What is the 8i Plugin?

9 CHAPTER 2 Why are we doing it? To introduce developers to our technology, its possibilities and constraints to gather feedback from you to build an 8i community of early adopters, developers, content creators and VR enthusiasts 5

10 6 Chapter 2. Why are we doing it?

11 CHAPTER 3 Supported Unity Versions and Platforms Unity Versions Unity 5.4.x Unity Editor Platforms Windows x64 Build Targets Windows x64 Android 7

12 8 Chapter 3. Supported Unity Versions and Platforms

13 CHAPTER 4 Supported Unity Versions and Platforms Unity Versions Unity 5.4.x Unity 5.5.x Unity 5.6.x Unity Editor Platforms Windows x64 Linux x64 Build Targets Windows x64 Linux x64 Android Contents: Installation Download All the files you need to access, including the plugin, example content and documentation can be found in the dropbox folder that was shared with you. Adding the 8i Unity Plugin 1. Ensure you have Unity or greater installed 9

14 2. Create a new Unity Project 3. Extract the 8i folder from the plugin zip file into your project Updating the 8i Unity Plugin 1. Close the Unity Editor. 2. Completely remove the previous version by deleting the 8i folder from your project. 3. Extract the 8i folder from the plugin zip file into your project It is very important to close the Unity Editor before updating the plugin. This is required as the Unity Editor will lock the native dlls from being overwritten or deleted while they are in use. This can cause issues where the plugin can get into a state where the native dll does not get updated and the plugin will be in an unstable state. Quick Start Create If you are brand new to Unity then there are some excellent tutorials for learning the interface and it s features here 1. Create a new Unity Project. 2. Add the 8i Unity Plugin into your Unity Project. 3. Create a new folder within your Unity Project. 4. Copy the hvr frames you wish to use into the folder. 5. Create a HvrActor by right clicking in the Hierachy and select 8i/Create HvrActor 6. Drag the folder that was created earlier into the data slot on the HvrActor. 7. Select the Main Camera object from the Hierachy 8. At the bottom of the inspector click Add Component and search for HvrRender. Click it to add it to the Main Camera. 9. Position the Main Camera in front of the HvrActor. The HvrActor should now be visible in the Unity Editor Game View. If the HvrActor or HvrRender components do not appear in the Unity Menus, double check the installation instructions or the Troubleshooting section of this documentation. Build 1. Follow the Creating a scene steps above. 2. Save the scene. 3. Open Build Settings window under File/Build Settings Click the Add Open Scenes button on the right of the window, or drag the scene from the Project window of the Unity Editor. 10 Chapter 4. Supported Unity Versions and Platforms

15 5. Click the Build and Run and follow the onscreen prompts and select your build location. Unity will now compile the project and create an exe. If it does not, check the Troubleshooting section of this documentation. VR The 8i Unity Plugin fully supports VR rendering. To enable VR for your Unity Project 1. Open the Player Settings window from Edit/Project Settings/Player. 2. Tick the Virtual Reality Supported checkbox. 3. From the new list, make sure the target headset you wish to use is listed. If it isn t, add it by clicking the + icon. If you are new to VR development in Unity, there are some great tutorials here: tutorials/topics/virtual-reality If you encounter any issues getting VR working in your Unity Project, please consult the Unity Manual Objects and Components Contents: HvrActor This component acts most like Unity s built in Mesh Filter component. Where instead of a mesh, it takes as input a folder from the Unity Project which contains.hvr frames. How to Create 1. Right click in Unity Scene Hierarchy 2. Select 8i/Create HVR Actor Parameters Data Slot which a folder containing.hvr frames can be dragged to from the Project. Asset Render Method The rendering method the HvrActor s HvrAsset will render with. Options Point Blend Renders the actor with smooth points which soften the look of the actor Direct Renders the actor with hard edges. Renders faster than PointBlend 4.3. Objects and Components 11

16 Renderer Render Method The rendering method the actor will use to draw into the scene. Options Standard Supports per-actor colour grading Supports custom shaders Direct Faster than Standard Does not support colour grading Does not support custom shaders Note: This setting will be overridden if the rendering camera s HvrRender component is not set to Standard. Material The material the HvrActor will render with if using the Standard RenderMethod. Receive Shadows Whether the HvrActor will receive shadows when used alongside the HvrShadowRender component. This feature is still in preview and is not recommended for production use. Enabled by default. Cast Shadows Whether the HvrActor will cast shadows when used alongside the HvrShadowRender component. This feature is still in preview and is not recommended for production use. Enabled by default. Occlusion Culling Whether when rendering the HvrActor the HvrRender should use Unity s Occlusion Culling system to check whether the object is visible. A wireframe sphere will be drawn in the Editor SceneView to show the size and location of the bounding sphere which is used to check the visibility of the HvrActor. Options Occlusion Radius Offset Allows you to offset the size of the bounding sphere radius. HvrActorClone This component allows you to clone a specified HvrActor and render copies of it with a reduced performance cost. The performance savings are on the CPU and come from letting the clone share the source s HvrAsset and reducing the total disk read and file decompression. Three clones are still just as expensive to render as three regular HvrActors. How to Create 1. Right click in Unity Scene Hierarchy 2. Go to 8i/Create HVR Actor Clone 3. Assign the Source Actor by dragging an HvrActor into the slot, or searching the scene by clicking on the icon next to the slot. 12 Chapter 4. Supported Unity Versions and Platforms

17 Parameters Source Actor Slot for assigning the HvrActor to clone. Actor Render Method Functionality matches HvrActor. Asset Render Method This functionality is disabled, as the source HvrActor defines this setting. Occlusion Culling Functionality matches HvrActor. HvrAsset The purpose of a HvrAsset object is to store the data for an HvrActor to play. A HvrAsset acts most like a MovieTexture, AudioClip or an Animation in Unity. Unless manually creating HvrActors using code, this object will not be encountered when creating scenes as they are automatically created behind the scenes for HvrActors when actors are assigned data. HvrRender In order to render HvrActors a HvrRender component must be attached to a Camera. There are three different render modes that the HvrRender component can be set to. While Standard is the default and recommended mode, there are two others which each have different properties and performance costs. Performance information can be viewed on the component and will show the cost of rendering the current view in milliseconds. Lower is better. Keep in mind that render performance is closely tied to the number of actors in the scene, the level of detail in the actors and the rendering resolution. How to Create 1. Select the main rendering camera in the scene 2. In the Inspector click Add Component 3. Type in HvrRender into the search box 4. Click HvrRender in the results to add it to the camera Rendering Modes Standard Performance Bronze How it renders Renders the scene in multiple passes, where it renders each actor that is visible one by one. This allows for a lot of customization as each HvrActor can have unique shaders and be color-graded individually. HvrColorGrading support Allows individual color grading of HvrActors if the HvrActor Unity Render Method is set to Standard Objects and Components 13

18 Composite Performance Silver How it renders Renders the scene in a single pass by compositing into the scene by using a fullscreen shader. HvrColorGrading support Supports color grading if the HvrColorGrading component is attached to same camera as the HvrRender component. Invididual HvrColorGrading components on HvrActors are ignored. Direct Performance Gold How it renders Rendering the scene in a single pass by rendering directly into the framebuffer. HvrColorGrading support Does not support color grading. HvrColorGrading This component allows the color grading to be applied to the rendering of HvrActors. How to Create 1. Select your HVRActor from the Hierachy panel 2. Below the HVRActor component click the Add Component and search for HVRColorGrading 3. Add it by highlighting it in the Add Component window and pressing Enter, or clicking it. Usage Adjusting the values and sliders of the this component will affect the look of the HvrActor it is attached to. Note: The Direct render mode available on HvrRender and HvrActor are not compatible with this component. Note: When using HvrRender s Composite mode. The HvrColorGrading component can be attached to the Camera GameObject and used to grade the look of the HvrActors rendered by that camera. 14 Chapter 4. Supported Unity Versions and Platforms

19 HvrActor3DMask This components allows you to mask sections of an HvrActor in 3D space. Instructions Limitations 1. Attach a HvrActor3DMask component to the HvrActor you wish to mask. 2. Create a new GameObject. 3. Attach a HvrActor3DMaskObject to the new GameObject. 4. Position and scale the HvrActor3DMaskObject as desired. 5. The HvrActor will now no longer be visible within the volume of the HrvActor3DMaskObject. This effect is essentially done in 2D. This means masking a hand in front of a face will remove the hand, but the face will. See the example scene for a demonstration of this. Useful Components PlayHvrActor Assigning an HvrActor to this component will trigger it to begin playing as soon as the scene loads. TriggerHvrActor By attaching this component to a GameObject that has a Collider and assigning a HvrActor, the HvrActor will begin playing when the Collider is hit by any collidable object. HvrActorProjectorShadow Can be used in conjunction with a Projector component to project a texture onto a surface based on the bounds of an HvrActor. This is useful for creating blob shadows. HvrActorAudioSourceSync This component has two slots, one for an Animation and one for an HvrActor. Filling both these slots will cause the AudioSource s playback to match the HvrActor. Create by 1. Select (or create) a Unity GameObject 2. Click on Add Component in the inspector 3. Go to 8i/HvrActor Audiosource sync 4. Slot the HvrActor and Audiosource into the component. From now on the audio source will match the playback time of the HvrActor Useful Components 15

20 Note: The HvrActorAudioSourceSync does not take an audio clip itself as a value. It expects the audiosource to have an audioclip assigned. Once you have assigned a Audiosource and HvrActor, you will not need to manage this component - If the actor is playing, the audio source will be playing. If the AudioClip assigned to the AudioSource is shorter than the HvrAsset s duration it will play as long as it can, if it is longer it will stop once the HvrActor stops playing. HvrActorAnimationSync This component has two slots, one for an Animation and one for an HvrActor. Filling both these slots will cause the Animation s playback to match the HvrActor. Create by 1. Select (or create) a Unity GameObject 2. Click on Add Component in the inspector 3. Go to 8i/HvrActor Animation sync 4. Slot the HvrActor and Animation into the component. From now on the animation will match the playback time of the HvrActor. Platform Support Contents: Android Building for Android When building for Android, a custom build step must be used. This can be found under the 8i/Android drop down menu at the top of the Unity Editor window. Android is similar to desktop systems in that.hvr files are read from disk when being used to play a HvrActor. The Unity Plugin provides support to export.hvr data using an OBB file, and then unpack this data into the correct location within the app s internal storage. Hvr data cannot be read from disk while packed in the OBB. They need to be unpacked and written to the app s internal storage. The Unity Plugin provides a helper scene which does this. The scene is called HvrUnpackingScene, and can be found under 8i/core/platform support/android/scenes/. By making this scene the first scene in the editor build settings (i.e. it will be launched when the app is launched), it will automatically unpack the assets, and then load the next scene. HvrUnpackingScene uses the LoadingSceneManager class to unpack the data from the built OBB file. This class can be easily modified if any project specific changes are required. Custom Build Menu Build This step will copy any hvr data that is required by the project into the StreamingAssets directory and then build the project to a specified location. Once the build step is complete, the files that were copied to the StreamingAssets directory will be cleared in order to reduce project bloat. 16 Chapter 4. Supported Unity Versions and Platforms

21 Build and run Same as the Build step, but will attempt to run the build on any attached devices once complete. Prepare Assets for Build This is a helper utility which will cpoy any required hvr data to the project s StreamingAssets folder, but will not build the project. Once this step is complete, the project can be built again via the standard Unity build process. There is no need to run the Prepare Assets for Build option again unless any new hvr data is required or has been removed from the build. Note: The Google Play Store imposes a size limit of 100mb for APK files uploaded for distribution. To allow larger data to be shipped with an APK, Android supports OBB files, also known as Application Expansion Files (Read More.) Unity also supports these files, via a player setting called Split Application Binary. This option will export many of the project resources to an OBB, rather than packaging them with the APK. In order to build succesfully with.hvr files in the project, this option must be turned on. Note: Some devices do not seem to correctly allow the OBB file to be copied to the device when using the Build and Run option in Unity, and in some cases will silently fail to update the OBB when the project is built. If this occurs, the OBB file will need to be manually copied to the development device. So far only the Samsung Galaxy Note 5 has been observed with this issue. Performance Performance Tips Point Count As Android performs much slower than desktop systems it is recommended that hvr frames with point counts of 600k or less are used, with the recommended point count being around 300k. Rendering Settings It is recommended to use the Direct HvrRender render method on Android as it is the best performing renderer. Render Method It is recommended to use the Point Sprite render method in all cases. It is the best performing render method provided. The current alternative, Point Blend does not work on some older devices, and is around twice as expensive to render. Third Party Support Contents: Cinema Director Cinema Director is a third party plugin for creating cutscenes in Unity, it is available on the Unity Asset Store here. Within the 8i Unity Plugin download folder there is a zip folder under 8i/Integrations which includes our integration for the Cinema Director plugin. The script is fully functional and can be used with the current version of Cinema Director (as of writing, version: , Jan 25, 2016). For an introduction to Cinema Director, tutorial videos can be found here. Installation 4.6. Third Party Support 17

22 Extract the CinemaDirector folder from the zip into your project located under 8i/Integrations Usage 1. Open the Creator window for Cinema Director by going to Window/Cinema Suite/Cinema Director/Create Cutscene 2. Set the Actor Track Groups to 1 3. Drag the GameObject that has a HvrActor component attached into the Actor 1 slot 4. Press Create Cutscene 5. If it is not visible already, open the Director window by going to Window/Cinema Suite/Cinema Director/Director 6. In the Director window click the + icon next to Actor 1 Group and add a Actor Track 7. Click the + button next to the Actor Track and select 8i/Play Hvr 8. Click the play symbol in the Director window to preview the cutscene. How to use HVR Actor Fade 1. Open the Creator window for Cinema Director by going to Window/Cinema Suite/Cinema Director/Create Cutscene 2. Set the Actor Track Groups to 1 3. Drag the GameObject that has a HvrActor component attached into the Actor 1 slot 4. Press Create Cutscene 5. If it is not visible already, open the Director window by going to Window/Cinema Suite/Cinema Director/Director 6. In the Director window click the + icon next to Actor 1 Group and add a Actor Track 7. Click the + button next to the Actor Track and select 8i/HVR Actor Fade 8. In the hierarchy click the Cutscene/HvrActorGroup/ActorTack/HVRActorFade and you could moldify the start color/end color/firsttime/duration. 9. If you wish to do the alpha fade, please make sure Rendering Mode of your HVR Actor shader is set to AlphaBlend Node Canvas Node Canvas is a third party plugin for creating a complete visual behaviour authoring framework for Unity, it is available on the Unity Asset Store here. Within the 8i Unity Plugin download folder there is a zip folder under 8i/Integrations which includes our integration for the Node Canvas plugin. The script is fully functional and can be used with the current version of Node Canvas (as of writing, version: 2.6.0b Sep 20, 2016). For an introduction to NodeCanvas, example scenes can be found here. downloads/ Installation Extract the NodeCanvas Integrations folder from the zip into your project located under 8i/Integrations 18 Chapter 4. Supported Unity Versions and Platforms

23 Troubleshooting HVR Actors are not rendering Windows Android The Graphics API may not be supported. Go to Edit/Project Settings/Player and make sure that the Player- Settings Graphics API for Windows is set to either DirectX11 or OpenGL2. These are the only supported graphics APIs at this time. The Graphics API may not be supported. Make sure that under the PlayerSettings that the targeting GraphicsAPI is GLES3 and that your device supports GLES3. HvrRender fails to load Standard shaders There is a known issue that on some devices, that when the Split Application Binary option is enabled, the HvrRender component may not be able to load the Standard shaders. Go to Edit/Project Settings/Player and make sure that the option Split Application Binary is not checked. Android - DayDream The current release of the Unity Editor for DayDream ( 5.4.2f2-GVR13 ), will not use the AndroidManifest.xml file that we provide within the plugin. This means the hvr frames will not be extracted or read from the devices storage upon installing the build. It appears as though this build of the Unity Editor has a bug where it will not use and AndroidManifest.xml file that is not located at this specific location project_name/assets/plugins/android/androidmanifest.xml Until this is fixed within Unity, we recommend copying the AndroidManifest file from this location 8i/core/platform support/android/plugins/androidmanifest.xml to project_name/assets/plugins/android/androidmanifest.xml Writing Custom HvrActor shaders When using the Standard rendering mode for HrvActors, there is a slot where a material may be assigned. When writing custom shaders, you should make sure your Cull shader parameter is set to Front. Culling Settings Off It will look correct, but any transparency would have alpha applied twice. Back Since the mesh that the HvrActor uses is a cube based on the bounds of the HvrAsset, if the camera entered the bounds of the actor, the actor will dissapear as the backfaces of the mesh will not be rendered. Front Effectively draw onto the backfaces of the mesh and looks as expected. Rendering Issues when using VR If when using a Vive, if the actors do not render or there is a upside down and white copy of the world visible in the headset the issue may be VSync being enabled. Fix: Go to build QualitySettings Edit/Project Settings/Quality and make sure that all Quality Settings have the V Sync count option set to Don t Sync Troubleshooting 19

24 Cannot create HvrActor Possible Fixes: Check that the plugin was fully extracted from the 8i Unity Plugin. Make sure the Unity version is compatible with this version of the plugin. Check the console to see whether there are any errors blocking Unity from compiling. Support Support Private Community Chat 8itoolbelt.slack.com (contact us for an invite, if you haven t received one) - we ll be monitoring this from our LA and NZ offices over the alpha testing period 20 Chapter 4. Supported Unity Versions and Platforms