Using zspace in Linux

Transcription

1 Using zspace in Linux October 2013

2 zspace and Infinite Z are registered trademarks of zspace, Inc. All other trademarks are the property of their respective owners. Copyright 2013 zspace, Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or any means electronic or mechanical, including photocopying and recording for any purpose other than the purchaser s personal use without the written permission of zspace, Inc. zspace, Inc. 490 De Guigne Drive, Suite 200 Sunnyvale, CA (408)

3 Contents Overview 4 Setting Up Stereo 4 Installing the GPU Driver 4 Updating the X Server Settings 5 Getting Tracking Data 9 Two PC Approach 10 Virtual Machine Approach 12 Using Tracking Data 13 zspace System Specifications 13 Tracker Space Coordinate System 14 Tracker Space to Display Space 17 VRPN Limitations 18 Conclusion 18 References 18 3

4 Overview This document provides information to help you develop a zspace application for the Linux platform. Although the zspace Software Development Kit (SDK) does not support Linux, you can still develop zspace applications for the Linux platform. The section Setting Up Stereo discusses the hardware and software required to enable stereoscopic 3D applications on Linux. The section guides you through the basic setup requirements and how to test your setup. The section Getting Tracking Data discusses the hardware and software required to receive head tracking and stylus tracking data from the zspace tracking system. We discuss different methodologies as well as the pros and cons of each method. The section Using Tracking Data discusses how to apply the tracking data to the application. Normally the zspace SDK provides the necessary calculations and state management. However, since the SDK does not support Linux, this section describes what the application must do to integrate the tracking information. Setting Up Stereo After you install Linux, you will need to install the video card driver and enable stereoscopic 3D to work with X Windows. Although fairly straightforward, you should understand the steps necessary so the GPU and zspace work together to provide stereoscopic 3D. Note: This document focuses on our experiences enabling zspace with Ubuntu and an NVIDIA GPU. We expect that you can use other Linux platforms and other GPUs that support stereo. Installing the GPU Driver To get started, you need to install the driver. The examples in this document use the NVIDIA driver for the Quadro 4000 GPU. However, you can use any GPU that supports OpenGL stereo. For information on AMD Linux drivers, refer to Note: Linux developers have created their own versions of various GPU drivers, so it may appear that your GPU driver is pre-installed on your system. For example, Ubuntu comes pre-installed with the Nouveau driver. This is not an NVIDIA driver. We strongly recommend that you install and download the Linux driver from your GPU vendor s website. 4

5 Setting Up Stereo Updating the X Server Settings After you install the GPU driver, you should verify that things are properly working. Follow the steps below: 1. Start the NVIDIA X Server Settings application by typing the following from the terminal: sudo nvidia-settings 2. Verify the NVIDIA driver version is the one that you just installed. 3. Modify the zspace display to a resolution of 1920 x You can do this by opening the Linux System Settings and then Displays. 5

6 Setting Up Stereo If the only resolutions supported are lower resolutions (for example 1024 x 768), the Extended Display Identification Data (EDID) may not be recognized. The EDID contains the information for the display, such as the supported resolutions. The GPU reads this information to provide the supported options. To resolve this: a. Open the NVIDIA X Server settings and click on the display, DFP-0 in this example. If the Acquire EDID... button is grayed out, the driver cannot read the EDID. 6

7 Setting Up Stereo b. Since the EDID for the DisplayPort is different than the one for the DVI port, try switching from DisplayPort to DVI or vice versa to see if the EDID is recognized. 4. If the display resolution is set properly and everything appears to be working, you need to set the Stereo Mode to Onboard DIN Stereo. Enter the following command in the terminal: sudo nvidia-xconfig --stereo=3 For more information, please refer to the documentation for your GPU. 7

8 Setting Up Stereo The following screen shows the new setting for the Stereo Mode: 5. Finally, you may want to check some of the stereo settings under the OpenGL Settings. The Exchange Stereo Eyes option can be particularly useful. Since the L/R detection is not available on Linux, you can use this option to flip the eyes if the application does not already provide a way. This option is applied immediately. 8

9 Getting Tracking Data Getting Tracking Data As already mentioned, if you are developing for the Linux platform, you do not have access to the zspace SDK. However, you can use VRPN (Virtual Reality Peripheral Network) to obtain the tracking data. VRPN is a software library that provides a network translation between the application and supported VR devices, such as head-tracked eyewear, the tracked zspace stylus, and the stylus button states. Another benefit of VRPN is that it serves as an abstraction layer so that different physical devices share the same interface. This means that if your Linux application already supports any VRPN tracking device, then switching to a zspace tracking device is relatively simple. zspace has developed a VRPN server that sends head and stylus tracking data across the network. You must install the VRPN server on the same machine as the zspace System software. At this time, the zspace System can run on a Windows XP PC or Windows 7 PC. With the zspace VRPN server, you can send the tracking data from the Windows PC to an application with a VRPN client on a Linux PC. This section discusses two different hardware methods in order to accomplish this: Two PC Approach Virtual Machine Approach 9

10 Getting Tracking Data Two PC Approach The following diagram illustrates using two PCs. A dedicated PC with a Windows OS runs a VRPN server, which sends tracking data and stylus events to a Linux PC. The Linux PC has a zspace application containing a VRPN client. The zspace firmware sends the tracking data via USB, so you must connect the zspace USB port to the Windows PC. Because the video is rendered on the Linux PC, you must connect the Linux PC to the zspace display using either DisplayPort or DVI. Finally, both the Windows and Linux PC require a network connection in order to send and receive the tracking data. Advantages The Linux PC can dedicate machine resources to the application. The zspace System drivers are fully supported in Windows. Disadvantages You need an additional PC and Microsoft Windows license. The solution is not compact because two PCs are required. Each PC requires two network connections. Network latency can be a problem. 10

11 Getting Tracking Data The zspace System software is only partially supported. The zspace System Check will not work because the zspace video cable is not connected to the Windows PC. Test Setup This approach was tested with the following specifications: Linux PC: Ubuntu LTS 64-bit Intel Core i GHz 8 GB RAM NVIDIA Quadro 4000 (DVI to zspace) 80 GB HDD Windows PC: Microsoft Windows 7 32-bit Intel Atom Processor 2 GB RAM Known Issues There were no hardware issues using this approach. However, you need significant physical space. Since you will have two PCs, you need an additional monitor to help set up the server initially. This also requires an additional keyboard and mouse (or a KVM switch), at least initially. If you only have one network port, you will also need an additional network switch to allow both PCs to communicate with the router. 11

12 Getting Tracking Data Virtual Machine Approach The following diagram shows a Windows virtual machine installed on a Linux PC. The virtual machine runs a Windows OS with the zspace System and a VRPN server. The server sends tracking data and stylus button events to the zspace application on that contains a VRPN client. Unlike the previous approach, you can now connect the USB and either the DVI or DisplayPort interface to the same PC. Furthermore, only one ethernet connection is required as the virtual machine application usually provides a way to forward the traffic to both the Linux OS and virtual machine Windows OS simultaneously. Advantages The Linux PC can dedicate machine resources to the application. The zspace System drivers are supported. Disadvantages You need additional PC resources to run both the application and virtual machine. The zspace System software is partially supported. 12

13 Using Tracking Data The zspace System check will not work due to lack of OpenGL stereo support on the virtual machine. Test Setup The zspace EDID may not be recognized by the zspace Control Panel. This approach was tested with the following specifications: Linux PC: Ubuntu LTS 64-bit Intel Core i GHz 8 GB RAM NVIDIA Quadro 4000 (DVI to zspace) 80 GB HDD VMWare Virtual Machine Known Issues Microsoft Windows 7 32-bit There may be some issues in getting the USB and Network Driver Interface Specification (NDIS) drivers to communicate from the Linux OS to the virtual machine. Although the virtual machine should recognize them, you may need to disconnect the USB and NDIS driver from the Linux host and connect them to the Windows OS using the virtual machine software. There were no known performance issues with this approach. However, depending on the application requirements, running the virtual machine in addition to the application may not work due to lack of PC resources. Using Tracking Data After you set up the hardware to receive the tracking data using VRPN, you will need to apply the data in your Linux application. This requires understanding the specifications of the zspace system, the coordinate systems, and the algorithms used to calculate something the application desires. zspace System Specifications This section provides information you will need in order to use tracking data in your Linux application. For example, knowing the size of the display in meters is important, because the tracking data is also in meters and you will need to map between the tracking data and the application data. Understanding the tracking points and sizes of each trackable device is important because some applications may want to translate 13

14 Using Tracking Data these positions to a different position, for example moving the origin of the stylus pose from the stylus tip to the middle of the stylus. Specification Value Display Angle (degrees) x = 30.0, y = 0.0, z = 0.0 Display Offset (meters) x = 0.0, y = , z = Display Resolution (pixels) width = 1920, height = 1080 Display Size (meters) width = 0.525, height = Eyewear Size (meters) length = 0.148, width = 0.140, height = Stylus Size (meters) length = 0.148, width = 0.015, height = The following graphic illustrates these measurements. Tracker Space Coordinate System This section discusses how to get the tracking information from VRPN. The zspace specifications provide helpful background information for this section, such as which points on the eyewear and stylus are reported. 14

15 Using Tracking Data Specification Display Space Origin Eyewear Tracking Point Stylus Tracking Point Center of Display Center of Glasses Front Tip of Stylus When VRPN reports the tracking data, the position and orientation of the eyewear and stylus are relative to the tracker space origin. The tracker space origin is located at an offset from the center of the display with the Y axis opposite to gravity. As shown in the zspace System Specifications section, the tracking point on the eyewear is in the middle, near the center marker. The tracking point on the stylus is located at the stylus tip. The simple code below shows how to get the tracking data in tracker space using VRPN: #include <stdlib.h> #include <stdio.h> 15

16 Using Tracking Data #include <vrpn_tracker.h> void VRPN_CALLBACK handle_tracker(void *userdata, const vrpn_trackercb t) { // Sensor name if (t.sensor == 0) printf("head\n"); else if (t.sensor == 1) printf("stylus\n"); // Position (x, y, z) printf("\tpos (%f, %f, %f)\n", t.pos[0], t.pos[1], t.pos[2]); // Quaternion (x, y, z, w) printf("\tquat (%f, %f, %f, %f)\n", t.quat[0], t.quat[1], t.quat[2], t.quat[3]); } int main(int argc, char* argv[]) { // Open the tracker vrpn_tracker_remote* tkr = new vrpn_tracker_remote("tracker0@localhost"); // Set up the tracker callback handler tkr->register_change_handler(null, handle_tracker); } // Main interactive loop while (false) tkr->mainloop(); As shown in the code, sensor 0 contains the head data and sensor 1 contains the stylus data. The position vector is a translation in meters from the tracker space origin in x, y, z and the quaternion is the rotation as an x, y, z, w vector relative to the head space origin. Note: In addition to receiving tracking data from the zspace VRPN server, you can also receive notifications when there is a change of state of any of the stylus buttons. We do not cover the button presses in this document, so we recommend that you read more about the vrpn_button_ Remote API. 16

17 Using Tracking Data Tracker Space to Display Space This section describes how to transform the tracking information received via VRPN into data that can be mapped to the display. Understanding the display angle and display offset is important for this calculation. At this point, the application has the eyewear and stylus position and orientation. However, your application needs to convert the position and orientation from tracker space to display space. In order to go from tracker space orientation to display space orientation, you will need to use the display angle. From the picture above, the Y axis of the tracker space is perpendicular to the ground and the Y axis of the display space is perpendicular to the ground with an additional rotation around the X axis (i.e. display angle). For example: DisplayAngle.X = DisplayRotation DisplayAngle.Y = 0.0 DisplayAngle.Z = 0.0 The display plane can be defined as the XY plane: DisplayX = {1.0, 0.0, 0.0} DisplayY = {0.0, 1.0, 0.0} However, using the display angle, your application needs to scale the DisplayY vector as follows: DisplayY = {0.0, cos(displayangle.x), sin(displayangle.x)} 17

18 VRPN Limitations Therefore, assuming a DisplayRotation of 30 degrees, we have the following: DisplayX = {1.0, 0.0, 0.0} DisplayY = {0.0, 0.5, } At this point, we have simply rotated the frame so that it aligns with the display space coordinate system. However, we need to translate by the display offset so that the reference frame is now in display space. VRPN Limitations The VRPN server does not currently have the ability to sync the left and right frames of the video data with the zspace display. Therefore, the Linux application will need to allow the user to swap the eyes or simply swap the buffers that the left and right images are rendered to. Conclusion You can develop zspace applications on the Linux platform. NVIDIA and AMD GPUs and drivers support stereoscopic 3D through OpenGL on the Linux platform. Your application can retrieve head and stylus tracking data and stylus button state using the zspace VRPN server running on a Windows OS. The Windows OS can run on a separate PC or on a virtual machine, depending on the application requirements. Finally, you should have an idea of how to implement the zspace coordinate system calculations given the device specifications. References zspace VRPN Driver How to Set Up zspace VRPN 18