Background: Camera-Mirror System on a Remotely Operated Vehicle or Machine Authors: Harald Staab, Carlos Martinez and Biao Zhang This innovation relates to situational awareness for a remote operator of driverless moving and mobile machinery, such as tele-operated underwater vehicles (ROVs), ground vehicles, aerial vehicles, cranes, construction machines, machines for mining and drilling. It covers situational awareness of both, vehicle motion and operation of equipment on the machine itself such as grippers, robot manipulator arms, and tools. Problem Solved: A good situational awareness is essential for productive and safe machine operation, and it at least includes visual feedback. The three (3) technology components in a visual situational awareness system are a camera, a data transmission system, and a display system at the operator station. There is a huge variety of how to design each of the 3 systems. The main problem or challenge is to find the most cost efficient solution for a good enough situational awareness experience for any given application case. Novel Features: The core idea of this innovation is to use a mirror system along with a camera system which allows to easily elevate the viewpoint on the machine, and to change the field of view to what the operator finds most appropriate for the current operation (adaptive situational awareness). It typically includes one or more cameras with controllable orientation and zoom. Benefits: This innovation proposes a design that is more cost efficient than conventional designs, while providing an equal or better situational awareness. (Commercial panospheric camera systems with camera(s) and curved mirrors are relatively expensive because they only serve niche markets such as military, aerospace, or they are non-commercial and only used for special research purposes (mobile robot navigation, robocup contests. The innovation is also less expensive because it saves cables and connectors since the camera is installed at the main body of the machine or vehicle). Detailed Description This description is with reference to Figures 1, 2a and 2b, and 3a to 3b on pages 4 to 6, respectively, of this Technical Disclosure. 1
Fig. 1 illustrates two (2) general embodiments of the innovation with a ground vehicle. The vehicle has a main body 11 to which camera system 13 is attached. There is also a rigid extension 12 of the main body 11 to which a mirror system 14 is attached. The camera system 13 points to the mirror surface 33. The orientation of the mirror determines the center of view 34 and the curvature of the mirror and the viewing angle of the camera determine the field of view 35. The top left image in Fig. 1 shows an embodiment with a planar mirror 33 in a first orientation with a center of view 34 pointing to the front of the vehicle and a field of view 35 showing the front part of the vehicle 31 as well as the surroundings in front of the vehicle 32. A corresponding exemplary camera view is shown in the bottom image 30 in Fig. 1. The actual viewpoint 39 (see the top left image) is a virtual one and it is located behind the mirror. This makes the viewpoint 39 elevated higher 41 than the actual height of the extension 12 with the mirror 40. In a second orientation 36 of the mirror (dashed in the top left image) the center of view 37 is pointing behind the vehicle 31 and the field of view 38 includes both, rear parts of the vehicle and the surroundings behind the vehicle. In this example, the first orientation may be used to drive the vehicle forward and the second orientation may be used to back up. The illustration in the top right image in Fig. 1 shows an alternative mirror system 14 which has a curvature 42 and a camera system which can be changed in orientation 22 to point at different parts of the mirror surface. The illustration shows a first orientation of the camera (solid lines) where the camera view shows the front of the vehicle and the surroundings in front of the vehicle. It also shows a second orientation (dashed lines) where the camera points to a mirror area with stronger curvature. The camera view shows the rear of the vehicle and the rear surroundings with a larger field of view but also with a warp-effect (fish-eye effect). Figs. 2a and 2b show another example embodiment where the rigid extension is designed to also serve as a transparent enclosure 52 for both the camera system 13 and the mirror system 33. Figure 2a shows a perspective view of a design with a rectangular housing and 4 transparent window panes. Figure 2b shows the same perspective view and a sectional view and an exemplary field of view 55 through the front window. Figs. 3a and 3b show an optional additional feature which improves the situational awareness experience with Fig. 3a showing an exemplary ground vehicle and Fig. 3b showing an exemplary camera view which is displayed to the operator. The additional feature comprises one or more laser pointers 61 mounted in parallel to the camera and pointing into 62 the mirror such that the reflection of the laser on the surrounding environment is visible in the camera view 30 to the operator. The laser reflection may be a point or it may have other geometries such as a line or a cross 63, as shown in the example view 30. As can be appreciated the orientation 22 of the camera system or the orientation 21 of the mirror system can be changed automatically with a control system on the vehicle such that when the vehicle is moving around the camera view 30 to the operator always shows the same object in the environment. 2
As can also be appreciated, the remote operator sees the live camera image, can control the center of view of the camera system either by changing the orientation of the mirror system or by changing the orientation of the camera system and as an option the remote operator can control the field of view either by changing the zoom of the camera or by changing the curvature of the mirror surface or by changing the orientation of the camera system to point at a different part of the mirror with different curvature. As can be further appreciated the purpose of this design with mirrors 14 on extensions 12 is to provide an elevated viewpoint 39 that allows to see parts of the vehicle or the machine as well as the surroundings but without the need to install the camera system at an elevated viewpoint. The viewpoint 39 is actually elevated to a point above the mirror and is therefore higher 41 than the physical height 40 of the machine or vehicle. 3
Fig. 1 - General embodiment of the innovation and how proposed method works 4
Fig. 2a Fig. 2b 5
Fig. 3a Fig. 3b 6