Mobile Robot Control Interface Using Augmented Free-viewpoint Image Generation

Size: px

Start display at page:

Download "Mobile Robot Control Interface Using Augmented Free-viewpoint Image Generation"

Georgiana Holt
5 years ago
Views:

1 THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS TECHNICAL REPORT OF IEICE Abstract Mobile Robot Control Interface Using Augmented Free-viewpoint Image Generation Yuko UEDA, Fumio OKURA, Tomokazu SATO, and Naokazu YOKOYA Graduate School of Information Science, Nara Institute of Science and Technology (NAIST) Takayama-cho , Ikoma-shi, Nara, Japan In this report, we propose a novel interface for safely controlling a mobile robot from a remote site. Most existing interfaces for controlling a remote robot employ first-person view or third-person view whose viewpoint is fixed behind a robot. The problem of these interfaces is that it is difficult for operators to estimate the distances between the mobile robot and obstacles existing around the robot, because the viewpoint of the observation is fixed relative to the robot. In order to allow operators to easily recognize the 3-D environment around the mobile robot, our novel interface provides images of free viewpoint, whose position and orientation is synchronized with those of operator s head. Key words remote control, mobile robot, interface, free-viewpoint image, omnidirectional image sensor, augmented reality 1. [1] [2] [3] (1) (2) Sugimoto [4] 1

2 CG Nielsen [5] (1), (2) (1) (2) Model-Based Rendering (MBR) [6] Image-Based Rendering (IBR) [7] MBR IBR Hybrid Rendering [8] MBR IBR 4 Hybrid Rendering 3D Augmented Reality: AR) : 3 Point Grey Research Ladybug2 1 Microsoft Kinect 4 Polhemus Fastrak HMD Sony HMZ-T1 Sanwa HDMI B. 全方位カメラ全方位画像 D.PC ロボットの位置姿勢 A. ロボット距離画像ロボット側 C. 距離画像センサ (a) E. 無線送受信機 F.PC 1: 2: G. 磁気センサ操縦者の位置姿勢 J. 無線送受信機 I. コントローラ操作命令操縦者側 (b) H. HMD ( ) (HMD) (A) (B) (C) (B) (C) (B) (C) ( 2 ) PC(D) (E) 3(a) (A) (E) A. 2

3 距離画像群の統合 Hybrid Rendring を用いた自由視点画像生成ロボットの AR 提示 (a) (b) HMD に拡張自由視点画像を提示 3: ( 3(a) ) B. 6 C. Kinect 1 Kinect Kinect 4 D. PC (B) (C) (E) E. PC LAN 2. 2 PC(F) (G) HMD(H) (I) 3(b) (F) (J) F. PC 4: G. HMD H. HMD PC(F) HMD I. J [9] [10] Hybrid Rendering AR

4 (a) (b) 5: 2 L 0 L 0 E(R t) n E(R t) = f(rp i + t {q j }) (1) i=1 { 0 ( j p f(p i {q j }) = i q j < = ϵ) 1 (otherwise) p i q j n p i R 2 2 t 2 ϵ 2 L Hybrid Rendering (2) 観察視点 6: メッシュ視点 1 視点 2 観察視点 7: 1. P i ( 6 ) P i d i 4. P i d i 5. P i d i 7 ( α 1 α 2 ) 3. 3 AR CG

8 3 CG 2 HMD 20 10 2 1 2 2 t ( 5%) 1 2 8(c) 5. 3 4 ( 3(a) ) 6 L 0 9 10 11 12 9 提示手法 2: 実験 1( 移動操縦 ) 実験 2( 物体への接近 ) 壁まで時間の距離 ( 秒 ) (mm) 衝突回数 ( 回 ) 平均標準偏差平均標準偏差平均標準偏差衝突した人数 ( 人 ) 主観視点 8.8 6.4 51.

5 8 3 CG 2 HMD t ( 5%) 1 2 8(c) ( 3(a) ) 6 L 提示手法 2: 実験 1( 移動操縦 ) 実験 2( 物体への接近 ) 壁まで時間の距離 ( 秒 ) (mm) 衝突回数 ( 回 ) 平均標準偏差平均標準偏差平均標準偏差衝突した人数 ( 人 ) 主観視点客観視点自由視点 ( 提案 ) (a) (c) 8: HMD 主観視点客観視点 (ii) 客観視点 (i) (b) ロボットの撮影地点 9: (i) (ii) (a) (b) (c) (b) Kinect 6. 5

(a) (b) (c) 10: (a) (b) (c) 11: (i) (a) (b) (c) 12: (ii) ( (A) No.23240024 (B) No.

Yamauchi: PackBot: A Versatile Platform for Military Robotics, Proc.

Workshop on Robot and Human Interactive Communication (ROMAN2004), pp. 107-112, 2004. [4] M.

Matsuno: Time Follower s Vision: A Tele- Operation Interface with Past Images, IEEE Computer

Ricks: Ecological Interfaces for Improbing Mobile Robot Teleoperation, IEEE Trans.

Mordohai, J-M. Frahm, R. Yang, D. Nister and M.

on Computer Vision, 8 pages 2007. [7] S. M. Seitz and C. R. Dyer: View Morphing, Proc.

Anandan: What Does the Scene Look Like frome a Scene Point?, Proc. European Conf.

6 (a) (b) (c) 10: (a) (b) (c) 11: (i) (a) (b) (c) 12: (ii) ( (A) No (B) No ) [1] :,, Vol. 23, No. 5, pp , [2] B. Yamauchi: PackBot: A Versatile Platform for Military Robotics, Proc. SPIE Unmanned Ground Vehicle Tehnology VI, Vol. 5422, pp , [3] N. Shiroma, N. Sato, Y. Chiu and F. Matsuno: Study on Effective Camera Images for Mobile Robot Teleoperation, Proc. 13th IEEE Int. Workshop on Robot and Human Interactive Communication (ROMAN2004), pp , [4] M. Sugimoto, G. Kagotani, H. Nii, N. Shiroma, M. Inami and F. Matsuno: Time Follower s Vision: A Tele- Operation Interface with Past Images, IEEE Computer Graphics and Applications, Vol. 25, No. 1, pp , [5] C. Nielsen, M. Goodrich and R. Ricks: Ecological Interfaces for Improbing Mobile Robot Teleoperation, IEEE Trans. on Robotics, Vol. 23, No. 5, pp , [6] P. Merrell, A. Akbarzadeh, L. Wang, P. Mordohai, J-M. Frahm, R. Yang, D. Nister and M. Pollefeys: Real-Time Visibility-Based Fusion of Depth Maps, Proc. Int. Conf. on Computer Vision, 8 pages [7] S. M. Seitz and C. R. Dyer: View Morphing, Proc. SIG- GRAPH 96, pp , [8] M. Irani, T. Hassner and P. Anandan: What Does the Scene Look Like frome a Scene Point?, Proc. European Conf. on Computer Vision, vol. 2, pp , [9] T. Sato, H. Koshizawa and N. Yokoya: Omnidirectional Free-Viewpoin Rendering Using a Deformable 3-D Mesh Model, Int. J. of Virtual Reality, Vol. 9, No.1 pp , [10] P. Debevec,Y. Yu and G. Borshukov: Efficient Viewdependent Image-Based Rendering with Texture-Mapping, Proc. 9th Eurographics Workshop on Rendering, 13 pages,

Teleoperation of Mobile Robots by Generating Augmented Free-viewpoint Images

213 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) November 3-7, 213. Tokyo, Japan Teleoperation of Mobile Robots by Generating Augmented Free-viewpoint Images Fumio Okura 1,2,