Gaussian Process-based Visual Servoing Framework for an Aerial Parallel Manipulator

Transcription

1 Gaussian Process-based Visual Servoing Framework for an Aerial Parallel Manipulator Sungwook Cho, David Hyunchul Shim and Jinwhan Kim Korea Advanced Institute of Science and echnology (KAIS), Daejeon, South Korea his paper proposes a Gaussian process-based visual servoing work that not only overcome the weakness of the standard image-based visual servoing (IBVS) scheme but also improve the vision-enabled and real-time control performance. In particular, the proposed work provides the Gaussian process-based sampled image path consisting of a set of the references between the initial and desired positions of a stationary or moving target in the image plane with respect to the height under the large translation and rotation error in order to overcome the weakness of the standard IBVS scheme. Furthermore, we applied the proposed work to the aerial parallel manipulator during the picking-n-replacing mission. he developed vehicle has two vision system: one is the gimbal-stabilized pinhole camera on the host vehicle and the other is the fisheye camera with the one-dimensional light detection and ranging (LiDAR) fixed on the end-effector of the parallel manipulator. he proposed work can generate the feasible control input according to each sensor system`s features. In this paper, the preliminary results and analysis are represented. he results of simulations and flight tests will be conducted to verify the performance of the proposed work indicate that it can return the path points for the convergence of the desired position while the target is moving or stationary, even with the large scale difference. I. Introduction nmanned aerial vehicle(uav) have more required the commercial purpose than the traditional military purpose. U here is a core technology, as called as the vision sensor system consisting of the camera, light detection and ranging (LiDAR) and on-board embedded computer, to accelerate those needs. Especially, multirotor-type commercial UAVs, courtesy of its flight abilities such as a vertical takeoff, landing, and hovering, can process the valuable information by using the sensor system and perform a work by using own manipulator, like human. Recently, many researches have been conducted to expand the mission capabilities. he most special research area is the aerial manipulation, in which has extended the work capacity by attaching the sensor-based robot arm for interacting the environment actively. However, as the robot arm system is bulky, heavy and has a tightly-coupled nonlinear dynamics, the precise relative position control is being resolved using an external motion capture system generally, not done by own sensor system. Motivated by such the practical problem, we propose a Gaussian process-based visual servoing work to improve the vision-enabled work capacity for the aerial parallel manipulator in this paper. Gaussian process-based sampling algorithm can generate a set of the references in real-time between the initial and desired positions of a stationary or moving target in the image plane under the large translation and rotation error in order to overcome the weakness of the standard image-based visual servoing (IBVS) scheme. Furthermore, unlike the other aerial manipulator, as a parallel robot has the high-speed dynamics than the host vehicle, it affects the UAV`s maneuvering only slightly. Accordingly, it has an advantage of constructing a decoupled control system between robot arms and UAV because their dynamics connectivity is relatively low and the consideration on joint limit can be made simple. herefore, our proposed algorithm can use easily as the high-speed path planner under the allowed control system bandwidth. Graduate Researcher, Dept. of Aerospace Eng., KAIS, 9 Daehak-ro, Yusunggu, Daejeon, South Korea. AIAA Member Associate Professor, Dept. of Aerospace Eng., KAIS Institute, KAIS, 9 Daehakro, Yusunggu, Daejeon, South Korea, AIAA Senior Member Associate Professor, Graduate School of Ocean System Engineering, KAIS Institute, KAIS, 9 Daehakro, Yusunggu, Daejeon, South Korea, AIAA Senior Member

2 II. Development of the Aerial Parallel Robot here are many type for the aerial manipulator. In this paper, we developed the aerial parallel manipulator by using the Quattro-type mechanism as shown in Fig. because it can generate fast the perfect Cartesian motion. Furthermore, we developed the hardware and software for the onboard sensor-aided indoor navigation. Our developed aerial robot can fly by using own sensor such as the two-dimensional LiDAR, the vision and inertial sensor without the external motion capture in the indoor environment. he hardware and software architecture is shown in Fig. and. All system are developed as the onboard using the embedded computer and ROS environment. We are going to develop into a more complete form after submitting this paper. Fig.. Inverse kinematics for the Quattro parallel robot in the vector mechanics x USB-to-Serial x USB. 6x Delta Robot with Arduino Mega.GHz.GHz Wi-Fi Modem D LiDAR altitude sensor module (stabilized) (on the -axis gimbal) (fixed) (on the gripper) D scanning LiDAR Image H/W setup option Image H/W setup option USB. USB. USB-to- port Robot command and USB-to-Serial Odroid XU (powered by ROS) USB. USB. Point cloud H/W setup option Flight and Mission... ROS information USB-to- port...7 ROS information...76 Switching hub Fig.. Hardware architecture for the proposed aerial manipulator by using the Odroid XU, small embedded computer...86 Sensor measurement...9 Flight and Mission.GHz Flight Control Computer

3 Odroid XU, LiDAR-processing Flight control computer (stabilized) (on the -axis gimbal) Image Image D scanning LiDAR D LiDAR altitude sensor module Relative range Pointclouds (on the target and environment) to the target Correlative scan matching (about [Hz]) Flight Position and heading w.r.t Main platform Sensor-based flight estimation Flight Long-range target detection (Color, IR, Pattern..) Short-range target detection (Color, IR, Pattern..) Image (fixed) (on the gripper) arget candidates arget candidates Gaussian mixture model-based target modeling arget tracking and clustering Fig.. Software architecture by using ROS environment Selected target [pixel/sec] Selected target [pixel/sec] UKF-based target motion estimator Short-range distance Image-based visual servoing Relative range Velocity commands to the target D LiDAR on the gripper Selected target [m], [m/s] Position-based visual servoing Long-range distance For the main delta robot Odroid XU, Image-processing Delta Robot with Arduino Mega Velocity commands For the main platform Velocity commands For the main platform Relative velocity controller (body-axis) III. Gaussian Process-based Feature Path Generation A Gaussian process is a generalization of the Gaussian distribution where observations occur in the interested domain. here are many purposes to solve the practical problem through the Gaussian process. In this paper, we focus on the uses to estimate the model by using input data to make the feasible predictions. o solve the large initial error comes from the large scale difference, we apply to Gaussian process-based path generation in the image plane for making the best inferred data. In our proposed algorithm, the detected target is assumed to a convexhull shaped expressed as the rectangle for the IBVS. However, as the target representation with respect to the scale (i.e. height axis) is linear, the state and the measurement configuration can be defined in Eq. () by using the center point of the target expressed it in the principal axis of the image plane as shown in Fig.. x u v z ref x u v z m p ref c ref ref c z c ref ref u v () where uv, is the two-dimensional pixel position of the vertices of the detected target, uv, is same but represented the center pixel position using the principal axis in the image plane, z and t is the relative height and heading between the detected target and the UAV. Practically, the aspect ratio of the target is fixed as shown in Fig.. herefore, he relationship between feature points and height is shown in Eq. () and a set of the feature reference can be calculated with respect to the relative height by using this relationship. c c zref p p p p () where c is the proportional constant to determine the relative depth information from the image sensor

4 x B P B Body y cb x B z B x cb y B Gimbal Stabilized Camera P cb arget On the image Image p im y im x im y ce v z c y ce P ce z B arget On the image End-effector fixed Camera u Large depth environment Point-like feature arget On the World u Width v x N World z N P N y N Fig.. Geometrical relationships among each s he initial detected result comes in, the prediction range vector can setup by using the desired target position and heading. In detail, we can make the prediction range vector from the initial target to the desired target as linearly spaced vector. hen, the hyper-parameter estimation is performed by using these vectors through the gradient-descent algorithm. In this paper, we adopt the squired-exponential (SE) kernel as Eq. (). f, exp K xc xc f xc xc x I xc xc wi () where is the signal variance, x is the length of the scale and is the noise variance. hese hyper-parameters play a role as not only the mathematical parameters for Gaussian process but also the performance design parameters with respect to the physical side. Following this step, the Gaussian process for regression is performed by using Eq. (). w K Κ K λ K μ K xc xc xc xc xc xc xc xc xx c c xx c c K K I λm c w xc xc xc xc xcxc Σ K K λk () where μ, Σ are the regression results in the form of the Gaussian distribution. herefore, the feature path can be calculated by using the mean value of this result. he feature path generation result is shown in Fig. 5. otal time to generate the feature path is.896[sec] and the frequency is 5.76[Hz] in terms of the twenty of the prediction range vector. It means that this algorithm can be used as the feasible path planner and it has the real-time performance in the embedded system. In this paper, we use the standard pinhole camera geometry. herefore, we will extend the algorithm to the fisheye lens camera geometry for enhancing the performance in the terminal phase.

5 Gaussian Process-based Feature Path Generation: w.r.t Height Height [m] 8 6 normalized x-axis, normalized x-axis, normalized y-axis, (a) (b) Fig. 5. he feature path results; the x- and y-axis are the image plane, whereas the z-axis is the altitude: (a) relative heading angle 6 [deg.] case in the image plane (b) three-dimensional representation (red box: initial position and desired position, blue box with dotted lines: reference position from the proposed algorithm result, blue solid line: Gaussian process results, mean value, green solid line: Gaussian process results, variance region) normalized y-axis, IV. Preliminary Flight est Result A preliminary flight result of the picking-n-replacing mission by using the proposed aerial manipulator is given in Fig. 6. Note that the start position is off the straight path, but that the proposed system aligns autonomously through the sensor-aided air navigation by using the two-dimensional light detection and ranging (LiDAR) sensor mounted above the top of the host UAV without the external motion capture system. Furthermore, the end-effector can track and grip the target autonomously by using our proposed algorithm. V. Future Work In this paper, the proposed feature path generation algorithm does not consider the dynamics of the total system. Our aerial parallel manipulator has two vision system and the sub-optimal control input allocator is necessary to distribute the control input to the manipulator and the host UAV. herefore, we will study this part after submitting this paper. We will also integrate total algorithm into the MALAB/Simulink-based simulation environment and the onboard embedded system for the validation and verification. Furthermore, our proposed work is suitable to the picking-n-replacing mission. We plan to study this aspect also and generate the flight test results and analysis in the indoor and outdoor environment. Acknowledgement his work was supported by a National Research Foundation of Korea (NRF) grant (No. 6) funded by the Korea government (Ministry of Science, IC and Future Planning) References odd W. Danko, Kenneth P. Chaney and Paul Y. Oh, Parallel Manipulator for Mobile Manipulating UAVs, 5 IEEE International Conference on echnologies for Practical Robot Applications, pp. -6, Woburn, MA, USA, - th, May, 5 Sungwook Cho, Dasol Lee, and David Hyunchul Shim. "Image-based Visual Servoing Framework for a Multirotor UAV using Sampling-based Path Planning", AIAA Guidance, Navigation, and Control Conference, AIAA Sciech, Kissimmee, FL, USA, 5-9 th, Jan., 5 Victor Rosenzveig, S ebastien Briot and Philippe Martinet, Minimal Representation for the Control of the Adept Quattro with Rigid Platform via Leg Observation Considering a Hidden Robot Model, IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. -5, okyo, Japan, -7 th, Nov., 5

6 Fig. 6. Preliminary flight test results (picking-n-replacement mission) 6