Intelligent Robotic System with Fuzzy Learning Controller and 3D Stereo Vision

Transcription

1 Recent Researches in Syste Science Intelligent Robotic Syste with Fuzzy Learning Controller and D Stereo Vision SHIUH-JER HUANG Departent of echanical Engineering National aiwan University of Science and echnology No. 4, Keelung Road, Section 4, aipei, 6, AIWAN shuang@ail.ntust.edu.tw Abstract: - It is well known that robotic anipulators are highly nonlinear coupling dynaic systes. It is difficult to establish an appropriate atheatical odel for designing a odel-based controller. he odel-free feature of fuzzy logic control strategy was eployed to design robotic otion controller. However, there is no guide rule for designing the fuzzy rule bank and paraeters, it still needs tie consuing trial-and-error work for rules bank and fuzzy paraeters adustent. We had developed self-organizing fuzzy control and adaptive fuzzy sliding ode control two intelligent learning echanis for solving this ipleentation proble. In addition, a low cost stereo vision syste is developed on chip processor. It can be integrated into robotic syste for executing visual servo robotic otion control purpose. Both systes can be constructed on Nios II SOPC developing board with ALERA FPGA chip to anipulate a retrofitted itsubishi robotic syste. he -D position inforation between the target and stereo vision syste was extracted by stereo vision algorith first. hen, the relative otion between the robotic end-effector and the target can be planned to guide robot ar to catch the obect. he self-organizing fuzzy control and fuzzy sliding ode control algoriths are eployed to onitor the traectory otion of each oint. he experiental results show that this visual servo robotic syste can track and catch a oving target in D space and execute soe interaction functions with player. Key-Words: - visual servo, robotic syste, self-organizing fuzzy control, fuzzy sliding ode control, FPGA chip Introduction Since caeras are useful robotic sensors for the robotic autoation operation in non-autonoous environent, it was first integrated with robotic syste as the visual feedback control in 979. How to integrate the achine vision into the robotic syste for iproving anipulator working ability is still a hot research topic in recent years [,]. For the obile robot or huan interaction robot, they need stereo vision to detect the environent obstacle and achieve iage navigation or oving obect following purposes []. Due to the developent of sei-conductor and digital circuit design technology, the new syste- onprograable-chip (SOPC) provides fertilize functions for the servo control, iage processing and network counication purposes. FPGA chips are firstly used in counication and signal processing. Recently, it has been eployed in otor control [4], and the PID control of robotic ar [5]. Here, a SOPC syste is eployed to ipleent a vision servo robotic control syste for huan interaction function. In addition, the accurate dynaic odel for a ultiaxis anipulator is difficult to establish for odel based controller design and real tie coputation, the odel-free intelligent fuzzy control schees are adopted in the robotic otion control field [6]. However, the design of a traditional fuzzy controller depends fully on an expert or the experience of an operator to establish the fuzzy rule bank. Hence, a self-organizing fuzzy controller with learning ability was proposed in [7], new establishing processes of the fuzzy rules bank had been found for reducing the trial-and-error effort. Later, the odified learning ethods for self-organizing fuzzy controller (SOFC) were proposed in [8,9] for further siplification. For each sapling instant, the output response error and error change stiulate two fuzzy subsets of their corresponding universe of discourse E and CE. Hence only four rules are odified in each sapling step. Since, this approach has learning ability to establish and regulate the fuzzy rules bank continuously, its control ipleentation can be started with zero initial fuzzy rules. he fuzzy rules were adusted on-line based on a siple equation instead of a perforance decision table. ISBN:

2 Recent Researches in Syste Science However, it is still a D fuzzy rules. Here, ID adaptive fuzzy sliding ode control strategy [] is odified and eployed to design the individual controller for each oint. A novel gain scheduling algorith is introduced to odify this control algorith for iproving the overall control perforance. he real-tie huan-robot interaction control strategies are developed for the robotic syste to achieve the functions of tracking oving obects and catching the or deonstrate the real tie interaction gaes with player. Robotic Syste Structure he visual servo robotic control structure with Atera Nios II ebedded developent kit is shown in Fig.. Verilog HDL (Hardware Description Language) is selected to code the hardware circuits of this visual servo robotic control syste on FPGA chip. he ain servo control syste can be divided into FPGA internal hardware ipleentation and Nios II icro-processor software progras ipleentation two parts. he ain functions of FPGA hardware circuits are otor optical encoder decoding, liit switch detecting, pulse width odulation (PW) generating and COS iage signal capturing. he functions of the Nios II icro-processor software progras are the counication with PC by using UAR, robotic inverse kineatics calculation, robotic otion traectory planning, robotic otion control schees, and digital iage processing algorith. wo COS color iage sensors with 56x9 pixels resolution are used to extract the environent tie-varying change for robotic visual-based otion control. he robotic syste is an old itsubishi oveaster RV- anipulator with a retrofitted FPGA control structure to substitute original coercial controller. raectory planning and Inverse Kineatics Generally, the end-effector working position or otion path in Cartesian space are converted into control variables in oint coordinates for controlling purpose by using the inverse kineatics and Denavit-Hartenberg transforation atrix. Although soe efficient analysis ethods had been proposed [], they are tie consuing and coplicated atheatical operations. Since the chessboard and the operation range of this vision based robotic syste are liited on horizontal plane, the end-effector orientation is specified as orthogonal and point down to the X-Y horizontal plane. hen the Denavit-Hartenberg transforation atrix of end-effector with respect to the reference inertia coordinate is ref tool A A A A 4 4 A 5 x y z ( ) Based on the itsubishi RV- robot link paraeters, and forward kineatics calculation, the Denavit-Hartenberg transforation atrix can be derived and described by using the robotic D-H paraeters ai and θ i. he oint angle θ i can be solved by coparing the D-H atrix coponents and soe trigonoetric functions operations based on following steps: Step : θ θ 5 A tan ( p y, p x ) Step : b ± ( x + y ) Step : b + ( d d5 z) a a( b a ) a a θ cos () Step 4: ( a θ A tan ( a a a + a C )( d d 5 z ) ( a S ) b + a C ) b + a S ( d d z ) Step 5: θ 4 θ θ his approach can reduce the trigonoetric functions calculation fro 7 ties to 7 coparing with that of traditional inverse kineatics. he coputer tie on the Nios II SOPC can be reduced fro 4.5 s to.5 s for increasing the syste closed loop frequency. For the ulti-input and ulti-output syste, the otion traectory of each oint corresponding to the end-effector point to point (PP) otion in Cartesian space need be planned to ove and stop at the sae tie for the siultaneous otion purpose. he trapezoid speed curve with a constant acceleration and deceleration tie a, a constant acceleration value a and the total otion tie, is the ost popular traectory planning for the PP otion. here relationships are a a v and X + ( ) he axis with largest angular oving range is chosen to calculate the total otion tie based on the specified axiu acceleration, a, and axiu velocity, v. hen the acceleration tie and the axiu velocity of the other axes can be derived based on these oving angles, total otion tie and the axiu acceleration inforation. v a 5 ISBN:

3 Recent Researches in Syste Science ai 4 hose traectory planning paraeters of this 5 DOF robot are derived. 4 Controller Design 4. Self-organizing fuzzy learning controller Generally, the syste dynaic response feature can be represented as an auto-regression and oving average (ARA) odel: θ ( n) Aq ( ) θ( n ) + un ( ) + Bq ( ) u( n ) ( r ) A ( q ) a + a q + L + a r q (5) ( s ) B ( q ) b + b q + L + b s q where is the syste tie delay. is the direct syste forward gain for this position control syste. he values of r, s and depend on the dynaic characteristics of the control syste. Due to syste nonlinearity and uncertainty, they are difficult to estiate for this 5 DOF robotic syste. Fortunately, fuzzy control has a odel free feature. It does not require definite atheatical odel and syste paraeters. Hence, they are not eployed in the following controller design. If the syste is excited with a different control input u' ( n ) at tie step n, there will be a new output value θ' ( n ) at tie step n. he input difference, Δu, of the servo otor control voltage will cause a syste oint output deviation Δθ. If the deviations Δθ and Δθ are sall, then the relationships between control input and corresponding output deviations can be described Δθ Δu and Δ θ Δ u (6) If a syste at tie step n has angular output error Δθ and error rate Δθ, the theoretical corresponding control input correction values are Δu e and Δ u ce, respectively. Δ θ Δ u e and Δ θ Δ u ce (7) Since the syste has only one control input u, the control input correction ust be an appropriate cobination of the above two ters. In general, the following for can be chosen: Δu ( ξ) Δu e + ξ Δu ce (8) ξ is a design paraeter representing the weighting distribution between Δu e and Δu ce. If there is a large difference between the syste angular output θ n and desired valueθ d, an appropriate design ( ) ( ) ( n ) X and v ( 4) i a ai choice is to select a valueθ' n between θ a and θ d. hen the syste oint output θ will approach the desired value θ d gradually with a weighting paraeterγ. θ'( n ) ( γ ) θ( n ) + γθd, <γ < (9) hen the syste output and output change deviations becoe Δ θ ( n ) γ[ θ d θ ( n )] γe( n ) () γ Δ θ ( n ) γe& ( n ) ce( n ) () Fro equations (6) and (), the correction value of the control input can be represented as γ Δu [( ξ) en ( ) + ξcen ( )] () he output error, e(n ), and error change, ce(n ), are divided into 5 fuzzy subsets fro - to + with interval.5. For each control step, the fuzzy input variables, i.e. the syste output error and error change will stiulate two fuzzy subsets of the E and CE universe of discourse, respectively. Since the control input u is derived fro the fuzzy rules inference, the rules odification will influence four fuzzy rules for each control step. he correction value of each fuzzy rule is proportional to its exciting strength w. he excitation strength is i designed as a triangular ebership function and calculated with a linear interpolation algorith. hen the new control input of the rule is ( n ) u ( n) ui + i + Δui () γ ui ( n) + weiwcei [( ξ) e( n) + ξce( n)] he ter γ in the above equation can be considered as the correction weighting. Here, is chosen as in order to eliinate the identification procedure and reduce coputing tie during ipleentation. he correction weighting is regulated by the paraeter γ only. A larger value of γ will introduce a large correction of fuzzy rules and syste output oscillation. his paraeter only influences the transient response but not the steady state perforance. According to experiental experience, this paraeter can be selected as a large value (for exaple.9), and it can be adusted to a saller value (for exaple.) after the learning procedure has converged. It is not crucial for this control strategy. Generally, a γ value of between. and.9 can achieve stable convergent SOFC i th ISBN:

4 Recent Researches in Syste Science systes. he SOFC control block diagra is shown in Fig.. 4. Fuzzy Sliding ode Joint Controller Here the sliding ode concept is cobined with fuzzy control strategy to design a odel-free fuzzy sliding ode controller (FSC) robotic otion control. In addition, the fuzzy variables gains scheduling strategy is integrated into the odel-free fuzzy sliding ode control schee for iproving the transient response and steady state error perforance. It is an enhanced and extended developent fro the original FSC approach he control voltage change for each sapling step is derived fro fuzzy inference and defuzzification calculation instead of the equivalent control law derived fro the noinal odel at the sliding surface. It can eliinate the chattering phenoenon of a traditional sliding ode control. he one diensional fuzzy rules, Fig, (b), is designed based on the sliding surface reaching condition, s s& <. he sliding surface variable, s, is eployed as the one diensional fuzzy input variable. Only eleven fuzzy rules are eployed in this control syste to obtain appropriate dynaic proposed by Huang and Lin [] to achieve response and control accuracy. he input excellent perforance. A sliding surface on the phase plane is defined to ebership functions are scaled into the range of - and + with equal span. Hence a scaling factor gs specify the error converging line in pahse plane. is eployed to ap the sliding surface variable, s, d s ( t ) ( + λ ) e into this universe of discourse. A scaling factor e + λ e (4) gu dt is eployed to adust the value of control voltage. Where e are defined as the state i xid xi he fuzzy sliding control block diagra and control errors. his sliding variable, s, will be used as the input signal for establishing a fuzzy logic control syste to approxiate the specified perfect ebership functions of fuzzy input and output variables, and the fuzzy rules of the FSC are shown in Fig. (a) and (b), respectively. control law, u. With this perfect control law, the he ebership function used for the eq fuzzification is of a triangular type. he function closed loop control syste has an asyptotical can be expressed as stability dynaic behavior. s& ( t) + λs( t) (5) μ ( x ) ( x a + w) (7) w Since λ is a positive value, the sliding surface where w is the distribution span of the variable, s, will gradually converge to zero. Based ebership function, x is the fuzzy input variable on the definition of sliding surface variable, s, in and a is the paraeter corresponding to the value eq. (5), the syste output error will converge to of the ebership function. he height ethod is zero, too. In this study, a fuzzy syste is eployed eployed to defuzzify the fuzzy output variable for to approxiate the apping between the sliding obtaining the control voltage of each oint control variable, s, and the control law, u, instead of otor. odel-based calculation. his control law ay have certain difference with the perfect control law μ U μ C (8) l l u φ C u eq, then the following equation can be derived. μ μ l l s& ( t) λs( t) + b( X, t)[ u eq ( t) u( t)] (6) where is the rules nuber and C is the Generally, b( X ) is a positive constant or a positive slow tie-varying function for practical physical systes. Based on the Lyapunov theore, the sliding surface reaching condition is s s& <. If a control input u can be chosen to satisfy this reaching condition, the control syste will converge to origin of the phase plane. It can also be found that s& increases as u decreases and vice versa in (6). If s >, then the increasing of u will result in s& s decreasing. When the condition is s <, s& s will decrease with the decreasing of u. based on this qualitative analysis, the control input u can be designed in an attept to satisfy the inequality s s& <. consequent paraeter. Here, eleven equal-span triangular ebership functions are used for the fuzzy input variable, s, and the fuzzy output variable, u. he divisions of this ebership functions can be expanded or shrunk by changing the scaling paraeter of ebership functions. he gain scheduling paraeter is used to ap the corresponding variables into this noinal range. hese apping paraeters are specified as gs, and gu for the sliding variable and control voltage, respectively, whose values are linear functions of sliding variable with different slopes. his approach is a novel gain scheduling D fuzzy sliding ode control structure. he values of these paraeters are ISBN:

5 Recent Researches in Syste Science not critical they can be roughly deterined by siple experiental tests. 5 Digital Iage Processing and Stereo Vision he iage raw data extracted fro COS sensor ust go through certain processes to obtain useful inforation. he process flow chart is shown in Fig. 4. Huan eye has good acuinous with respect to brightness Y and it is described as iage gray signal. he iage pre-processing has transfored the 8 bits iage raw data of the pixel into 4 bits RGB color space data. In order to reduce the un-necessary calculation of the Nios II icro-processor, the RGB iage data is converted into 8 bits gray signal by the following forula Y R (9) Cr G Cb B Here, edian filter is used to filter out the iage noise by selecting the edian chroa value of the pixels gray scale around a target pixel as the gray scale of this target pixel for soothing operation. hen a histogra equalization can be used to enhance certain subects by selecting an appropriate threshold value for the grey histogras of the iage features to distinguish the obect fro the background. his threshold value is chosen to convert the analog iage brightness signal into binary signal of each pixel. his iage processing schee is eployed to identify the target location in D space by using COS iages. wo COS sensors are installed with L c relative distance in horizontal line. he coordinate origin is located in the center of two COS sensors, as Fig. 5. hen, the D obect coordinates can be derived based on triangular theore. sin( θ ϕ ) L c () X arg et sin( θ + ϕ ) sin θ sin ϕ Z et L () arg c sin( θ + ϕ ) Y tan( θ Z () Where θ y arg et y ) arg et is target viewing angle fro XZ plane. hen the FPGA controller will send this coordinate inforation to the robotic controller for traectory planning and control. he robotic otion controller guides the anipulator to follow and pick up the target or execute interaction behavior with player. 6 Real-ie Huan-robot Interaction Control Strategy For the huan-robot interaction obective, the robotic syste reaction speed in response to the player otion and the player safety should be considered. Here, player s real-tie decision logic and operation processes are developed to udge the robotic reaction choice based on the target relative otion situations calculated fro stereo vision syste. his robotic syste has basic intelligence to self udge the ost appropriate interaction behavior based on the environental change or target otion situation. he flow chart of the overall robotic interaction control strategy is shown in Fig. 6. he dynaic reaction behaviors of real-tie huanrobot interaction are divided into stand-by, approach obect, ove to the up-right of obect and grasp obect four odules. he corresponding action is planned for each step, for exaple, otion path planning and oving tie specification. When the stereo vision syste has not extracted any interaction target or the target is out of the huan-robot interaction area, the anipulator is stayed in the robot origin with 5 Z axis height as the stand-by step. Since the stereo vision and robot otion control use the sae FPGA processor in this hardware control structure, both actions are worked by turn. If the target is too far away fro the robot hand, the anipulator otion control interval is liited to.5 sec in order to avoid the target go out of the stereo vision range in next visual extraction or the player waiting too long lose the interaction feeling during the approaching obect step. For the safety reason during the robot approaching target step, the anipulator should not too close to the player hand. he end-effector is specified to ove to the target Z axis up-right position with 4 height before executing grasp work or oving approach. he grasping action is processed only when the target is closed to the end-effector and their relative distance in Z direction less than 5. 7 Experiental Results In order to avoid the collision in the nonautonoous environent, the traectory planning is required for the robotic otion ISBN:

6 Recent Researches in Syste Science control, and an appropriate controller is designed to onitor the end-effector otion traectory. Here, the fuzzy sliding ode controller was developed to control this retrofitted itsubishi RV- five DOF robot. For investigating the syste control perforance, the following experients were perfored. he sapling frequency in the oint control experients was Hz. he paraeters gs and gu are used to adust the sliding variables and control input. he choice of these paraeters is not sensitive to controller ipleentation. If these paraeters are varied within 5% and % of the original specified values, the control syste perforance is not changed significantly. 7. Point to point (PP) control accuracy and static obect grasping Case (A): Self-organizing fuzzy controller he planning traectory for the robotic endeffector is a trapezoid speed curve with a constant acceleration and deceleration for each oint and it is oving fro (, 5, ) to (4,, ) in Cartesian space with 4 sec total traveling tie. he axiu angular acceleration of each oint is liited to o / sec. he otion traectory in Cartesian space and the position error in each coordinate axis are shown in Fig. 7. he axiu o angular tracking error of each oint is less than.. he overall position traectory tracking error is less than. he destination steady state position error is.97. Case (B) Fuzzy Sliding ode controller In order to evaluate the FSC control perforance of robotic spot welding and assebly ipleentations etc, a PP otion traectory of the end-effector was planned first. he planning traectory for the robotic end-effector is a trapezoid speed curve with a constant acceleration and deceleration for each oint and it is oving fro (, 4, 5) to (5,, ) in Cartesian space with 4 sec total otion tie. he axiu angular acceleration of each oint is liited to 5 o / sec. he traectory tracking error in each coordinate axis of Cartesian space and the oints angular error histories are shown in Fig. 8. he axiu angular tracking errors of each oint is o less than.5. he overall position traectory tracking error is less than. he destination steady state position error is Huan interaction and oving target following and grasping Case ( C) Chess gobang gae In order to evaluate the dynaic response of this vision servo robotic otion control syste, a gobang gae is playing within a 7x9 chessboard specified in X-Y plane. he flow chart of this non-autonoous huan-robot gobang gae includes copetitor playing turn, end-effector oves to the calculate chess piece position, robot put down the piece, end-effector grasp a piece, robot oving back to the origin and the copetitor play again. Six executing pictures are shown in Fig. 9. his vision based robotic control syste can run successfully in this gobang gae. Case (D) oving target following and grasping In order to evaluate the huan-robot interaction dynaic response of this visual servo robotic otion control syste, the robot end-effector is specified to follow the player finger otion and keep a 4 height distance for safety reason. he dynaic pictures are shown in Fig. (a). In addition, the oving obect grasping experient is planned as the anipulator need to grasp a white sall oving ball which sticks at the tip of a hand bar oving by a player. he stereo vision syste is used to extract the oving target in iage space and calculate the D coordinates with respect to the robot end-effetor. hen the robotic fuzzy sliding ode control syste is eployed to onitor the corresponding following and grasping process based on this visual inforation. he dynaic operation pictures are shown in Fig. (b). 5 Conclusion A stereo visual servo control SOPC structure is ipleented on a retrofitted 5 DOF robot for otion control and huan-robot interaction gaes. he self-organizing fuzzy controller and fuzzy sliding ode controller with D fuzzy rule only were coded inside the FPGA chip for each oint otion control of this robot. his control strategy establishes the appropriate fuzzy rules bank by continuous learning instead of by trial-and-error process to siplify the ipleentation difficulty of a fuzzy controller. he fuzzy rules table can be initialized as zero. he feature of the proposed SOFC structure is that four rules are odified only by learning for each sapling interval instead of all rules bank. It can reduce the coputing tie and data base. D FSC is a siple control algorith ISBN:

7 Recent Researches in Syste Science and easy to ipleent on the on-board FPGA control structure. he experiental results show that this visual guided robotic intelligent control syste can effectively and accurately onitor the robotic end-effector to track planning traectories, extract the oving target relative distance in iage space and guide robotic syste to grasp the oving target and execute soe interaction actions with player. his SOPC control structure can be eployed in a flexible non-autonoous environent for executing rando assebly or pick-and-place, collision avoidance and obile robot visual guided operations. [] Shiuh-Jer Huang and Wei-Cheng Lin,, Adaptive Fuzzy Controller with Sliding Surface for Vehicle Suspension Control, IEEE ransactions on Fuzzy Systes, Vol., No. 4, pp [] L.. Wang and C.C. Chen, A Cobined optiization ethod for Solving the Inverse Kineatics Proble of echanical anipulator, IEEE rans. On Robotics and Autoation Vol.7, N.4, 99. Acknowledgeent his research is supported by the National Science Council under the contract NSC 99--E- Y. Fig. SOPC Robotic control syste structure. References: [] J. Blasco; N. Aleixos; J.. Roger; G. Rabatel; E. olto, Autoation and Eerging echnologies: Robotic Weed Control using achine Vision, Biosystes Engineering Vol. 8, Issue:, pp , October,. [] Chern-Sheng Lin; Li-Wen Lue, An iage syste for fast positioning and accuracy inspection of ball grid array boards, icroelectronics Reliability Vol. 4, Issue:, pp. 9-8, January. [] Han, S.-H., Seo, W. H., Yoon, K. S., and Lee,.-H., Real-ie Control of an Industrial Robot Using Iage-Based Visual Servoing, Proceedings of the 999 IEEE/RSJ International Conference on Intelligent Robots and Systes, pp , 999. [4] F. J. Lin, D. H. Wang and P. K. Huang, FPGA-based Fuzzy Sliding-ode Contrp; for a Linear Induction otor Drive, Proceedings of the IEEE Int. Conf. on Electrical Power Application, Vol. 5, No. 5, pp. 7-48, Sept. 5. [5] Y. S. Kung and G. S. Shu, Developent of a FPGAbased otion Control IC for Robot Ar, Proceedings of the IEEE Int. Conf. on Industrial echnology, pp. 97-4, 5. [6] Shiuh-Jer Huang and Ji-Shin Lee, A Stable Selforganizing Fuzzy Controller for Robotic otion Control, IEEE ransaction on Industrial Electronics, Vol. 47, No., pp. 4-48,. [7] Procky. J. and adani E. H., A linguistic selforganizing process controller, Autoatica, Vol. 5, pp. 5-, 979. [8] Zhang B. S. and Edunds J.., Self-organizing fuzzy logic controller, IEE Proceedings-D, Vol. 9, No. 5, pp , 99. [9] Shao S., Fuzzy self-organizing controller and its application for dynaic processes, Fuzzy Sets Syste, Vol. 6, pp. 5-64, 988. Fig.Self-organizing fuzzy controller block diagra. (b) Fig. (a) Fuzzy sliding ode control block diagra and (b) ebership function of sliding variable and D fuzzy rules.. ISBN:

8 Recent Researches in Syste Science Fig. 4 Iage processing flow chart. Fig. 8 he traectory tracking error in each coordinate axis of Cartesian space and the oints angular error histories Fig. 5 Geoetric relation between two COS sensor and target. Fig. 6 he overall huan-robot interaction control strategy and flow chart. Fig. 9 Robotic gobang gae executing pictures. Fig. 7 SOFC Position tracking error in each axis and D path error. Fig. (a) Robot end-effector follows the oving of player finger and (b) grasp a ball. ISBN: