J.L. BUESSLER, J.P. URBAN Neurobiology Suggests the Design of Modular Architectures for Neural Control Advanced Robotics, 16 (3),

Transcription

1 MIPS EA33 - Groupe TROP Contrôle Neuro-mimétique 4 r. des Frères Lumière Mulhouse Cedex J.L. BUESSLER, J.P. URBAN Neurobiology Suggests the Design of Modular Architectures for Neural Control Advanced Robotics, 16 (3), (00) Preprint MIPS-TROP-TR-0-05

2 Neurobiology Suggests the Design of Modular Architectures for Neural Control J.L. Buessler J.P. Urban TROP Research Group at the University of Mulhouse 4 rue des Frères Lumière, Mulhouse, France JP.Urban@univ-mulhouse.fr Abstract The existence of modular structures in the biological world strongly suggests that the training of this kind of structures is actually feasible. It is a key indication for the development of neural networks applications, especially in the field of robotics. Indeed, a single network can only efficiently treat problems with few independent variables; the combining of several networks is necessary to address more complex tasks. We investigate learning techniques and show that using a particular form of architecture can ease the training of a modular structure: a bi-directional structure that allows combining several neural networks. The approach is illustrated with Kohonen s self-organizing maps for a robotic visual servoing task. Our group investigates the possibilities of learning in such modular structures. Adaptation in neural systems appears a priori very difficult. Indeed, the design of architectures combining several ANN stumbles currently on the difficulty of training the various networks. Besides, since the brain presents both a large modularity and an extraordinary learning capability, it is important to search the mechanisms in action. This article shows that a bi-directional architecture can considerably ease learning and could be a key to modular neural structures. The approach is introduced in a very general manner; no specific assumptions are made on the learning mechanism. The second part of the article describes an implementation with Kohonen s Self-Organizing Maps, and a robotic arm-positioning task using data supplied by a stereoscopic vision head. 1. Introduction Neurobiologists collect more and more precise data on the working of the CNS (Central Nervous System). The brain appears as a highly modular structure. Recent reviews on visually guided reaching [1,] reveal a distributed functioning over multiple areas. There is no unique sensorial space perception preceding action, as has been supposed for along time. A multiplicity of areas brings about distinct sensorimotor transformations, each area using specific sensorial data [1]. These transformations are realized progressively by strongly functionally related areas linked by reciprocal connections []. This information is important for the design of artificial systems. For example, Kawato and his group proposed interesting applications of bi-directional architecture where Artificial Neural Networks (ANN) are linked by reciprocal connections [3].. Modular Architecture and Learning.1. Architectural units Considering the brain as an aggregation of specialized modules, many kinds of combinations can be considered. Two basic architectural units are used to illustrate the problems of learning. x Expert 1 Expert Expert 3 Gating Figure 1: An example of parallel organization: mixture-of-experts Σ ŷ IROS 000 Tagamatsu, Japan, Oct Nov. 5,000 organized session Robotics & Neuroscience (A. del Pobil)

3 x 1 x x 3 NN 1 NN NN 3 NN 4 NN 5 NN 6 Figure : An ANN sequential composition example Jacobs and Jordan s mixture-of-experts [4] represents an example of parallel organization (see Figure 1). Each neural network is considered as an expert. It can carry out the task by itself, but is mainly efficient on a fraction of the input space. A gating network weighs out the experts responses to determine the global output response. On the other hand, a modular decomposition can break up the treatment into a sequence (see Figure ). One module carries out part of the treatment. Its response is an intermediate result transmitted to another module. The interest of this architecture becomes considerable when sub-tasks can be defined that will only treat a fraction of the variables. This modularity reduces thus the dimensionality of the modules to be implemented. Figure 3 describes the basic architectural motif. Input vector x is partitioned into two sets x 1 and x (not necessarily disjoint). Multiple variants of units and composition are possible. Each individual module can itself be decomposed... Supervised learning To examine the compatibility of these modular structures with a learning capability, the nature of this learning must be defined. Our approach is developed with a stochastic supervised learning scheme, but the reasoning can be generalized to other training variants. Supervised learning is defined in terms of example-based adaptation of system parameters (i.e. synaptic weight). It is stochastic when the data are used online, without being stored. The weights are iteratively updated with randomly presented examples that depend on the interactions of the system with its environment. To define the notations more formally, a neural network responds with yˆ R x to input vector x. The neural function can also be denoted yˆ k R xk, Wkto show the dependence of the response to parameters W k. Presenting example xk, y k, the weights are modified to reduce error ε y yk Гyˆ k yk ГR xk, W k, or a cost function J ε y. In the parallel organization of Figure 1, each expert can use examples x, y. They compete to learn the k k x x 1 training patterns. The gating network mediates the competition and is itself adapted. Jacobs and Jordan developed these learning algorithms [4]. In a sequential structure, on the other hand, the examples are not sufficient anymore to supervise the adaptation of each of the modules. The intermediate variables zk introduce new degrees of freedom in the system that are not defined in the response model (see figure 3). Today, there is no practical solution available for the learning of such a modular structure. And thus, this type of modular structure cannot be used in ANN applications, whose use is limited to problems of small dimensionality. The composition can only be implemented if each network can be trained independently. Ingenious application examples can be found in robotics (e.g. [5])..3. Internal representations The systematic interconnection reciprocity that is noticed within the brain areas suggests an extension of the neural module concept. Instead of a single ANN, we consider that each module contains a second network R inv associated to the first, that learns the inverse correlation, from the outputs to the inputs. This extension defines a data flow of opposed direction. To simplify the analysis, we assume that function y F x, that is to be learned by the neural structure, can be decomposed in the form: z yf z, x with F 1 x 1 (1) and there exists a function G(.) such as: z G y () ẑ R ŷ R 1 Figure 3: Basic architecture motif of a sequential composition. The learning flows appear in dashed lines.? Functions F 1, F, and G, are smooth. They are not necessarily known. They only establish the possibility to carry out learning. For example, supervised learning of module M 1 has only meaning if z is defined and unique for a given vector x 1. The neural structure thus reproduces a mathematically possible decomposition and can provide a precise response if the training of both modules is carried out suitably. y page /

4 It is easy to see on Figure 4 that module is entirely supervised with data known at instant k: x, k, zˆ k, y k. We can define the adaptation of modules R 1 and R inv by taking each other s response as desired output to minimize the difference between their estimations. Consider the class of solutions of functional equation: R x R y 1 1, k inv k z ^ R x R F, x using equation (1) 1 1, k inv k k Since the relation must stand x1, x, the second term depends only on z and thus represents a function h(z). Function h is in this case a scalar function of z. If h can be inverted, then zˆ hz ( ) can be defined as an internal representation of variable z. The training phase leads the neural networks to represent respectively: R h F x ˆ x R zˆ, x F h z, R x y h G y inv k k In this configuration, the third network serves only to ease learning phase. The composition of the two first realize the desired general function: yˆ R zˆ, x R R x, x F F x, x F( x) Regularization Any invertible function h will fit the relation above. It is therefore not necessary to impose the choice of the internal representation, for example hz ( ) z. However, one must verify that the adaptation does not converge to a trivial solution hz ( ) constant. One can also take advantage of the freedom of representation to favor a smooth function with an amplitude range compatible with the elements of x. The adaptation of the networks will be more efficient. (3) The regularization techniques are difficult to implement in the context of stochastic learning. The most efficient approach seems here to be an algorithm introduced by J.F. Cardoso and B. Laheld [6], to adapt iteratively a whitening matrix. The training examples are adapted for one of the networks, for example R 1, which will no more take x1, k, z ˆ, k as model, but x1, k, k with k k ˆ, k k v z Г m (4) where v k and m k correspond respectively to a whitening factor and a mean estimation of z ˆ, k. These two variables allow to enforce a constraint of zero mean and unit variance for the learning flow of network R 1. They are iteratively estimated with z ˆ, k, using equations: m m 0 zˆ Гm k1 k m, k k v v 0 v 1Г v zˆ k1 k v k k, k where γ m and γ v are learning rates comprised between 0 and 1. They can be time varying..5. Bi-directional architecture This learning approach, based on bi-directional data flows, can be easily generalized. It uses exclusively local data, that belong to the considered neural stage. The adaptation technique is identical when the considered modules belong to a structure that gathers multiples stages. Its principle is simple and allows to design more complex variants and modular architectures. The design of these architectures is based on the possible decompositions of data processing and the necessity to reduce the dimensionality of the subtasks. In many applications, a judicious data organization permits to meet the constraints expressed in equations (1) and (). The example in the following section shows that learning is really feasible and efficient when several bi-directional stages are chained and produce several internal representations. (5) x x 1 R 1 V m ẑ 1 ẑ R R inv Figure 4: Bi-directional extension of the learning diagram depicted in Figure 3. The regularization variables appear symbolically on Module 1. ŷ y 3. An example of composition The application we introduce uses several Self-Organizing Maps (SOM) to represent the motor and sensorial position correlations of a robotic platform. Two active cameras follow the movements of a robot manipulator in 3-D space. The mapping of image positions and camera orientations into arm angular joint page 3/3

5 = Σ * ˆ F A SOM (3-D) ˆ4 v SOM 1 (3-D) m - + ˆ F * B SOM.inv (3-D) 4 Module A Module B Figure 5: Modular decomposition for the learning of visuo-motor correlations : the global transformation F: [V,α] = θ is decomposed into two steps G: [α, V] = α F and H: α F = θ. The bi-directional organization consists of a 3-D SOM map for Module A, two 3-D SOM maps for Module B. positions could be learned by a neural network. However, decomposing the problem and using several neural networks turns out to be a better way Visual and robotic task We consider a robotic arm in movement in its 3-D workspace and study the control with visual information. Active vision techniques are implemented to ensure the vision system's performances. The cameras of a robotic head are continuously oriented in direction of the interest zone. The use of visual data for robotic control turns out to be difficult with classical techniques. Various works showed that ANN could facilitate the learning of such a control scheme. Learning replaces the determination of kinematics models and the tedious system calibration phase Especially Self-Organizing Maps, based on Kohonen's algorithm, result in a relatively rapid learning by organizing and using a topological representation of the input space. The neural algorithm can therefore be used to control the robot-arm or robot-head movements, expressed in terms of positioning tasks, visual tracking, and so on. In this paper the use of the data for controls is not described. The emphasis is placed on the idea that the learning of discrete correlations with SOM algorithms can be easily decomposed in modules. In this experiment, the data have been gathered on a robotic platform. The experimental setup is composed of a robotic head that follows the movements of an arm manipulator fixed upside down to the ceiling for maximum workspace (see Figure 5). The active vision head can be defined using three motor angles: α P, α T, pan and tilt of the whole head, and α V, the vergence variable. In the described experiments, the cameras' vergence angles (referenced to the head basis) are maintained symmetrical and can therefore be represented by a single variable α V. Let α = [α P, α V, α T ] denote the camera configuration. A 3-D target point P is projected on the image plane of the two cameras and forms [x R, y R ] and [x L, y L ], the right and left image coordinates of the projected points. Let V = [x R, y R, x L, y L ] denote these image coordinates. Any 3-D target point P is represented by its image coordinates V for a given camera configuration α. The particular camera configuration that fixates a point P is denoted by α F (P) (P is fixated when its image appears at the center of the image plane for both cameras denoted V F ). The non-redundant robot arm configuration is denoted θ = [θ 1, θ, θ 3 ]. The purpose of the learning scheme is to construct the correlations between the sensorial data and the two motor systems, G: [α, V] = α F and F: [V,α] = θ. 3.. Supervised Self-Organizing Maps Among neural networks algorithms, we choose Kohonen s Self-Organizing Maps, and its extension by Ritter et al. [7]. The algorithm is simple, well-suited for online adaptation, and converges rapidly. The SOM algorithm is based on 3 major elements: a pre-defined grid of fixed dimension where each unit (neuron) r is defined by its position r, the distance between neurons r and s and the neighborhood function h rs = h(r,s), weight vectors, associated to each unit r: w in r in the input space, w out r in the output space. page 4/4

6 At a given time instant, the competition between the units within the layer results in a winner s, the neuron whose weights best fit the input vector x: x - w in s x - w in r, r (6) This neuron activates the associated output vector: ŷ= w out s (7) the learning step for the input and output weight vectors are respectively: w in r = w in r + ε.h rs.(x - w in r) (8) w out r = w out r + ε'.h' rs.(y - w out r) (9) ε and ε' are learning rates and h rs and h' rs are neighborhood functions of the form: h exp r Г s rs Г 6 (10) The values of the parameters ε, ε', σ, and σ' decrease with the number of learning steps Neuro-controller architectures In a previous work [8], we analyzed the decomposition into two modules whose structure is depicted in Figure 5. This architecture is briefly described, and then a complementary decomposition of Module B is proposed. The results presented here confirm the working of a structure that chains 4 bi-directional modules. a) Bi-directional Learning The diagram in Figure 5 depicts the two steps involved in the arm motor command based on active vision data: G: [α, V] = α F and H: α F = θ. This modular decomposition is interesting because it is based on an internal representation which corresponds to α F, the robot head commands to fixate the 3-D target point with the cameras. The elements of vector α F can be expressed as the sum α F = α + α = α + f(v). We consider that the camera orientation variation is only a function of vector V, and not of the current orientation. This is an acceptable approximation that is necessary for the dimensional reduction of the data to be treated. Module A uses a 3-D SOM map. The adaptation of this module is based on the bi-directional learning principle. Module B combines two 3-D maps, one for the direct data flow, the other for the inverse flow. The learning parameters are detailed in [8]. b) Extended modularity (figure 6) Module A use an additive composition of maps, respectively of dimensions 1-D and -D. An analysis of the robot s geometrical invariances leads to the decomposition of Module B into a 3-stage bi-directional structure. The neuro-controller s global architecture is then based on 8 SOM maps, seven -D and one 1-D. With this thorough decomposition, we will verify that the advantage of having dimensionally reduced maps is not annihilated by the induced more complex modular structure. c) Comparison tests Two tests are proposed to give elements of comparison. The independent learning assumes that the data necessary to perform supervised independent adaptation of the two modules are available. This test provides an ideal learning reference. The second reference, the single network test, attempts to learn F: [V,α] = θ with a single SOM map. It shows the failure of this approach when too many independent variables are involved. = X R, X L Y R +Y L SOM (-D) SOM (1-D) Σ - + ˆ F * A F * B * F V * F T * F P Module B1 SOM (-D) Module B SOM (-D) Module B3 SOM (-D) θ 1 θ θ 3 Module A Module B Figure 6: Extended modularity. Module A uses an additive composition of two maps. Module B is broken up into three -D bi-directional modules. Each module Bi reproduces the structure of Module B in Fig. 5. page 5/5

7 3.4. Results and discussion The main purpose of the presented results is to validate the learning technique. The modular architecture is compatible with rapid learning and good convergence. We present the results of experiments conducted on a real robotic platform. Table 1 presents the positioning error for a test set after a learning phase (10,000 presented examples). The mean error value is close to the minimum, for the considered discretization, when the training of the networks can be supervised independently (independent learning test). In the general case, in which the data are not available to supervise each network, the bi-directional learning technique shows its efficiency, even when the decomposition introduces several modular stages. Table 1 : Architecture comparison and performance Evaluation after 10,000 training cycles Single neural network Independent learning Bi-directional structure (fig 5) Extended modularity (fig 6) error (cm) positioning error mean(max) in cm number of SOM networks presented vector # number of SOM neurons response computing time (with learning) ms/sample 10.1 (31) 1 (3-D) 40, (10) (3-D) 84, (11) 3 (3-D) 16, (10) 8 (-D) 15, Figure 7: Convergence of learning for the extended modularity structure. The neural responses positions the end-effector, and the distance to the desired position gives an error measure. The curve represents the mean values for sequences of 100 vectors; the '+' signs the maximal values. Figure 7 presents the error evolution during the training of the extended modular structure. The resulting curve is very similar to the one obtained with the supervised adaptation of each of the SOM maps. The interest of the modular approach appears clearly with the examination of the performances displayed by the single network test. Although a large number of neurons are used, it does not converge on a satisfying representation of the function. Besides, when modularity permits to reduce the dimensionality of the networks, and therefore the number of neurons implied for an equivalent discretization, the computational cost is greatly reduced. 4. Conclusion The bi-directional learning technique enables the creation of an internal representation that does not have to be supervised during the training. It provides the means to design modular architectures and to break up the treatments. The experiments have been presented using supervised SOM networks. They converge rapidly, but result in a relatively coarse space discretization. The presented bi-directional technique is however not dependent on a given learning algorithm. 5. References [1] Rizzolatti, G., Fogassi, L. and Gallese, V. (1997), Parietal cortex: from sight to action, Current Opinion in Neurobiology, Vol. 7, pp [] Caminiti, R., Ferraina, S. and Mayer, A. B. (1998), Visuomotor transformations: early cortical mechanisms of reaching, Current Opinion in Neurobiology, Vol. 8, pp [3] Miyamoto, H., Schaal, S., Gomi, Wada and Kawato (1996), A Kendama Learning Robot Based on Bi-directional theory, Neural Networks, Vol. 9, pp [4] Jacobs, R. A., Jordan, M. I., Nowlan, S. J. and Hinton, G. E. (1991), Adaptive Mixtures of Local Experts, Neural Computation, Vol. 3, pp [5] Kun, A. L. and Miller, W. T. (1999), Control of Variable-Speed Gaits for a Biped Robot, IEEE Robotics & Automation Magazine, Vol. 6, pp [6] Cardoso, J.-F. and Laheld, B. (1996), Equivariant Adaptive Source Separation, IEEE Trans. on Signal Processing, Vol. 44, pp [7] Ritter, H. J., Martinetz, T. M. and Schulten, K. J. (199), Neural Computation and Self-Organizing Maps, Addison-Wesley, Reading, MA. [8] Buessler, J. L., Kara, R., Wira, P., Kihl, H. and Urban, J. P. (1999), IEEE SMC'99, Int. Conf. on Systems, Man and Cybernetics, Tokyo. page 6/6