A Network for Extracting the Locations of Point Clusters Using Selective Attention 1

Transcription

1 INTERNATIONAL COMPUTER SCIENCE INSTITUTE 1947 Center Street Sute 600 Berkeley, Calforna (415) FAX (415) A Network for Extractng the Locatons of Pont Clusters Usng Selectve Attenton 1 Subuta Ahmad Internatonal Computer Scence Insttute and Unversty of Illnos at Urbana-Champagn ahmad@cs.berkeley.edu Stephen Omohundro Internatonal Computer Scence Insttute om@cs.berkeley.edu Techncal Report # July 16, 1990 ABSTRACT Ths report explores the problem of dynamcally computng vsual relatons n connectonst systems. It concentrates on the task of learnng whether three clumps of ponts n a 256x256 mage form an equlateral trangle. We argue that feed-forward networks for solvng ths task would not scale well to mages of ths sze. One reason for ths s that local nformaton does not contrbute to the soluton: t s necessary to compute relatonal nformaton such as the dstances between ponts. Our soluton mplements a mechansm for dynamcally extractng the locatons of the pont clusters. It conssts of an effcent focus of attenton mechansm and a cluster detecton scheme. The focus of attenton mechansm allows the system to select any crcular porton of the mage n constant tme. The cluster detector drects the focus of attenton to clusters n the mage. These two mechansms are used to sequentally extract the relevant coordnates. Wth ths new representaton (locatons of the ponts) very few tranng examples are requred to learn the correct functon. The resultng network s also very compact: the number of requred weghts s proportonal to the number of nput pxels. 1. A verson of ths report s to be presented as a talk at the 12th Annual Conference of the Cogntve Scence Socety at MIT, July 25-28th, 1990.

2 Introducton 1. Introducton Consder the vsual task of determnng whether a set of three pont clusters form an equlateral trangle (Fgure 1). People are very good at solvng ths knd of problem, but a connectonst mplementaton for mages wth reasonable resoluton s not obvous. The dffcultes posed by ths problem are common to a wde varety of vsual tasks and so we have used t as a touchstone aganst whch to test vsual neural archtectures. In ths paper we descrbe some of the fundamental operatons the problem seems to requre, bologcally nspred mplementatons of those operatons, and a computer smulaton whch combnes them nto a complete system. Fgure 1 Left trangle s equlateral.the rght one s not. The most straghtforward vsual neural representatons assgn a dstnct unt to each vsual pattern that must be classfed. Unfortunately, the space of possble trangles s much too large for ths knd of approach to be bologcally possble. The optc nerve conssts of about a mllon fbers from each eye, and so t s reasonable to consder square mages whch are a thousand pxels on a sde. Snce each of the three vertces can occupy any of these pxels, the total number of possble trangles n such an mage s about = A brute force representaton would requre about a mllon tmes as many neurons as we have n our entre bran for just ths one task. Restrctng the unts to represent just the set of equlateral trangles would stll requre about unts. If coarse coded representatons are used, these numbers can be reduced somewhat, but many more examples wll stll be needed to learn to properly classfy equlateral trangles than s bologcally possble. The spatal relatonshps whch defne equlateralness wll have to be dscovered for each poston, scale, and orentaton of the trangle. Technques have been proposed for ntroducng translaton and rotatonal nvarance nto networks (Gles et. al., 1987) whch elmnate the need for ndependent feature detectors at every locaton. Unfortunately these methods requre that every unt have a large (quadratc) number of connectons wth complcated weght lnkages between them. Furthermore, postonal nformaton s lost n these representatons - one cannot retreve the locaton and orentaton of the objects n the mage. These dffcultes would dsappear f we could drectly extract the real valued coordnates of the cluster centers, say n the actvatons of 6 unts. Usng ths representaton t s easy to construct unts whch compute the dstance between a par of ponts. The network would then only need to learn a smple classfcaton functon - recognzng when 3 numbers are equal. Ths s the knd of task that smple backpropagaton networks have been successful at. The man dffculty, of course, s to transform the representaton from a set of pxel values to a set of vertex coordnates. Ullman (1987) has argued that many hgh-level vsual tasks may be mplemented as vsual routnes. The dea s that a hgh-level system solves a vsual task by choosng and combnng a set of prmtve vsual operatons. He has suggested some general purpose sequental prmtves whch mght be useful. Among them 1

3 Evdence for Sequental Vsual Processng n the Bran are prmtves for decdng whch portons of the mage are relevant, selectng out these sectons, and storng ther locatons for later processng. Gven these prmtves, a natural soluton to the trangle problem s a system whch uses a focus of attenton to sequentally select the vertces and store ther locatons. The vsual routnes framework s well-suted to our task, but a connectonst soluton s non-trval. Ths report descrbes our mplementaton of prmtves for detectng and rememberng the locatons of clusters of ponts based on an effcent focus of attenton mechansm. 2. Evdence for Sequental Vsual Processng n the Bran There s a large body of psychology lterature whch supports the dea of sequental vsual routnes. One of the best examples s the work by Tresman and her colleagues (Tresman & Gormcan, 1988). Ther work suggests that certan smple vsual tasks are performed by people n parallel (response tme s ndependent of the number of objects n the mage) whereas other tasks requre seral processng (response tme s lnear n the number of objects). Jolcoeur et al. (1986) have provded further evdence. They fnd that the tme to report whether two stmul le on the same curve ncreases lnearly wth the dstance between them along the curve. In both cases saccadc eye movements are ruled out suggestng that nternal processes are responsble for the sequental behavor. An attentonal mechansm whch can selectvely nhbt or excte regons of the vsual nput s central to the explanatons of the above results (Ullman, 1987, Tresman & Gormcan, 1988). Varous psychophyscal experments have attempted to pn down the precse characterstcs of ths mechansm. It was shown n (Posner et. al, 1982) that the mechansm was very fast wth notceable effects begnnng as soon as 50 msecs after presentatons of a cue. They also reported a very flexble scheme for drectng the attenton to dfferent loc. A shft of attenton can be nduced by the appearance of a cue, the absence of a cue, or even the expected onset of a cue. Ths suggests that a combnaton of bottom-up and top-down nformaton s used to decde where to focus attenton. (Shulman et. al, 1979) have provded evdence that the regon of nfluence moves contnuously from one pont to another. (Erksen & Yeh, 1985) have shown that attenton s allocated at only one contguous regon n any gven nstance but suggest that the area of nfluence can vary contnuously n sze. There have also been neurophysologcal studes showng that some neurons can dynamcally change ther response propertes. Moran & Desmone (1985) report evdence that the szes and locatons of the receptve felds of certan neurons n the monkey vsual cortex (areas V4 and IT) change wth the task that the anmal s tryng to accomplsh. 3. Implementng a Focus of Attenton Along wth these dscoveres a number of detaled models have been proposed for mplementng selectve attenton. In the next secton we brefly revew the most relevant ones and then descrbe our mechansm. 3.1 Prevous work (Koch and Ullman, 1985) have suggested a model based on a salency map combned wth a pyramdal wnner-take-all network. The salency map conssts of a retnotopc grd of neurons whch accumulate bottomup nformaton from varous feature maps. Actvty of a neuron n ths map s proportonal to the relevance of the correspondng locaton. The model then uses a parallel log-depth tree to compute the most salent val- 2

4 Implementng a Focus of Attenton ue. Interor nodes at each level compute the max of ther ncomng sgnals and transmt t to the next level. The tme to focus on a spot s thus logarthmc n the number of pxels. Chapman (1990) has extended the model to allow ponter addressng. To move the focus to an arbtrary locaton an address s routed from the top of the pyramd down to the approprate leaf. The value at that leaf s then passed up the tree to the root node. (Mozer, 1988) has descrbed a model of selectve attenton whch elmnates the need for a log-depth tree. Hs attentonal mechansm (AM) conssts of a layer of retnotopc unts. The layer receves nput from all the feature maps. Unts n the AM gate the outputs of correspondng unts n the feature maps. Thus a sngle regon of actve unts n the AM corresponds to a sngle regon of actvty n the feature maps. A locally compettve rule s used to decde whch unts n the AM should be actve. At each step the current actvty of each unt s averaged wth the actvty of ts neghbors mnus the actvty of all other unts. After some teratons the network stablzes to form a roughly crcular spotlght representng the most actve regon. One advantage of ths scheme s that the focus of attenton s a crcular regon nstead of a sngle pont 1. In addton, there s a parameter whch can roughly nfluence the sze of the crcle. However, snce regon selecton s based on local competton, the tme requred before the network stablzes on a sngle regon s qute senstve to dfferent mages. It could be qute large f there are several smlar regons of actvty whch are wdely separated n the mage. (Fukushma, 1986) descrbes a model of selectve attenton whch combnes aspects of the above two systems. Hs system conssts of a log-depth herarchcal network n whch the output level contans one unt per pattern. Gven an mage wth two or more patterns the output nodes correspondng to the dfferent patterns wll respond. The unt wth the largest response s selected and a sgnal s transmtted back through the pathways actvated by ths partcular pattern. Ths creates a postve feedback loop whch sharpens the detecton of ths pattern. The pathways correspondng to the other patterns gradually attenuate n the absence of ths facltaton. Thus the network attends to one of the nput patterns. One advantage of ths scheme s that the shape of the focus of attenton depends only on the shape of the nput patterns and thus can be arbtrary. However the process s qute slow snce many teratons may be requred for the network to settle, and for each teraton the actvty has to flow up and down the herarchy. 3.2 A Constant Tme Mechansm for Selectve Attenton In all of the above models two fundamentally dfferent operatons are bundled together: the mechansm for selectng the locaton of the focus and the mechansm for suppressng rrelevant regons of the mage and allowng relevant portons through. Ths affects the tme to actually shft to a new locaton as well as lmtng the flexblty of the models. A response tme of 50 msecs (Posner et. al, 1982) only allows 5 to 10 neurons to fre n sequence. Consderng the fact that actvty may have to flow through several pre-processng layers before reachng the attenton system, there doesn t seem to be much tme for a relaxaton model (such as Mozer s and Fukushma s) to settle or for actvty to flow up herarches. In our model we separate these two operatons. Our attenton mechansm s able to select any crcular porton of the nput space n two tme steps. So gven a new locaton the system s able to effcently dsengage from the prevous locaton and focus on the new one. The locatons of nterest are determned by external networks and can nclude top-down as well as bottom up nformaton. (In our current mplementaton we use a cluster detector to detect the vertces. See Secton 4.) In the next few sectons we descrbe the detals of our system. 1. See (Chapman, 1990) for an mplementaton whch removes ths restrcton somewhat. 3

5 Implementng a Focus of Attenton z = x 2 + y 2 parabolod Threshold plane y c 1 c 2 (x1-a1) 2 (x2-a2) 2 c n (xn-a n ) 2 crcle (a) x x 1 a1 x 2 a 2 x n Fgure 2 (a) The plane ntersects the parabolod n a curve whch projects to a crcle. (b) Archtecture of threshold unt computng the ntersecton n n dmensons. (b) a n R 2 Locally Tuned Receptve Felds We frst descrbe a mechansm by whch lnear threshold unts can gve a localzed response n a feature space. The followng fact s exploted: f one maps the ponts n R n 1 onto the parabolod defned by n 1 2 z = x, then the ntersecton of a hyperplane n R n wth ths parabolod projects onto a sphere n R n 1. = 1 Thus there s a mappng between planes n R n and spheres n R n 1. To select a set of ponts whch le wthn a sphere n some space one just has to project the ponts onto the parabolod and slce t wth the plane correspondng to the sphere. Ponts whch le beneath the plane are wthn the sphere. Fgure 2 llustrates ths for R 2. Notce that the computaton of a threshold unt s exactly that of decdng on whch sde of a hyperplane an nput pont les. To encode crcular receptve felds wth threshold unts, you just need to nclude an extra nput: the sum of the squares of all the other nputs. An equaton of the form: ( w x + 2 x + const) > 0 (1) wll be postve f x les wthn a sphercal volume determned by the weghts and constant. The above method creates crcular receptve felds wth hard boundares. A smooth boundary wth a flat top may be obtaned by usng a sgmod nstead of a threshold functon. The steepness of the sgmod (ts gan) wll then control the steepness of the receptve feld boundary. Non-crcular receptve felds are also possble by changng the nature of the non-lnearty. Ellptcal receptve felds may be obtaned by usng the parabolod z = c n 1 x2 where c denotes the amount of stretchng along each axs. In prncple arbtrary shapes = 1 can be obtaned by approprately choosng the non-lnearty. Dynamc Receptve Felds In addton to beng able to select a porton of the nput space, we need the ablty to shft the locaton and sze of the receptve feld around quckly n response to changng demands. In our model there are two ways to do ths. The frst method nvolves changng the slope of the hyperplane. In Fgure 2 (a) note that as the slope ncreases the center of the projected crcle wll shft to the rght. For any sphere t s possble to compute the coeffcents of the hyperplane whch produces that sphere. Gven a plane m x = m c where m and c are real-valued vectors, the projecton of the ntersecton of the plane wth the parabolod s a sphere whose center s: ( a 1, a 2,, a n 1 ) = m 1 mn,, 1 m n m n (2) 4

6 Implementng a Focus of Attenton and whose radus s: In a threshold unt, changng the slope of the hyperplane corresponds to changng the weghts of the nputs. One of these unts could eventually learn the correct poston of ts receptve feld, however the tme scale for weght changes s too slow for dynamc computatons. Another way to alter the sphere s to fx the plane but shft the parabolod, by computng n 1 z = c ( x a ) 2 + r 2. Ths moves t a dstance a along dmenson (changng the locaton of the = 1 sphere) and a dstance r 2 along the z-axs (changng the radus of the sphere). If the a s and r 2 are avalable as nput then the receptve feld can be changed an arbtrary amount n one tme step. Fgure 2(b) shows how such a unt would be confgured. For each nput dmenson there s a sub-unt whch computes the square. ((Suarez & Koch, 1989) present a neurally plausble mechansm for computng a quadratc.) The outputs of these unts are fed nto a threshold (or sgmod) unt. The net effect s that the threshold unt wll respond only when the nput vector x les wthn the sphercal receptve feld determned by a and r. Focus Of Attenton Wth Value Coded Unts R = m m n 1 + 4m n ( m c) 2m n m 2 2 So far we have assumed an n-dmensonal nput space that s encoded as n analog sgnals. In our trangle task however, we have to mplement a crcular patch n a 2-dmensonal retna. The unts n ths representaton are lad out on a flat sheet, wth each unt explctly encodng a pont n the space. To create dynamc receptve felds here, we construct a gatng layer, wth one gate unt per nput unt. Each gate unt receves three global nputs: A x, A y, and A r representng the parameters of the focus of attenton. The gate unts are bascally the same as the localzed unts descrbed above wth one small modfcaton: x 1 and x 2 are fxed for each gate unt. To encode ths we have two separate connectons from the nput unt to the gate unt. The weghts of these connectons are x 1 and x 2. If the nput unt fres, the threshold unt wll fre only when ts nput unt s wthn the crcle determned by A x, A y, and A r. The effect s a layer of unts whch flters the nput mage accordng to a global control sgnal. The system can select any crcular porton of the mage n one tme step. The hardware requred to mplement ths s mnmal: 8 extra connectons per nput unt. It s also farly easy to extend our mechansm to allow foc of dfferent shapes once the parameters are avalable. Fgure 3 shows the graphcal output of our smulator for two dfferent mages. In each dsplay, the lower left quadrant dsplays the current nput mage (3 pont clusters). The upper left quadrant shows the output of the gate unts. The crcle shows the mplct focus of attenton represented by the 3 parameters. In Fgure 3 (a) note that only the actvty wthn the focus s allowed to propagate. 2 (3) 4. Decdng Where To Focus... And How To Get There. A focus of attenton system s useless wthout a mechansm to select the nterestng locatons. In general, many dfferent crtera can be used to defne nterestng. One possblty s a bottom up method whch depends on the characterstcs of the current mage (.e. choosng the brghtest mage pont). The locaton could also be chosen n a top-down fashon (.e. as a result of pror expectatons). In our doman, the natural crteron s the cluster wth the largest number of ponts. Note that our attenton mechansm tself s ndependent of the partcular crteron used - all t requres are the coordnates of the locaton. 5

7 Implementng a Focus of Attenton (a) (b) Fgure 3 Examples of the system behavor for a 256x256 mage wth seven clusters of ponts. In both dsplays, the lower left quadrant shows the mage; the upper left quadrant shows the output of the gate unts. The error vectors are dsplayed n the upper rght and the outputs of selected unts are shown n the lower rght. (a) shows a snapshot of the system wth the focus of attenton near one of the clusters. (b) shows the dynamc scalng behavor as the focus tres to ft the cluster of ponts wthn t. Our method uses a coarse grd of error unts. Encoded n each locaton n ths grd s a value representng ts mportance and values encodng the dscrepancy between the current focus of attenton and the cluster wthn ts receptve feld. The most actve grd locaton thus provdes a rough estmate of the next locaton to vst. Once attenton has been drected to that locaton, t s fne tuned to settle exactly on the center of mass of the cluster. These two systems are descrbed n detal n the followng two sectons. 4.1 Fne Tunng the Focus of Attenton We frst descrbe a mechansm for fne tunng A x, A y, and A r to settle on the center and sze of the cluster of ponts wthn the current focus of attenton. The center of mass, ( C x, C y ), s defned as the average of the x and y coordnates of the actve ponts: C x X ( ) a Y ( ) a = and C y = (4) a where X ( ) and Y ( ) denote the x and y coordnates of the th unt and a denotes ts actvty. a can be computed by a unt whch receves nput from all gate unts wth a weght of 1 (unt 1 n Fgure 4). To compute the numerators we nclude two unts wth lnks to every gate unt (unts 2 and 3). The weghts from the th gate unt to each of these two unts are X ( ) and Y ( ), respectvely. A weghted sum of ther ncomng actvty thus computes X ( ) a and Y ( ) a. Two unts (unts 4 and 5) perform the dvson to obtan C x and C y. These values are fed nto two more unts whch compute the dfference between the center of mass and the attenton parameters: ( C x A x, C y A y ). The unts representng A x and A y receve as nput ths dfference as well as ther own output. By computng the sum of the two nputs, they contnually update the focus to the center of mass of the ponts wthn t. To get an estmate of the sze of the cluster we also nclude a unt whch contnually adjusts the sze of the a 6

8 Implementng a Focus of Attenton A y A x A r C x 4 C y Y() X() Gate Unts focus of attenton to match the sze of the set of ponts wthn t. As long as a remans constant, the scalng unt decreases A r by a small amount. If the sum decreases, ndcatng that the scale has become too small, the unt ncreases A r slghtly and stops. Fgure 3 (b) llustrates ths scheme on our smulator. The focus of attenton s ntalzed to contan all three clusters and be slghtly off center (dotted crcle). The set of concentrc bands show successve steps as the focus of attenton decreases n sze and shfts ts locaton to ft the cluster nsde. 4.2 Error Unts Fgure 3 The unts whch contnually fne tune the focus of attenton. See text for explanaton. The mechansm descrbed above does not gve us a way to fxate on clusters outsde the focus. To do ths we nclude a coarse grd of unts each of whch receves nput from a small patch n the mage. At each grd locaton there are three outputs. The frst two outputs encode the dfference between the center of mass of the cluster of ponts wthn ther receptve felds and the pont ( A x, A y ). In ths way each grd locaton encodes an error vector for adjustng the focus of attenton. These error vectors are contnually updated to compensate for changes n A x and A y. The thrd unt represents a confdence value from 0 to 1, ndcatng the mportance of ts locaton. By addng the vector wth the hghest confdence value to A x and A y, the focus of attenton can be shfted to the locaton of the most salent cluster. The focus can be shfted to the nearest locaton by selectng the locaton wth the smallest error vector. The error vector representaton was nspred by dscoveres of a smlar mechansm n the monkey superor collculus for controllng eye saccades (Sparks, 1986). The output of the confdence unts are smlar n sprt to the salency map n (Koch & Ullman, 1985). In general many factors may contrbute to the salency of a gven locaton. In our mplementaton the salency s smply the number of actve ponts wthn the receptve feld passed through a sgmod. The upper rght quadrant of Fgure 3 (b) show some example error vectors. (Only those locatons whose confdence value s hgher than 0.2 s dsplayed.) Each arrow represents the error vector for that locaton. 7

9 Storng Locatons Output Hdden Unt - -1 Hdden Unt - Input Control Fgure 4 Schematc of a bndng network. The network contnually tracks ts nput sgnal untl a control sgnal s sent, at whch pont the output s frozen to be the current sgnal. The shaded square represents the confdence value - the darker the square the hgher the confdence. 5. Storng Locatons As the network vsts each vertex t should store the values of A x, A y, and A r whenever the focus of attenton stablzes. We accomplsh ths wth small recurrent networks for each value that needs to be stored. Each of these bndng networks contnually tracks a partcular unt (one of A x, A y, and A r ) untl a control sgnal s sent, whereupon t freezes the output to be the current value of the unt. Ths s done by wth the network shown n Fgure 5. Whle the control unt s off, the hdden unts compute the dfference between the value of the assgned unt and the current output and sends t to the output unt. The output functon of the hdden unts s lnear whle ts net nput s postve, zero otherwse. The left hdden unt ndcates when the output should be decreased whereas the rght unt ndcates when t should be ncreased. When the control unt s turned on, the hdden unts are shut off by large negatve weghts. An exctatory lnk from the control unt to tself ensures that once the control unt has fred, t stays on, preventng further adjustments. Three of these bndng networks are used for each set of parameters that are stored. There s one other ssue to consder. As the network fxates on successve clusters, we would lke dfferent sets of bndng networks to be nstantated. To do ths we need some sort of a sequencng mechansm whch wll send control sgnals to successve bndng networks. Ths s accomplshed by the network shown n Fgure 6. The sgnal unt fres when the focus of attenton has stablzed for three teratons. On successve frngs of the sgnal unt, approprate control unts are turned on. (The unt cntrl- n the fgure corresponds to the control sgnal for the th bndng network.) Fgure 8 (b) shows an example of ths. The bottom left quadrant shows the output of seven sets of three bndng unts, each representng the scale, x and y coordnates of the focus of attenton. For example the frst set of unts has been frozen to (161,116,114) whereas the rest of them are stll free to follow the attenton parameters. If a control sgnal were to be generated now, the second set would be frozen at (174,29,8) to represent the current parameters Fgure8 (c). 8

10 System Archtecture Cntrl-1 Cntrl-2 Cntrl (a) Sgnal A r (b) Sgnal cntrl-1 fres cntrl-2 fres cntrl-3 fres Fgure 6 A detaled dagram of a porton of the sequencer. The unts shown above sends control sgnals to the approprate bndng networks. The sgnal unt fres when all of ts nputs are equal. The frst tme t fres, cntrl-1 starts frng. The next tme, cntrl-2 fres, and so on. Ths structure s used to send control sgnals to successve bndng networks. 6. System Archtecture Fgure 7 shows a schematc of the whole archtecture. The module whch controls the attenton feld s an autonomous network that contnually attempts to wrap the focus around the ponts wthn ts feld of vew. When t has stablzed, the sgnal unt fres, a control sgnal s transmtted to the next bndng network and the focus of attenton s updated to the locaton of the next cluster. To sequentally process each cluster n the mage the system has to repeatedly select the largest confdence value, nhbt the correspondng unt, and send the error vector to the system controllng A x and A y. One way to do ths s to construct a wnner-take-all network wth competng confdence unts such that the system settles nto a state where only one unt s actve. However these networks can take a relatvely long tme to settle, especally when the competng values are very smlar. It s also qute dffcult to fnd the correct set of nhbtory weghts to create a robust wnner-take-all network. Another alternatve s to construct a log- 9

11 Smulaton Bndng networks Dstance unts R X Y A x, A y, A r Output Unt Hdden unts Attenton Control Sequencer A x, A y, A r Gate unts Error unts Input unts Fgure 5 Basc system archtecture. The module Attenton Control contnually tres to ft the focus of attenton to the ponts wthn t. The Sequencer updates the focus of attenton to vst all the clusters n sequence and also sends the control sgnals to the bndng network to store successve locatons. depth network as n (Koch and Ullman, 1985) to explctly compute the maxmum. In our current mplementaton we assume a unt wth a bult n max functon but are studyng other schemes for performng ths computaton effcently. Assumng that the system starts wth a focus of attenton coverng the entre mage plane, the network frst wraps the focus around the trangle and then sequentally vsts and stores the locatons of the three vertces. Once ths s done, the frst set of bndngs encode the poston and scale of the trangle. The next nne bndngs encode the postons and scales of the three vertces n the order that they were processed. A set of dstance unts then explctly computes the sx possble dstances between the frst four stored locatons. A standard feedforward network wth one layer of hdden unts s used to compute the fnal output. 7. Smulaton For the smulatons n ths paper we used 256x256 mages. The complete network conssted of 131,912 unts, 131,072 of whch were the nput and gate unts. We generated a tranng set consstng of random trangles (approxmately 50% of whch were equlateral) wth Gaussan nose added around each vertex. For each trangle the focus of attenton was ntalzed to cover the entre mage plane. The system was allowed to run untl all the bndng unts were frozen. The outputs of the frst 4 bndng unts were fed to 6 dstance unts whose outputs were then used as nputs to tran the backprop network. The teacher sgnal was gener- 10

12 Smulaton Fgure 7 Results of parsng an equlateral trangle and a non-equlateral trangle. (Only the frst four bndng unts are used here.) ated accordng to: l 1 l 2 + l 2 l 3 + l 1 l 3 1 l 1 + l 2 + l 3 where l s the length of the th sde. Ths s a functon whch s 1 for equlateral trangles and degrades gradually to 0 as the trangles devate from equlateralness. Wth a tranng set of only 100 trangles the network score was consstently greater than 0.9 for equlateral trangles on ndependent test sets. Note that, snce the nputs to the feed-forward network are relatonal, the number of tranng examples necessary for good generalzaton does not depend on the sze of the mage. Fgure 8 shows a seres of mages from our smulator at several stages of a typcal recognton sequence. Fgure 9 shows the state of the network after parsng two dfferent trangles. The system correctly classfed the left trangle as beng equlateral and the rght one as not beng equlateral. The outputs of the bndng networks show the vertex coordnates that were dscovered by the network. (5) 8. Dscusson We have descrbed a network for extractng the locatons of pont clusters. Although the system s geared towards a narrow class of mages, t s temptng to speculate how these mechansms would ft nto a general purpose vson system. The focus of attenton mechansm tself s ndependent of the actual mage and could work n any real-valued space. The system should be easly adaptable for general vsual search of the type dscussed n (Tresman & Gormcan, 1988) whch requres the ablty to solate regons n dfferent feature maps. Mnmal hardware s requred to adapt our system work wth orentaton selectve neurons, color detectors, etc. The noton of what locatons are nterestng would also need to be modfed to nclude clusters n arbtrary feature maps. The above deas would represent relatvely straghtforward extensons to the system, but there are some 11

13 Smulaton (a) (b) (c) (d) Fgure 6 Four steps n a typcal run. (a) Determnng the sze and locaton of the whole trangle. (b) - (d) Network after fxatng on each of the three vertces. Note that the bndng networks are updated correctly. Once all four postons are avalable, the network correctly classfes the trangle to be equlateral (bottom rght of (d)). more dffcult ssues whch also have to be dealt wth. The lbrary of prmtves must be expanded to handle more features of realstc mages. We would need to extract relatons based on curves, regons, shapes, etc., and so would need the approprate prmtves. For example, (Jolcoeur et. al, 1986) have provded evdence that people use a curve-followng routne to determne whether two ponts le on the same curve. Fnally, a major ssue that we have not addressed s the ssue of complng multple vsual prmtves to accomplsh a dynamcally specfed task. 12

14 Acknowledgments In concluson, a central pont of ths paper has been to demonstrate mechansms for performng sequental vsual computatons n connectonst networks. We have descrbed an effcent mplementaton of some prmtves, and have used them to extract mage propertes that are neffcently represented n parallel. There s evdence from psychology and neurophysology that bologcal organsms actually mplement smlar routnes at an early processng level. 9. Acknowledgments We thank Jerome Feldman for suggestng equlateral trangles as an nterestng doman to study. We would also lke to acknowledge Chedsada Chnrungrueng, Davd Chapman, Chrstof Koch, Terry Reger, and Andreas Stolcke for helpful dscusson. 10. References Ahmad, S., and Omohundro, S. (1990) Equlateral Trangles: A Challenge for Connectonst Vson. To appear n: Proceedngs of the 12th Annual Meetng of the Cogntve Scence Socety, MIT, Chapman, D. (1990). Vson, Instructon, and Acton. Ph.D. Thess, Massachusetts Insttute of Technology. Also MIT AI-Lab Techncal Report #1204. Erksen, C.W. and Yeh, Y. (1985). Allocaton of Attenton n the Vsual Feld. Journal of Expermental Psychology: Human Percepton and Performance, 11 (5), pp Fukushma, K. (1986) A Neural Network Model for Selectve Attenton n Vsual Pattern Recognton. Bologcal Cybernetcs, 55, pp Gles, C.L., Grffn, R.D., & Maxwell, T. (1987). Encodng Geometrc Invarances n Hgher Order Neural Networks. In Advances n Neural Informaton Processng, Davd Touretzky, Ed. Morgan Kaufmann. Jolcoeur, P., Ullman, S., and Mackay, M. (1986). Curve tracng: A possble basc operaton n the percepton of spatal relatons. Memory and Cognton, 14 (2), pp Koch, C. and Ullman, S. (1985) Shfts n selectve attenton: towards the underlyng neural crcutry. Human Neurobology, 4, pp Mnsky, M. & Papert, S. (1969). Perceptrons: An Introducton to Computatonal Geometry. MIT Press, Cambrdge, MA. Mozer, M. (1988). A Connectonst Model of Selectve Attenton n Vsual Percepton. Unversty of Toronto Techncal Report CRG-TR-88-4 Posner, M.I., Cohen, Y., and Rafal, R.D. (1982). Neural Systems Control of Spatal Orentng. Phl. Trans. R. Soc. Lond. B 298, pp Sparks, D. L. (1986). Translaton of Sensory Sgnals nto Commands for Control of Saccadc Eye Movements: Role of Prmate Superor Collculus, Physologcal Revews, 66 (1). Suarez, H., & Koch, C. (1989). Lnkng Lnear Threshold Unts wth Quadratc Models of Moton Percepton. Neural Computaton, 1 (3), pp Tresman, Anne, and Gormcan, Stephen. (1988) Feature Analyss n Early Vson: Evdence from Search Asymmetres. Psychologcal Revew, 95 (1). Ullman, S. (1984) Vsual Routnes. Cognton, 18, pp