Nearest Neighbor Search using Additive Binary Tree

Transcription

1 Nearest Neighbor Search using Aitive Binary Tree Sung-Hyuk Cha an Sargur N. Srihari Center of Excellence for Document Analysis an Recognition State University of New York at Buffalo, U. S. A. Abstract Classifying an unknown input is a funamental problem in Pattern Recognition. One stanar metho is fining its nearest neighbors in a reference set. It woul be very time consuming if compute feature by feature for all templates in the reference set; this naïve metho is O(n) where n is the number of templates in the reference set an is the number of features or imension. For this reason, we present a technique for quickly eliminating most templates from consieration as possible neighbors. The remaining caniate templates are then evaluate feature by feature against the query vector. We utilize frequencies of features as a pre-processing to reuce query processing time buren. The most notable avantage of the new metho over other existing techniques occurs where the number of features is large an the type of each feature is binary although it works for other type features. We improve our OCR system at least twice (without a threshol) or faster (with higher threshol value). Keywors Nearest Neighbor searching, Filtration, Aitive Binary Tree, OCR, GSC Classifier 1. Introuction One common metho for classifying an input vector of unknown class involves fining the most or top k similar templates in the reference set. Not surprisingly, this problem, so-calle k-nearest neighbor or simply KNN problem has receive a great eal of attention because of its significant an practical value in pattern recognition (see [, 3] for extensive surveys). One straight-forwar metho is computing feature by feature for all templates in the reference set an it takes O(n) where n is the number of templates in the reference set an is the number of features. This is very time consuming for users to wait for the output. Hence, there is a wealth of literature regaring computational expenses of the KNN problem ating as far back as 197. Papaimitiou an Bentley showe O(n 1= ) worst-case algorithm [13] an Frieman et al suggeste possible O(log n) expecte time algorithm [6]. There are three general algorithmic techniques for reucing the computational buren: computing partial istance, prestructuring, an eiting the store prototypes [4]. First, the partial istance technique is often calle a sequential ecision technique; ecision for match between two vectors can be mae before all features in the vector are examine. It requires a preetermine threshol value to reuce computation time. Next, the pre-structuring metho focuses on preprocessing the prototype set into certain well-organize structures for the fast classification processing. Many approaches utilizing multiimensional search trees that partition the space appear in the literature [7, 11, 1, 1]. In these approaches, the range of each feature must be large. Otherwise, if features are binary, we achieve little speeup. Furthermore, the imension of feature space must be low. Quite often in image pattern recognition, each feature is threshole for binarization an the imension is high. Finally, the prototype eiting or reuction metho reuces the size of prototype set to improve spee with sacrifice of accuracy. The conense nearest neighbor rule [9] an the reuce nearest neighbor rule [8] are use to select a subset of training samples to be the prototype set. In this approach, we must sacrifice accuracy for spee. Hong et al [1] successfully implemente a fast nearest neighbor classifier for the use of Japanese Character Recognition. They combine a non-iterative metho for CNN an RNN an a hierarchical prototype organization metho to achieve a great spee-up with a little accuracy rop. The new algorithm utilizes both partial istance an prestructuring techniques. We reuce computation time by using an Aitive Binary Tree (ABT) ata structure that contains aitive information which is frequency information in binary features. The iea behin the ABT approach in fining the nearest neighbor is filtration by which the unnecessary computation can be eliminate. It makes this approach unique from the others such as reunancy reuction or metric metho. First, take a quick glance at the reference set an select caniates for match. Next, take a harer look only at those caniates selecte from previous filtration to select a fewer caniates, an so on. After several filtration, take a complete thorough look only at the final caniates to verify them. All matches whose istance is less than or equal to a threshol are guarantee to be in all caniate sets. In this paper, we will escribe the aitive binary tree (ABT) an the new nearest neighbor search algorithm base on several famous similarity measures. We will give the simulate experimental results. Finally, we report the experimental results on our OCR system using the graient, structural an concavity or simply GSC classifier [5].. Preliminary The performance of pattern classification epens significantly on its efinition of similarity or issimilarity measure between pattern vectors. Several efinitions have been propose an encountere in various fiels such as information retrieval an biological taxonomy [4]. Among many

2 efinitions, Eucliean istance, absolute ifferences, an Minkowski istance are the most famous ones. Section 3 iscusses the algorithm where the istance measure function is the absolute ifference. We chose this efinition as it is the most simple measure to explain the new algorithm. Eucliean an Minkowski s istances are almost ientical when features are binary. The absolute ifference, well known as the city block or Manhattan metric,isefineasfollows: D[x y] = X i=1 jx i ; y i j (1) where is the imension of the vector. A normalize inner prouct appears often as a non-metric similarity function: x t y S[x y] = where p kxkkyk kxk = x t x () It is the cosine of the angle between two vectors. Another similarity efinition use in OCR system using GSC classifier evelope in CEDAR is: S[x y] = x t y + xt y (3) where x is the negation of the vector x an is the contribution factor ( > 1). This efinition has the highest accuracy in our OCR system among known efinitions an it performs best when = 1:9. The ifference among these efinitions can be explaine by its weight where features are binary. The Minkowski family efinition incluing Manhattan istance gives the same weight for the case where both patterns have the feature an the case where both patterns o not have the feature. The normalize inner prouct computes the creit for the case where both patterns have the feature but also reflects the other case as it is normalize. The efinition 3 gives a full creit for the case where both patterns have features an also allows to control the creit for the other case. Section 4 iscusses the thir efinition of similarity an OCR. We consier the case where a threshol value, t, is known. The KNN with threshol problem is that we reject a query vector if the nearest neighbor is not close enough, meaning that the istance excees t. LetM be a set of matches whose issimilarity value is less than or equal to t or whose similarity value is greater than or equal to t. Let C(x) enote the class of the vector x an n is the number of template vectors in the reference set, R. Definition 1 k-nearest neighbor with threshol Input: A query vector q, a reference set, R = fr 1 r r n g an a threshol value, t. ( on the votes of k -NN : jm jk Output: C(q) = on the votes of jm j -NN : < jm j <k Rejecte : M = In this moel, the classification ecision is mae by up to k nearest neighbors because not enough matches may be in the set M. This moel is frequently use in many applications consiering the fact that the higher reject rate usually lessens the error rate. We introuce an aitive binary tree ata structure, ABT in short. It is a binary tree whose noes are sums of their irect P chilren. Consier the example in Fig. P 1. The root is i=1 xi. Noes at the secon level are = i=1 x i an P i==+1 x i from left to right. Leaves are x 1 x x. Let B(x) be a list of noes, fb 1 (x) B (x) B ;1 (x)g in breath first orer. Property 1 Aitive Binary Tree 1. There are l;1 number of noes at level, l.. Total number of noes = ; The epth of ABT = log. 4. B i = B i + B i+1 The structure is name aitive because aitive informations prouce by property 4. are appene in every vector. Each element in a vector lies in the leaf level from left to right corresponingly. If vectors are binary, the root is the frequency of 1 s an each noe is the frequency of 1 s in subpart of the vector. 3. k-nn using ABT in City block istance There are three phases: the pre-processing, caniate selection an verification, an voting for classification phases. During the pre-processing phase, the ABT s for all templates are built by aing a pair of elements an storing it in their parent noe. Builing an ABT for one template takes O() an thus, total pre-processing takes O(n) to buil ABTs for all templates. The space require is also O(n) which is at most twice bigger than the size of the reference set. In the secon phase, we search for matches before choosing top k similar templates for votes. The objective of the caniate selection an verification proceure is quickly to fin the smallest subset M of the reference set which contains top or up to k templates. Consier the following pseuo coe for fining matches. Algorithm 1 Caniate Selection an Verification 1 Buil ABT(q) for every template x i 3 for every level l=1tolog 4 if P l ;1 j= l;1 jb j (x i ) ; B j (q)j >t 5 break 6 else if l = leaf 7 Verify the Match 8 ifverifie, upate t The inner loop computes the sum of absolute ifferences of every noe in level, l. The sequential ecision technique may be applie to this step; we may not have to compare all noes in the level because the value may excee the threshol before all noes in a specific noe are examine. We use the parent level operations as filtration functions. Those caniate templates survive from the parent level are only consiere in the next level. After fining all matches, search for top or k-nearest neighbors only in the set, M. In the line 8, we woul like to upate the threshol if k number of matches are foun an maximum istance is less

3 than t. This is critical because the lower the t is, the more filtration occurs. Also, if a threshol value is not given at the start, then it is assigne at the point right after k templates are examine an upate if a less value is foun later on Example Consier a following set with n =3 =8an t =, q = 1 R = 8< : r 1 = r = 111 r 3 = 111 When a brute force metho is performe, 4 number of comparisons take place. The unerline parts of each reference vectors are those contributing the comparisons when the sequential ecision technique is use; there are 19 number of comparisons. Fig. 1 shows ABTs for the query vector an all templates. First, the root level is compare. r an r 3 are r q r Figure 1. Sample ABTs 1 r consiere as caniates. r 1 is iscare because (jb 1 (r 1 ) ; B 1 (q)j =4)> (t =). Next, the secon level of trees are compare. For r, jb (r ) ; B (q)j + jb 3 (r ) ; B 3 (q)j = while that of r 3 is 4. But only the B (r ) is compute because jb (r ) ; B (q)j alreay excees the threshol. Hence, only r is consiere as a caniate. At the level 3, r still is a caniate an at the leaf level, it is verifie as the nearest neighbor to q. jrj =3 jl 1 j = jl j =1 jl 3 j =1 jm j =1The number of comparisons occurre is 18, the number of circle noes as shown in the example in Fig Correctness Let f M an f l be functions for matching an for selecting caniates at P P the level, l: f M (x q) = i=1 jx i ; q i j an f l (x q) = l ;1 i= jb i(x) ; B l;1 i (q)j. LetL l be a set of templates, x s chosen by the function f l (x q) t. Theorem 1 A set of matches is always a subset of all caniate sets. M = L log L l L l;1 L 1 R. Proof: f L1 f Llog = f M. Prove by the wellknown fact ja + b + cj jaj + jbj + jcj. For example, x = (x 1 x x 3 x 4 ) an q = (q 1 q q 3 q 4 ). Then, f M (x q) = jx 1 ; q 1 j + jx ; q j + jx 3 ; q 3 j + jx 4 ; q 4 j an f l=1 (x q) = jx 1 + x + x 3 + x 4 ; q 1 ; q ; q 3 ; q 4 j.anf l= (x q) = jx 1 +x ;q 1 ;q j+jx 3 +x 4 ;q 3 ;q 4 j. Clearly, f 1 f f M. Now, if there exists x M meaning that f M t,thenf l t an x L l. Thus, M L l+1 L l L l;1 L 1 R. The theorem means that all patterns whose istance is within the threshol are guarantee in all caniate sets. Hence, this metho is a sort of ynamic prototype reuction metho but there is no loss of accuracy at all. It is as accurate as the brute force search metho but much faster. The extra space require for ABT is ;1 per vector which is the number of inner noes by efinition. Therefore, the total space is O(n). Selecting Caniate process is mae in the fewer computational time than that of the verification process. Number of comparisons neee at the level, l, is the number of noes at the level which is l;1. That of the top root level, for example, is 1 while that of the leave level is m. The number of operations is multiplie by in every filtration process. The number of caniates is, on the other han, reuce in every filtration process Simulate experiment We present simulation results which emonstrate the performance of the new algorithm on 4 ifferent reference sets with ifferent feature numbers, = f g. The size of the reference set, jrj = 1 for each reference set. Each feature in vectors is generate by a ranom function which returns a number between an 99. Possible min(d(x q)) = which is an exact match an possible max(d(x q)) = 6336 when = 64. Corresponing test sets, Q s with = f g, are prepare as well. The size of each query set is jqj = 1. The experiment is performe on machine with a 3MHz UltraSparc CPU, SunOS 5.6, 588 MIPS, an56 Memory. Fig.shows the performance of a naïve metho an methos using ABT structure with several ifferent threshol values. "8" t=the threshol 6 "3" t=the threshol "16" t=the threshol 1 "64" t=the threshol Figure. Cumulate elapse time for 1, queries over 1, templates Observation 1 The smaller threshol value, t, the faster ABT algorithm runs but the reject rate increases. A smaller threshol means fewer caniates an the algorithm runs faster. As the threshol increases, on the contrary, caniates aboun. Too small threshol, on the other han, might reject most of inputs. Observation The larger imension is, the better performance the ABT algorithm achieves the better execution time performance. 3

4 It is necessary to compare the results of s with the same ratio of t. Consier the case that =64 t = 3, then the ABT runs 13:8 times faster than the naive metho. On the other hans, when =3 t= 15, it is only 9:6 times faster. Filtering from top to all the way own to the level, l ; 1 may not give always the most expeitious running time. It woul be wise to stop filtering an jump to the verification stage if the number of caniates is small. Table 1 inicates this behavior. Table 1. Comparisons for methos. Metho time in millisec. Naïve 13.3 Sequential Decision 4.1 Filtration l = Filtration l = 1.4 Filtration l =3 7. Filtration l =4 9.5 Observation 3 There exists a level, ^l which results the minimum elapse time. The eeper the level is, the more computations which is the number of noes at the level, are require. ^l is the starting point that jl^l j^l j^l+1 jl^l+1 which means that we gain no avantage of filtration from this level. Therefore, fining ^l is important because it gives not only the minimum elapse time, but also less require space for ABT ata structure. Observation 4 The ABT technique is superior to the sequential ecision technique experimentally. In the worst case when every reference is a match, both sequential ecision an ABT woul not give any spee up. Technique using ABT woul be even twice slower than the naive metho in case of a full ABT Auxiliary Lookup Table: Suppose =51an all features are binary, then a vector is store as an array of 16 unsigne integer values. We buil a lookup-table for counts of 1 s for all 16 bit combination. The size of lookup-table is 6K bytes. This look-up table for unsigne integers is built at preprocessing stage. To fin D[x y], we perform table look-ups for high an low 16 bits of each unsigne integer (= x y). There are total 3 table look-ups to etermine the istance. Where vectors are store as unsigne integers, we consier counts of 1 s of each unsigne integer value as leaves of ABT. Lookup table enables significant speeup over the metho counting the number of bits without the table. Orere List: The simulate experiment tells that the smaller threshol is, the faster algorithm runs (see Observation 1). The threshol value is ynamically change as more templates are examine. It woul be esirable if the smaller threshol value is assigne early. Therefore, we woul like to orer the templates so that the less threshol value is set early in the search. Fact 1 The closer in frequency two vectors are, the smaller in absolute ifference istance they ten to be. If we orer templates by its frequency an search from a template whose ifference of frequency is smallest, then the low threshol value might be set in the earlier search stage resulting in a great spee up. First, orer the reference set by B 1. For each bin, the average number of templates is n=m. Next, orer each bin by B. During the search stage, start searching from the template, x whose min(jb 1 (x) ; B 1 (q)j) an then min(jb (x) ; B (q)j). As a result of orering, not only we fin the low threshol value quickly, but also, our search space is reuce own to the size of L. Selection: We have seen that ABT facilitates filtrating some reference templates from consieration by using the lower bouns at each level. In aition to the lower bouns, ABT also provies upper bouns. If an upper boun is less than or equal to the threshol, the match is verifie without further calculation. This selection technique is useful in a query such that we woul like to fin all matches to the query vector within a threshol. Consier w-ary vectors x an y where each element can have a value between an w ; 1. Lemma 1 D[x y] has upper an lower bouns at the root level: jb 1 (x) ; B 1 (y)j D[x y] min(b 1 (x) + B 1 (y) (w ; B 1 (x)) + (w ; B 1 (y))). P i=1 jx i ; y i j P i=1 x i + P Proof: The lower boun is given in Theorem 1. Similarly, P i=1 y i because P ja + bj jaj + jbj. Now, D[x y] = i=1 jx i ; y i j = i=1 j(w ; x i ) ; (w ; y i )j. This is the absolute P ifference of two inverse vectors an it is the same as the absolute ifference of the original vectors. Clearly, i=1 j(w ; x i) ; (w ; y i )j P i=1 (w ; x i)+ P i=1 (w ; y i) Thus, D[x y] min( P i=1 x i + P i=1 y i P i=1 (w ; x i)+ P i=1 (w ; y i)) Consier two vectors, x y where they are binary, w = x = : B1 (x) =5 1. Accoring to the y = 1111 : B 1 (y) =4 lemma 1, we achieve 1 D[x y] 8 immeiately. Theorem P l ;1 i= l;1 (Bi(x) ; Bi(y)) D[x y] P min( l ;1 (Bi(x) + Bi(y)) P l ;1 (( w i= l;1 i= l;1 l ( w l ; Bi(y)))) ; Bi(x)) + Proof: Again, the lower bouns are prove in Theorem 1. For the upper bouns, ivie vectors into l;1 number of subvectors. For each sub-vectors, Lemma 1 hols. For the above example, we have the upper an lower bouns at the secon level of ABT: 1 D[x y] Using ABT for GSC classifier Among many classifiers in OCR,theGraient, Structural, an Concavity classifier, simply known as GSC classifier,has 98% accuracy on 4 hanwritten igits taken from various atabases [5]. It is base on the philosophy that feature sets can be esigne to extract certain types of information from the image. These types are graient, structural, an concavity informations. Graient features use the stroke shapes on a small scale. Next, structural features are base on the 4

5 stroke trajectories on the intermeiate scale. Finally, concavity features use stroke relationship at long istances. Graient, Structural, an Concavity has 19, 19, an18 feature vectors corresponingly. In all, there are 51 number of features in a vector. The K-NN approach is often use in the GSC classification. It selects the top k similar templates in the training set an returnsits class base on the vote of thesek template vectors. The efinition of this similarity, S[x y], currently use in the GSC classifier follows: S[x y] = nx i= S[xi yy] where S[xi yi] =( 1 : xi = yi =1 1= : xi = yi = : otherwise When both x i an y i are 1 s, it is the case that the esire feature exists. It is enote as S 11 [x y] = x t y. When both x i an y i are s, it is the case that the feature oes not exist. It is enote as S [x y] = x t y. It woul be reasonable to rely more upon S 11 than S. The efinition of the similarity measure becomes : S[x y] =S 11 [x y] + S [x y] is the contribution factor an usually 1. The recent experiment shows that = 1:9 gives the best performance. Note that when =1, it is the inverse of the city block istance between two vectors k-nn for GSC classier There is a slight ifference in line 4 of the previous algorithm 1 as its efinition of similarity iffers. Let B i (x) an W i (x) be the number of 1 s an s at the i s noe of vector x respectively. P The line 4 is replace by: if l ;1 i= (min(b i(x) B l;1 i (q)) + min (W i(x) W i(q)) ) t. Fact B i (x) = l;1 ;W i (x) where l is the level of i s noe. Lemma If min(w i (x) W i (y)) = W i (x), then min(b i (x) B i (y)) = B i (y) an vice versa. Proof: B i (x) = ; W i (x) an B i (y) = ; W l;1 i (y). l;1 Now if B i (x) B i (y),then ; W i (x) ; W l;1 i (y). l;1 By rearranging the formula, we get W i (x) W i (y). Lemma 3 S[x y] min(b 1(x) B 1(y)) + min(w 1(x) W 1 (y)). Proof: When only frequency values are available, the maximum value S 11 (x y) is when 1 s in B 1 (x) an B 1 (y) are aligne together. S 11 (x y) cannot excee min(b 1 (x) B 1 (y)). ThesameforS (x y). The maximum similarity value that two vectors x an y can have at the root level of ABT is min(b 1 (x) B 1 (y)) + min (W 1(x) W 1(y)). Suppose =. In the examples in Fig. 1, f 1 (q r 1 ) = min (7 3) min(1 5) + which is :5. Similarly, f 1 (q r ) = 3:5 f 1 (q r 3 ) =3:5. These values are always larger than or equal to S[x y] s efine in GSC classifier. When there is an exact match meaning that x = y, S[x y] =B 1 (x) + W 1(x). Therefore, the maximum similarity value varies from vector to vector by their frequencies. Let f l be P a function for selecting caniates at the level, l: f l (x q) = l ;1 i= (min(b i(x) B l;1 i (q)) + min(w i(x) W i(q)) ). Let L l be a set of templates, x s chosen by the function f l (x q) t. Corollary 1 All members in M are guarantee to be in all caniate sets. M = L log L l L l;1 L 1 R Proof: Let B(x) an W (x) be the frequency of 1 s an s of one internal noe. Let B l (x) W l (x) an B r (x) W r (x) be those of the left an right chilren of the noe. By the efinition, B(x) = B l (x) + B r (x) an W (x) = W l (x) + W r (x). Consier two vectors x an y. Suppose B(x) = min(b(x) B(y)), then the maximum possible value at the internal noe can have becomes B(x) + W (y) = B l (x) + W l(y) + B r (x) + W r(y). Now at its chilren s noes, there are three possible cases an one impossible case. First, if B l (x) = min(b l (x) B l (y)) an B r (x) = min(b r (x) B r (y)), then the maximum value at the both chilren noes can have is the same as that of their parent s noe by efinition : B l (x)+b r (x) B l (y)+b r (y) an W l (x) +W r (x) W l (y) +W r (y). Next, if B l (x) = min(b l (x) B l (y)) but B r (y) =min(b r (x) B r (y)), thenit is smaller than or equal to that of their parent s noe. Bl(x) + W l(y) + Br(y) + W r(x) Bl(x) + W l(y) + Br(x) + W r(y). When B l (y) = min(b l (x) B l (y)) but B r (x) = min(b r (x) B r (y)), it is also smaller than or equal to that of their parent s noe. It is impossible by efinition if B l (y) = min(b l (x) B l (y)) but B r (y) =min(b r (x) B r (y)). Therefore, the upper boun of a parent level is always greater than or equal to that of its chilren s noes. This is true for all internal noes. The upper boun at a certain level is the sum of all upper bouns at all noes in that level. Let f l be the upper boun function at a level, l, thenf L1 f Llog = f M. Thus, M = L log L 1 R: This corollary is the sine qua non which guarantees the correctness an speeup of Algorithm for GSC classifier. The following lemma further accelerates the search process, which we state it without a proof. Lemma 4 S (x y) = ; B 1 (x) ; B 1 (y) +S 11 (x y). This means that once we compute the similarity for S 11 (x y), S (x y) is compute in constant time. 4.. Experiment Before embarking on the results, it is important to iscuss the relationship between error an reject rates of a recognizer. As shown in Fig. 3, the error versus reject percentages of recognition graph is a great way to evaluate the performance of OCR systems. A goo recognizer must be as close to the axes as possible. Typically, the higher reject rate, the lower error rate. The threshol value must be mae epening on the costs of rejects an errors. As the threshol value increases, the reject rate also increases an the error rate ecreases. We use two sets of isolate mixe han-printe/cursive character images. The first set is the reference set whose 5

6 Reject Rate "GSC" Error Rate Error rate = Number of Errors Number of Accepte Queries 1 Number of Rejections Reject rate = Number of Total Queries 1 Figure 3. Error vs. Reject graph for GSC classifier Running time in Milli sec Threshol "tvsr." Figure 4. Threshol vs. running time size is 18 an approximately 8 per character (A-Z). The other set is a test set whose size is While the average running time of the brute force metho is millisecon, one using ABT is millisecon where the epth of tree is an t =. The higher threshol value the classifier has, the faster it runs. as shown in Fig. 4 The higher threshol value means the more rejects which save running time. Hence, the importance of a threshol is twofol; it not only enables to control the error an reject rates, but also gives spee up. 5. Finale Nearest Neighbor searching is an extremely well stuie area an a couple of techniques to spee up the OCR system were previously evelope. They are prototype thinning an clustering techniques which cause a little egraation in performance. Because even a slight egraation in performance is often too costly in real OCR applications, we stuie it an a new fast nearest neighbor search algorithm with no egraation is propose. Filtration with a threshol is the key iea for expeition. To perform this, we introuce ABT structure. The aitive information of pattern feature vectors ensures that the query processing time reuces significantly. Furthermore, the iea is effective even in combination with other techniques like prototype thinning or clustering. There are two parameters that can affect the spee of search : i) epth of ABT an ii) the number of branches. As state in Observation 3, the epth of ABT influences the spee significantly. We introuce the aitive binary tree, however, the tree can be an arbitrary branch tree. Aitive N-ary Tree might perform better than the binary tree. Moreover, the iea of filtration using ABT can exten to other efinitions with a little embellishment. Consier the normalize inner prouct efine in equation. The maximum value for this efinition is 1 when there is an exact match. We exclue the exceptional case where all features are as it P makes the enominatorbe. A filtration occurs in case that l ;1 min (B i(x) B i(q)) i= t Again, consier the l;1 kxkkqk example in Fig. 1 with a threshol = :5. At the root level filtration, r 1 is filtrate because the upper boun is :447. References [1] A. J. Broer. Strategies for efficient incremental nearest neighbor search. Pattern Recognition, 3(1): , 199. [] B. V. Dasarathy. Visiting nearest neighbors- a survery of nearest neighbor pattern classification techniques. In Proceeings of the International Conference on Cybernetics an Society, p IEEE, [3] B. V. Dasarathy. Nearest Neighbor Pattern Classification Techniques. IEEE Computer Society Press, [4] R.O.Dua,D.G.Stork,anP.E.Hart. Pattern Classification an Scene Analysis. John Wiley & Sons, Inc., n eition,. [5] J. T. Favata, G. Srikantan, an S. N. Srihari. Hanprinte character/igit recognition using a multiple feature/resolution philosophy. In IWFHR-IV, p 57 66, [6] J. Frieman, J. Bentley, an R. Finkel. An algorithm for fining best matches in logarithmic expecte time. ACM Tran. on Mathematical Software, 3:9 6, [7] K. Fukunaga an P. Narenra. A branch an boun algorithm for computing k-nearest neighbors. IEEE Tran. on Computers, 4:75 743, [8] G.W. Gates. The reuce nearest neighbor rule. IEEE Tran. on Information Theory, IT-18(3): , 197. [9] P.E. Hart. The conense nearest neighbor rule. IEEE Tran. on Information Theory, IT-14(3): , 197. [1] T. Hong, S. W. Lam, J. J. Hull, an S. N. Srihari. The esign of a nearest-neighbor classifier an its use for japanese character recognition. In Proceeings of IC- DAR, p IEEE, [11] B. S. Kim an S. B. Park. A fast k-nearest neighbor fining algorithm base on the orere partition. IEEE Tran. on PAMI, 8(6): , [1] H. Niemann an R. Goppert. An efficient branch-anboun nearest neighbour classifier. Pattern Recognition Letters, 7:67 7, [13] C. Papaimitriou an J. Bentley. A worst-case analysis of nearest neighbor searching by projection. In In Automata, Languages an Programming LNCS,v.85,p Springer-Verlag,