Proximal Manifold Learning via Descriptive Neighbourhood Selection

Transcription

1 Applied Mathematical Sciences, Vol. 8, 2014, no. 71, HIKARI Ltd, Proximal Manifold Learning via Descriptive Neighbourhood Selection James F. Peters a,b,1 and Randima Hettiarachchi a a Computational Intelligence Laboratory Department of Electrical & Computer Engineering University of Manitoba, WPG, MB R3T 5V6, Canada 1 Corresponding author b Department of Mathematics, Faculty of Arts and Sciences Adıyaman University, Adıyaman, Turkey Copyright c 2014 James F. Peters and Randima Hettiarachchi. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract This article introduces proximal manifold learning via descriptive neighbourhood selection, which imposes a descriptive form of isometric mapping on high dimensional proximal data spaces into lower dimensional proximal manifolds in descriptive Efremovič(EF) proximity spaces. Proximal data spaces are defined by n-dimensional feature vectors in the Euclidean space R n. This leads to results concerning descriptively near manifolds in pictures (digital images), which are useful in discerning picture patterns. Mathematics Subject Classification: 57M50, 54E05, 54E40, 00A69 Keywords: Neighbourhood, Isometry, Manifold, EF-Proximity 1 Introduction This article introduces descriptive neighbourhood selection in manifold learning, which supports the descriptive form of isometry in a mapping on a high

2 3514 J.F. Peters and R. Hettiarachchi dimensional data space into a low dimensional manifold in a descriptive Efremovič(EF) proximity space [3]. Descriptive isometry was introduced in [4]. M. Spivak [8, 1, p. 1] observes that locally a manifold is the metric space R n. Two well known manifold learning algorithms are ISOMAP (isometric feature mapping) [9] and LLE (locally linear embedding) [6, 5]. The first phase in most of the manifold learning algorithms is the neighborhood search. In conventional approaches, a neighbourhood is determined based on the spatial distance between data points. In our case, we consider the descriptive neighbourhood of a point [4] in the neighbourhood selection phase of manifold learning algorithm. Selection of descriptive neighbourhoods of points ensures that descriptively near points in high dimensional data space are mapped to descriptively near points in the low dimensional manifold. In other words, the descriptive isometry is preserved. 2 Preliminaries Let X be a nonempty set of non-abstract points and let Φ = {φ 1,...,φ n } be a set of probe functions that represent features of each x X. In a discrete space, a non-abstract point has a location and features that can be measured[2, 3]. This leads to a proximal view of sets of picture points in digital images. A probe function Φ : X R represents a feature of a sample point in a picture. Let Φ(x) = (φ 1 (x),...,φ n (x)) denote a feature vector for x, which provides a description of each x X. To obtain a descriptive proximity relation (denoted by δ Φ ) that satisfies the descriptive EF-axioms [4, 4.15], an extension of the spatial EF-axioms [7], one first chooses a set of probe functions. Let A,B 2 X and let Q(A), Q(B) denote sets of descriptions of points in A, B, respectively. The expression A δ Φ B reads A is descriptively near B. Similarly, A δ Φ B reads A is descriptively far from B. The descriptive proximity of A and B is defined by A δ Φ B Q(A) Q(B). A descriptive isometry is defined in the context of a descriptive proximity space [3]. Briefly, a set X endowed with a descriptive proximity relation δ Φ is called a descriptive proximity space. A manifold [8] is a metric space M with the following property: If x M, then there is some neighbourhood U of x and some integer n 0 such that U is homeomorphic to R n. 3 Main Results Let X = [x 1,x 2,...,x N ] T be a set of points sampled from the high dimensional data space. The description of each point x i is given by a D-dimensional feature vector Φ(x i ) = (φ 1 (x),φ 2 (x),...,φ D (x)). Let M be the d-dimensional manifold extracted from the D-dimensional dataset X such that d D.

3 Proximal manifold learning 3515 In contrast to spatial neighbourhood selection in conventional manifold learning algorithms, we introduce the de- r x i x scriptive neighbourhood of a point. A i descriptive neighbourhood of point x i Spatial Neighbourhood Descriptive Neighbourhood (brieflydenotedbyn Φ(xi ))isasetofpicture points each with a description that matches the description of x Figure 1: BdNbd of a Point i. Different types of descriptive neighbourhoods of points are introduced in [4, 1, p. 29]. In this article, we focus on the bounded descriptive neighbourhood of a point. Definition 3.1. Bounded Descriptive Neighbourhood of a Point [4]. N Φ(x),ε,r denotes a bounded descriptive neighbourhood (BdNbd) of a point x X with upper bound ε > 0 on feature value differences and fixed radius r in a pseudometric descriptive proximity space, defined by: N Φ(x),ε,r = {y X : d(φ(x),φ(y)) < ε and d(x,y) < r} In conventional manifold learning techniques such as LLE, the mapping f on X to M is a local isometry (see Def. 3.2 [1]). Definition 3.2. Local Isometry The mapping f : X M is locally isometric such that, for any ε > 0 and x in the domain of f, let N ε (x) = {y : y x 2 < ε} denote an ε-neighbourhood of x using Euclidean distance. We have f(x) f(x 0 ) 2 = x x 0 2 +o( x x 0 ) for any x 0 X and x N ε (x 0 ) and o( x x 0 ) is the error due to mapping f. By replacing N ε (x) in definition 3.2 with bounded descriptive neighbourhoodn Φ(x),ε,r, thedescriptiveformofisometricmappingonahighdimensional data space into a low dimensional manifold can be derived. Definition 3.3. Descriptive Isometry Let (X,δ Φ ), (Y,δ Φ ) be pseudometric descriptive EF-proximity spaces and let Q(A) Q(X) R n, Q(B) Q(Y) R n. A mapping f : Q(A) Q(B) is a descriptive isometry, provided f (Φ(x)) = f (Φ(y)) when Φ(x) = Φ(y), for Φ(x),Φ(y) Q(A) and f(φ(x)),f(φ(y)) Q(B), i.e.,. d(f(φ(x)),f(φ(y))) = d(φ(x),φ(y)). Lemma 3.4. Let X, M be pseudometric descriptive proximity spaces and let f be a descriptive isometry on Q(X) into Q(M). The descriptive neighbourhood of a point x X maps to a descriptive neighbourhood of the point f(x) M.

4 3516 J.F. Peters and R. Hettiarachchi Proof. Let N Φ(x) be descriptive neighbourhood of point x X and let f map Φ(x) to f(φ(x)) in manifold Q(M) with mapping f. From definition 3.1, each point y N Φ(x) is subjected to d(φ(x),φ(y)) < ε. From definition 3.3, d(f(φ(x)),f(φ(y))) = d(φ(x),φ(y)). Thus, each point y N Φ(x) X maps to f(y) N Φ(f(x)) M. Theorem 3.5. Let digital images X, M be pseudometric descriptive proximity spaces, and let f be a descriptive isometry on Q(X) into Q(M), Φ(x) Q(X),Φ(m) Q(M). Then each point y N Φ(x) maps to f(y) N Φ(f(m)). Proof. Immediate from Lemma 3.4. xi f(xi) Fossil Image 2-D Manifold Figure 2: Mapping Descriptive Neighbourhood of Fossil Image Example 3.6. Let X be the set of points sampled from the edge map of the fossil in Fig. 2. Each point of the BdNbd of Φ(x i ) is mapped to a point in the BdNbd of Φ(f(x i )) on a 2-dimensional manifold. Each x i X is described with a feature vector Φ = {R,G,B,Gdir}, where R, G and B refer to red, green and blue colour channel values and Gdir is the gradient orientation of the point, respectively. The 8 descriptively nearest neighbours were considered in obtaining the 2-dimensional manifold using LLE. Descriptive isometry obtained through selection of descriptive neighbourhoods M1 Fossil 1 leads to descriptively near manifolds, which is beneficial in image pattern M2 recognition. Example 3.7 explains the Fossil 2 application of descriptively near manifolds. 2-D Manifolds Figure 3: Descriptively Near Manifolds Example 3.7. We consider 2 different images of the same class of fossil in this example. A portion of each fossil is extracted for comparison. Since the fossils belong to the same class, they can be considered to be descriptively near each other. The same feature vector Φ = {R,G,B,Gdir} is used to describe points in both fossil images. Fig. 3 illustrates the results of manifold learning using 8 descriptively nearest neighbours in LLE algorithm. Fossil 1 is mapped to manifold M1 (in green) and Fossil 2 is mapped to manifold M2 (in red). By observing M1 and M2, it is evident that descriptively near fossils result in descriptively near manifolds, i.e., Q(M1) Q(M2).

5 Proximal manifold learning 3517 Acknowledgements. This research has been supported by the The Scientific and Technological Research Council of Turkey (TÜBİTAK) Scientific Human Resources Development (BIDEB) under grant no: B and Natural Sciences & Engineering Research Council of Canada (NSERC) discovery grant References [1] X. Huo and A.K.Smith, Performance analysis of a manifold learning algorithm in dimension reduction, Technical Paper, Statistics in Georgia Tech, Georgia Institute of Technology (2006). [2] M.M. Kovár, A new causal topology and why the universe is co-compact, arxive: [math-ph] (2011), [3] J.F. Peters, Near sets: An introduction, Math. in Comp. Sci. 7 (2013), no. 1, 3 9, DOI /s [4], Topology of Digital Images. Visual Pattern Discovery in Proximity Spaces, Intelligent Systems Reference Library, vol. 63, Springer, 2014, ISBN , pp [5] S.T. Roweis and K.L. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science 290 (2000), no. 5500, [6] K.L. Saul and S.T. Roweis, Think globally, fit locally: unsupervised learning of low dimensional manifolds, The Journal of Machine Learning Research 4 (2003), [7] Ju. M. Smirnov, On proximity spaces, Math. Sb. (N.S.) 31 (1952), no. 73, , English translation: Amer. Math. Soc. Trans. Ser. 2, 38, 1964, [8] M. Spivak, A comprehensive introduction to differential geometry, volume i, 3 ed., vol. 1, Publish or perish, Berkeley, [9] J.B. Tenenbaum, V. De Silva, and J.C. Langford, A global geometric framework for nonlinear dimensionality reduction, Science 290 (2000), no. 5500, Received: February 16, 2014