2580 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 10, OCTOBER 2010

Transcription

1 2580 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 10, OCTOBER 2010 Joint Manifolds for Data Fusion Mark A. Davenport, Member, IEEE, Chinmay Hegde, Student Member, IEEE, Marco F. Duarte, Member, IEEE, Richard G. Baraniuk, Fellow, IEEE Abstract The emergence of low-cost sensing architectures for diverse modalities has made it possible to deploy sensor networks that capture a single event a large number of vantage points using multiple modalities. In many scenarios, se networks acquire large amounts of very high-dimensional data. For example, even a relatively small network of cameras can generate massive amounts of high-dimensional image video data. One way to cope with this data deluge is to exploit low-dimensional data models. Manifold models provide a particularly powerful oretical algorithmic framework for capturing structure of data governed by a small number of parameters, as is often case in a sensor network. However, se models do not typically take into account dependencies among multiple sensors. We thus propose a new joint manifold framework for data ensembles that exploits such dependencies. We show that joint manifold structure can lead to improved performance for a variety of signal processing algorithms for applications including classification manifold learning. Additionally, recent results concerning rom projections of manifolds enable us to formulate a scalable universal dimensionality reduction scheme that efficiently fuses data all sensors. Index Terms Camera networks, classification, data fusion, manifold learning, rom projections, sensor networks. I. INTRODUCTION T HE emergence of low-cost sensing devices has made it possible to deploy sensor networks that capture a single event a large number of vantage points using multiple modalities. This can lead to a veritable data deluge, fueling need for efficient algorithms for processing efficient protocols for transmitting data generated by such networks. In order to address se challenges, re is a clear need for a oretical framework for modeling complex interdependencies among signals acquired by se networks. This framework should support development of efficient algorithms that can exploit this structure efficient protocols that can cope with massive data volume. Manuscript received September 30, 2009; revised February 28, 2010; accepted May 18, Date of publication June 14, 2010; date of current version September 17, This work was supported in part by grants NSF CCF CCF , DARPA/ONR N , ONR N N , AFOSR FA , ARO MURI W911NF , ARO MURI W911NF , in part by TI Leadership Program. The associate editor coordinating review of this manuscript approving it for publication was Prof. Rama Chellapa. M. A. Davenport is with Department of Statistics, Stanford University, Stanford, CA USA. C. Hegde R. G. Baraniuk are with Department of Electrical Computer Engineering, Rice University, Houston, TX USA. M. F. Duarte is with Program in Applied Computational Mamatics, Princeton University, Princeton, NJ USA. Color versions of one or more of figures in this paper are available online at Digital Object Identifier /TIP Consider, for example, a camera network consisting of video acquisition devices each acquiring -pixel images of a scene simultaneously. Ideally, all cameras would send ir raw recorded images to a central processing unit, which could n holistically analyze all data produced by network. This naïve approach would in general provide best performance, since it exploits complete access to all of data. However, amount of raw data generated by a camera network, on order of, becomes untenably large even for fairly small networks operating at moderate resolutions frame rates. In such settings, amount of data can ( often does) overwhelm network resources such as power communication bwidth. While naïve approach could easily be improved by requiring each camera to first compress images using a compression algorithm such as JPEG or MPEG, this modification still fails to exploit any interdependencies between cameras. Hence, total power bwidth requirements of network will still grow linearly with. Alternatively, exploiting fact that in many cases end goal is to solve some kind of inference problem, each camera could independently reach a decision or extract some relevant features, n relay result to central processing unit which would n combine results to provide solution. Unfortunately, this approach also has disadvantages. The cameras must be smart in that y must possess some degree of sophistication so that y can execute nonlinear inference tasks. Such technology is expensive can place severe dems on available power resources. Perhaps more importantly, total power bwidth requirement will still scale linearly with. In order to cope with such high-dimensional data, a common strategy is to develop appropriate models for acquired images. A powerful model is geometric notion of a low-dimensional manifold. Informally, manifold models arise in cases where 1) a -dimensional parameter can be identified that carries relevant information about a signal 2) signal changes as a continuous (typically nonlinear) function of se parameters. Typical examples include a 1-D signal translated by an unknown time delay (parameterized by translation variable), a recording of a speech signal (parameterized by underlying phonemes spoken by speaker), an image of a 3-D object at an unknown location captured an unknown viewing angle (parameterized by three spatial coordinates of object as well as its roll, pitch, yaw). In se many or cases, geometry of signal class forms a nonlinear -dimensional manifold in where is -dimensional parameter space. In recent years, researchers in image processing have become increasingly interested in manifold models due to observation (1) /$ IEEE

2 DAVENPORT et al.: JOINT MANIFOLDS FOR DATA FUSION 2581 that a collection of images obtained different target locations/poses/illuminations sensor viewpoints form such a manifold [1] [3]. As a result, manifold-based methods for image processing have attracted considerable attention, particularly in machine learning community, can be applied to such diverse applications as data visualization, classification, estimation, detection, control, clustering, learning [3] [5]. Low-dimensional manifolds have also been proposed as approximate models for a number of nonparametric signal classes such as images of human faces hwritten digits [6] [8]. In sensor networks, multiple observations of same event are often acquired simultaneously, resulting in acquisition of interdependent signals that share a common parameterization. Specifically, a camera network might observe a single event a variety of vantage points, where underlying event is described by a set of common global parameters (such as location orientation of an object of interest). Similarly, when sensing a single phenomenon using multiple modalities, such as video audio, underlying phenomenon may again be described by a single parameterization that spans all modalities (such as when analyzing a video audio recording of a person speaking, where both are parameterized by phonemes being spoken). In both examples, all of acquired signals are functions of same set of parameters, i.e., we can write each signal as where is same for all. Our contention in this paper is that we can obtain a simple model that captures correlation between sensor observations by matching parameter values for different manifolds observed by sensors. More precisely, we observe that by simply concatenating points that are indexed by same parameter value different component manifolds, i.e., by forming, we obtain a new manifold, which we dub joint manifold, that encompasses all of component manifolds shares same parameterization. This structure captures interdependencies between signals in a straightforward manner. We can n apply same manifold-based processing techniques that have been proposed for individual manifolds to entire ensemble of component manifolds. In this paper we conduct a careful examination of topological geometrical properties of joint manifolds; in particular, we compare joint manifolds to ir component manifolds to see how properties like geodesic distances, curvature, branch separation, condition number are affected. We n observe that se properties lead to improved performance noise-tolerance for a variety of signal processing algorithms when y exploit joint manifold structure. As a key advantage of our proposed model, we illustrate how joint manifold structure can be exploited via a simple efficient data fusion algorithm based upon rom projections. For case of cameras jointly acquiring -pixel images of a common scene characterized by parameters, we demonstrate that total power communication bwidth required by our scheme is linear in mainfold dimension only logarithmic in number of cameras camera resolution. Recent developments in field of compressive sensing has made this data acquisition model practical in many interesting applications [9] [11]. Related prior work has studied manifold alignment, where goal is to discover maps between datasets that are governed by same underlying low-dimensional structure. Lafon et al. proposed an algorithm to obtain a one-to-one matching between data points several manifold-modeled classes [12]. The algorithm first applies dimensionality reduction using diffusion maps to obtain data representations that encode intrinsic geometry ofclass. Then, anaffinefunctionthatmatchesasetof lmark points is computed applied to remainder of datasets. This concept was extended by Wang Mahadevan, who applied Procrustes analysis on dimensionality-reduced datasets to obtain an alignment function between a pair of manifolds [13]. Since an alignment function is provided instead of a data point matching, mapping obtained is applicable for entire manifold rar than for set of sampled points. In our setting, we assume that eir 1) manifold alignment is implicitly present, forexample, via synchronization between differentsensors, or 2) manifolds have been aligned using one of se approaches. Ourmainfocusisananalysisofbenefitsprovidedbyanalyzing joint manifold versus solving task of interest separately on each of manifolds. For concreteness, but without loss of generality, we couch our analysis in language of camera networks, although much of our ory is sufficiently generic so as to apply to a variety of or scenarios. This paper is organized as follows. Section II introduces establishes some basic properties of joint manifolds. Section III provides discussion of practical examples of joint manifolds in camera network setting describes how to use rom projections to exploit joint manifold structure in such a setting. Sections IV V n consider application of joint manifolds to tasks of classification manifold learning, providing both a oretical analysis as well as extensive simulations. Section VI concludes with a brief discussion. II. JOINT MANIFOLDS: THEORY In this section, we develop a oretical framework for ensembles of manifolds that are jointly parameterized by a small number of common degrees of freedom. Informally, we propose a data structure for jointly modeling such ensembles; this is obtained simply by concatenating points different ensembles that are indexed by same articulation parameter to obtain a single point in a higher-dimensional space. We begin by defining joint manifold for setting of general topological manifolds. 1 In order to simplify our notation, we will let denote product manifold. Furrmore, we will use notation to denote a -tuple of points, or concatenation of points, which lies in Cartesian product of sets (e.g., ). Definition 1: Let be an ensemble of topological manifolds of equal dimension. Suppose that manifolds are homeomorphic to each or, in which case re exists a homeomorphism between for each. For a particular set of, we define joint manifold as 1 We refer reader to [14] for a comprehensive introduction to manifolds.

3 2582 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 10, OCTOBER 2010 Fig. 1. Pair of isomorphic manifolds M M, resulting joint manifold M. Furrmore, we say that are corresponding component manifolds. Note that serves as a common parameter space for all component manifolds. Since component manifolds are homeomorphic, this choice is ultimately arbitrary. In practice it may be more natural to think of each component manifold as being homeomorphic to some fixed -dimensional parameter space. However, in this case one could still define as is shown previously by defining as composition of homeomorphic mappings to to. As an example, consider 1-D manifolds in Fig. 1. Fig. 1(a) (b) shows two isomorphic manifolds, where is an open interval, where, i.e., is a circle with one point removed (so that it remains isomorphic to a line segment). In this case joint manifold, illustrated in Fig. 1(c), is a helix. Notice that re exist or possible homeomorphic mappings to, that precise structure of joint manifold as a submanifold of is heavily dependent upon choice of this mapping. Returning to definition of, observe that although we have called joint manifold, we have not shown that it actually forms a topological manifold. To prove that is indeed a manifold, we will make use of fact that joint manifold is a subset of product manifold. One can show that forms a -dimensional manifold using product topology [14]. By comparison, we now show that has dimension only. Proposition 1: is a -dimensional submanifold of. Proof: We first observe that since, we automatically have that is a second countable Hausdorff topological space. Thus, all that remains is to show that is locally homeomorphic to. Suppose. Since, wehave a pair such that is an open set containing is a homeomorphism where is an open set in. We now define for. Note that for each, is an open set is a homeomorphism (since is a homeomorphism). Now set define. Observe that is an open set that. Furrmore, for any in, we have that. Then for each. Thus, since image of each in under ir corresponding is same, we can form a single homeomorphism by assigning. This shows that is locally homeomorphic to as desired. Since is a submanifold of, it also inherits some desirable properties. Proposition 2: Suppose that are isomorphic topological manifolds is as previously defined. 1) If are Riemannian, n is Riemannian. 2) If are compact, n is compact. Proof: The proofs of se facts are straightforward follow fact that if component manifolds are Riemannian or compact, n will be as well. n inherits se properties as a submanifold of [14]. Up to this point we have considered general topological manifolds. In particular, we have not assumed that component manifolds are embedded in any particular space. If each component manifold is embedded in, joint manifold is naturally embedded in where. Hence, joint manifold can be viewed as a model for sets of data with varying ambient dimension linked by a common parametrization. In sequel, we assume that each manifold is embedded in, which implies that. Observe that while intrinsic dimension of joint manifold remains constant at, ambient dimension increases by a factor of. We now examine how a number of geometric properties of joint manifold compare to those of component manifolds. We begin by observing that Euclidean distances 2 between points on joint manifold are larger than distances on component manifolds. The result follows directly definition of Euclidean norm, so we omit proof. Proposition 3: Let be given. Then While Euclidean distances are important (especially when noise is introduced), natural measure of distance between a pair of points on a Riemannian manifold is not Euclidean distance, but rar geodesic distance. The geodesic distance between points is defined as where is a -smooth curve joining, is length of as measured by In order to see how geodesic distances on compare to geodesic distances on component manifolds, we will make use of following lemma. Lemma 1: Suppose that are Riemannian manifolds, let be a -smooth curve on joint manifold. Denote by restriction of to ambient 2 In remainder of this paper, whenever we use notation k1kwe mean k1k, i.e., ` (Euclidean) norm on. When we wish to differentiate this or ` norms, we will be explicit. (2) (3)

4 DAVENPORT et al.: JOINT MANIFOLDS FOR DATA FUSION 2583 dimensions corresponding to. Then each is a -smooth curve on, Proof: If is a geodesic path between, n Lemma 1 Proof: We begin by observing that (4) For a fixed, let, observe that is a vector in. Thus we may apply stard norm inequalities By definition. Hence, this establishes (7). Now observe that lower bound in Lemma 1 is derived lower inequality of (5). This inequality is attained with equality if only if each term in sum is equal, i.e., for all. This is precisely case when are isometries. Thus we obtain to obtain Combining right-h side of (6) with (4) we obtain Similarly, left-h side of (6) we obtain (5) (6) We now conclude that since if we could obtain a shorter path to this would contradict assumption that is a geodesic on, which establishes (8). Next, we study local smoothness global self avoidance properties of joint manifold. Definition 2. [15]: Let be a Riemannian submanifold of. The condition number is defined as, where is largest number satisfying following: open normal bundle about of radius is embedded in for all. The condition number controls both local smoothness properties global properties of manifold; as becomes smaller, manifold becomes smoor more self-avoiding, as observed in [15]. We will informally refer to manifolds with small as good manifolds. Lemma 2. [15]: Suppose has condition number. Let be two distinct points on, let denote a unit speed parameterization of geodesic path joining. Then We are now in a position to compare geodesic distances on to those on component manifold. Theorem 1: Suppose that are Riemannian manifolds. Let be given. Then Lemma 3. [15]: Suppose has condition number. Let be two points on such that.if, n geodesic distance is bounded by (7) If mappings are isometries, i.e., for any for any pair of points, n (8) We wish to show that if component manifolds are smooth self avoiding, joint manifold is as well. It is not easy to prove this in most general case, where only assumption is that re exists a homeomorphism (i.e., a continuous bijective map ) between every pair of manifolds. However, suppose manifolds are diffeomorphic, i.e., re exists a continuous bijective map between tangent spaces at corresponding points on every pair of manifolds. In that case, we make following assertion. Theorem 2: Suppose that are Riemannian submanifolds of, let denote condition number of. Suppose also that that define corresponding

5 2584 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 10, OCTOBER 2010 joint manifold are diffeomorphisms. If is condition number of, n Proof: Let. Since are diffeomorphisms, we may view as being diffeomorphic to ; i.e., we can build a diffeomorphic map to as We also know that given any two manifolds linked by a diffeomorphism, each vector in tangent space of manifold at point is uniquely mapped to a tangent vector in tangent space of manifold at point through map, where denotes Jacobian operator. Consider application of this property to diffeomorphic manifolds. In this case, tangent vector to manifold can be uniquely identified with a tangent vector to manifold. This mapping is expressed as since Jacobian operates componentwise. Therefore, tangent vector can be written as In or words, a tangent vector to joint manifold can be decomposed into component vectors, each of which are tangent to corresponding component manifolds. Using this fact, we now show that a vector that is normal to can also be broken down into subvectors that are normal to component manifolds. Consider, denote as normal space at. Suppose. Decompose each as a projection onto component tangent normal spaces, i.e., for such that for each. Then, since is tangent to joint manifold,wehave, thus. But,. Hence, i.e., each is normal to. Armed with this last fact, our goal now is to show that if n normal bundle of radius is embedded in, or equivalently, for any, that provided that. Indeed, suppose. Since for all, we have that. Since we have proved that are vectors in normal bundle of ir magnitudes are less than, n by definition of condition number. Thus result follows. This result states that in general, most we can say is that condition number of joint manifold is guaranteed to Fig. 2. Point at which normal bundle for helix manifold Fig. 1(c) intersects itself. Note that helix has been slightly rotated for clarity. be less than that of worst manifold. However, in practice this is not likely to happen. As an example, Fig. 2 illustrates point at which normal bundle intersects itself for case of joint manifold Fig. 1(c). In this case we obtain. Note that condition numbers for manifolds generating are given by. Thus, while condition number of joint manifold is not as low as best manifold, it is notably smaller than that of worst manifold. In general, even this example may be somewhat pessimistic it is possible that joint manifold may be better conditioned than even best manifold. III. JOINT MANIFOLDS: PRACTICE As noted in Introduction, a growing number of algorithms exploit manifold models for tasks such as pattern classification, estimation, detection, control, clustering, learning [3] [5]. The performance of se algorithms often depends upon geometric properties of manifold model, such as its condition number or geodesic distances along its surface. The ory developed in Section II suggests that joint manifold preserves or improves such properties. In Sections IV V, we consider two illustrative applications observe that when noise is added to underlying signals, it can be extremely beneficial to use algorithms specifically designed to exploit joint manifold structure. However, before we address se particular applications, we must first address some key practical concerns. A. Acceptable Deviations From Theory While manifolds are a natural way to model structure of a set of images governed by a small number of parameters, results in Section II make a number of assumptions concerning structure of component manifolds. In most general case, we assume that component manifolds are homeomorphic to each or. This means that between any pair of component manifolds re should exist a bijective mapping such that both are continuous. Such an assumption assures that joint manifold is indeed a topological manifold. Unfortunately, this excludes some scenarios that can occur in practice. For example this assumption might not hold in a camera network featuring nonoverlapping fields of view. In such camera networks, re are cases in which only some cameras are sensitive to small changes in parameter values. Strictly speaking, our ory may not apply in se cases, since joint manifold as we have defined it is not necessarily even a topological manifold. As a result, one might expect to see significant performance degradation when exploiting techniques that heavily rely on this joint manifold structure. We provide additional discussion of this issue in Section V-B, but for now we simply note

6 DAVENPORT et al.: JOINT MANIFOLDS FOR DATA FUSION 2585 that in Sections IV V we conduct extensive experiments using both syntic real-world datasets observe that, in practice, joint manifold-based processing techniques still exhibit significantly better performance than techniques that operate on each component manifold separately. While nonoverlapping fields of view do pose a challenge (especially in context of manifold learning), fact that this results in nonhomeomorphic manifolds seems to be more of a technical violation of our ory than a practical one. In context of manifold learning, we must actually assume that component manifolds are isometric to each or. This is certainly not case in a camera network with nonoverlapping fields of view. Even with restriction of a common field of view, this may seem an undue burden. In fact, this requirement is fulfilled by manifolds that are isometric to parameter space that governs m a class of manifolds that has been studied in [2]. Many examples this class correspond to simple image articulations that occur in vision applications, including: articulations of radially symmetric images, which are parameterized by a 2-D offset; articulations of four-fold symmetric images with smooth boundaries, such as discs, balls, etc.; pivoting of images containing smooth boundaries, which are parameterized by pivoting angle; articulations of discs over distinct nonoverlapping regions, with, producing a -dimensional manifold. These examples can be extended to objects with piecewise smooth boundaries as well as to video sequences of such articulations. In Section V, we describe heuristics for dealing with problem of nonoverlapping fields of view provide a number of experiments that suggest that se heuristics can overcome violations of isometry assumption in practice. In our oretical results concerning condition number, we assume that component manifolds are smooth, but manifolds induced by motion of an object where re are sharp edges or occlusions are nowhere differentiable. This problem can be addressed by applying a smoothing kernel to each captured image, inducing a smooth manifold [3]. More generally, we note that if cameras have moderate computational capabilities, n it may be possible to perform simple preprocessing tasks such as segmentation, background subtraction, illumination compensation that will make manifold assumption more rigorously supported in practice. This may be necessary in scenarios such as those involving multiple objects or challenging imaging conditions. B. Efficient Data Fusion via Joint Manifolds Using Linear Projections Observe that when number ambient dimension of manifolds become large, ambient dimension of joint manifold may be so large that it becomes impossible to perform any meaningful computations. Furrmore, it might appear that in order to exploit joint manifold structure, we must collect all data at a central location, which we earlier claimed was potentially impossible. In order to address this problem, we must exploit joint manifold structure to develop a more efficient fusion scheme. Specifically, given a network of cameras, let, denote image acquired by camera, which is assumed to belong in a manifold, let denote corresponding point in joint manifold. Rar than forming vector, one could potentially estimate a -dimensional parameter vector via nonlinear mapping of corresponding to manifold. By collecting at a central location, we would obtain a data representation of dimension. For example, in [16] this technique is implemented using a Laplace Beltrami embedding to extract parameter values for each camera. However, by simply concatenating each, this approach essentially ignores joint manifold structure present in data, which is evident due to fact that in an ideal setting same parameters will be obtained each of cameras. Moreover, given noisy estimates for, it is not obvious how to most effectively integrate to obtain a single joint -dimensional representation. Finally, while this approach eliminates dependence upon, it still suffers a linear dependence upon. To address this challenge, we observe that if we had access to vector, n we could exploit joint manifold structure to map it to a parameter vector of length only rar than. Unfortunately, this mapping will generally be nonlinear, each element of could potentially depend upon entire vector, preventing us operating individually on each. Thus, rar than directly extract features, we will instead restrict our focus to linear dimensionality reduction methods that, while acting on concatenated data, can be implemented in a distributed fashion. Specifically, we will aim to compute a dimensionally reduced representation of denoted, where is a stard linear projection operator. Since operator is linear, we can take local projections of images acquired by each camera, still calculate global projections of in a distributed fashion. Let each camera calculate, with matrices. Then, by defining matrix, global projections can be obtained by Thus, final measurement vector can be obtained by simply adding independent projections of images acquired by individual cameras. This gives rise to compressive data fusion protocol illustrated in Fig. 3. Suppose individual cameras are associated with nodes of a binary tree of size, where edges represent communication links between nodes. Let root of tree denote final destination of fused data ( central processing unit). Then fusion process can be represented by flow of data leaves to root, with a binary addition occurring at every parent node. Recalling that dimensionality of data is depth of tree is, we observe that total communication bwidth requirement is given by

7 2586 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 10, OCTOBER 2010 Fig. 3. Distributed data fusion using linear projections in a camera network., i.e., communication burden grows only logarithmically in. The main challenge in designing such a scheme is choice of a suitable matrix. Given a specific joint manifold, re may be an optimal that preserves Euclidean geodesic structures of while ensuring that is comparable to dimension of joint manifold (, hence, much less than ambient dimension ). Unfortunately, general problem of computing an optimal linear projection of a manifold remains unsolved, in our context, finding this projection matrix would also require full knowledge of objects to be classified as well as position/orientation of each camera in network. Such information would typically not be available within network. Fortunately, we can exploit recent results concerning rom projections to solve this problem without any prior knowledge of structure of network or objects to be captured. Specifically, it has been shown that essential structure of a -dimensional manifold with condition number residing in is approximately preserved under an orthogonal projection into a rom subspace of dimension [17]. This result has been leveraged in design of efficient algorithms for inference applications, such as classification using multiscale navigation [18], intrinsic dimension estimation [19], manifold learning [19]. In our context, this result implies that if joint manifold has bounded condition number as given by Theorem 2, n we can project joint data into a rom subspace of dimension that is only logarithmic in still approximately preserve manifold structure. This is formalized in following orem, which follows directly [17]. Theorem 3: Let be a compact, smooth, Riemannian joint manifold in a -dimensional space with condition number. Let denote an orthogonal linear mapping into a rom -dimensional subspace of. Let. Then, with high probability, geodesic Euclidean distances between any pair of points on are preserved up to distortion under. Thus, we obtain a faithful embedding of manifold via a representation of dimension only. This represents a massive improvement over original -dimensional representation, for large values of it can be a significant improvement over -dimensional representation obtained by performing separate (nonlinear) dimensionality reduction on each component manifold. 3 Recalling that total communication bwidth required for our compressive data fusion scheme is, we obtain that when using rom projections dependence of required bwidth on is ; this offers a significant improvement previous data fusion methods that necessarily require communication bwidth to scale linearly with number of cameras. Joint manifold fusion via linear projections integrates network transmission data fusion steps in a fashion similar to protocols discussed in romized gossiping [21] compressive wireless sensing [22]. Joint manifold fusion via rom projections, like compressive sensing [9] [11], is universal in that projections do not depend on specific structure of manifold. Thus, our sensing techniques need not be replaced for se extensions; only our underlying models (hyposes) are updated. We have not discussed complicating factors such as clutter, varying or unknown backgrounds, etc. Promising progress in [23] suggests that such difficulties can potentially be hled even after acquiring rom projections. Furrmore, recent years have also seen development of devices capable of directly acquiring arbitrary linear projections of images [11], [24]. Our fusion scheme can directly operate on measurements provided by such devices. IV. JOINT MANIFOLD CLASSIFICATION We now examine application of joint manifold-based techniques to some common inference problems. In this section, we will consider problem of binary classification when two classes corresponds to different manifolds. As an example, we will consider scenario where a camera network acquires images of an unknown vehicle, we wish to classify between two vehicle types. Since location of vehicle is unknown, each class forms a distinct low-dimensional manifold in image space. The performance of a classifier in this setting will depend partially on topological quantities of joint manifold described in Section II, which in particular provide basis for rom projection-based version of our 3 Note that if we were interested in compressing only a fixed number of images P, we could apply Johnson-Lindenstrauss lemma [20] to obtain that M = O(log P ) would be sufficient to obtain result in Theorem 3. However, value of M required in Theorem 3 is independent of number of images P, refore provides scalability to extremely large datasets.

8 DAVENPORT et al.: JOINT MANIFOLDS FOR DATA FUSION 2587 algorithms. However, most significant factor determining performance of joint manifold-based classifier is of a slightly different flavor. Specifically, probability of error is determined by distance between manifolds. Thus, we also provide additional oretical analysis of how distances between joint manifolds compare to those between component manifolds. satisfies, n we have that, hence A. Theory The problem of manifold-based classification is defined as follows: given manifolds, suppose we observe a signal where eir or is a noise vector, we wish to find a function that attempts to determine which manifold generated. This problem has been explored in context of automatic target recognition (ATR) using high-dimensional noisy data arising a single manifold [25], [26]; this body of work adopts a fully Bayesian approach in deriving fundamental bounds on performance of certain types of classification methods. A full analysis of our proposed framework for joint manifolds using this approach is beyond scope of this paper. Instead, we consider a classification algorithm based upon generalized maximum likelihood framework described in [27]. The approach is to classify by computing distance observed signal to each manifold, n classify based upon which of se distances is smallest; i.e., our classifier is where. We will measure performance of this algorithm by considering probability of misclassifying a point as belonging to, which we denote. To analyze this problem, we employ three common notions of separation in metric spaces: The minimum separation distance between two manifolds, defined as (9) Thus we are guaranteed that. Therefore, classifier defined by (9) satisfies. We can refine this result in two possible ways. A first possible refinement is to note that amount of noise that we can tolerate without making an error depends upon. Specifically, for a given, provided that we still have that. Thus, for a given we can tolerate noise bounded by. A second possible refinement is to ignore this dependence on while extending our noise model to case where with nonzero probability. We can still bound, since (10) Thus, we now provide lower bounds on se separation distances. The corresponding upper bounds are available in [28]. In interest of space, proof of this subsequent orems are omitted can be found in [28]. Theorem 4: Consider joint manifolds. Then, following bounds hold: (11) (12) The maximum separation distance between manifolds, defined as The Hausdorff distance to, defined as (13) As an example, if we consider case where separation distances are constant for all, n joint minimum separation distance satisfies, using upper bound for [28], we obtain Note that while, in general. As one might expect, is controlled by separation distances. For example, suppose that ; if noise vector is bounded In case where, we observe that can be considerably larger than. This means that we can potentially tolerate much more noise while ensuring. To see this, let denote a noise vector recall that we require to

9 2588 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 10, OCTOBER 2010 ensure that. Thus, if we require that for all, n we have that However, if we instead only require that, n we only need, which can be a significantly less stringent requirement. The benefit of classification using joint manifold is more apparent when we extend our noise model to case where we allow with nonzero probability apply (10). To bound probability in (10), we will make use of following adaptation of Hoeffding s inequality [29]. Lemma 4: Suppose that is a rom vector that satisfies, for. Suppose also that are independent identically distributed (i.i.d.) with. Then if,we have that for any Note that this lemma holds for any distribution on noise that is bounded. For Gaussian noise it is possible to establish a very similar result (differing only in constant inside exponent) by using stard tail bounds. Using this lemma we can relax assumption on so that we only require that it is finite, instead make weaker assumption that for a particular pair of manifolds,. This assumption ensures that, so that we can combine Lemma 4 with (10) to obtain abound on.note that if this conditiondoesnot hold,n this isavery difficult classification problem since expected norm of noise is large enough to push us closer to or manifold, in which case simple classifier given by (9) makes little sense. We now illustrate how Lemma 4 can be be used to compare error bounds between classification using a joint manifold versus using a pair of component manifolds. The proof can be found in [28]. Theorem 5: Suppose that we observe where is a rom vector such that, for, that are i.i.d. with. Define If we classify observation according to (9) n (14) (15) (16) This result can be weakened slightly to obtain following corollary [28]. Corollary 1: Suppose that we observe where is a rom vector such that, for that are i.i.d. with.if (17) we classify according to (9), n (15) (16) hold with same constants as in Theorem 5. Corollary 1 shows that we can expect a classifier based upon joint manifold to outperform a classifier based th component manifold whenever squared separation distance for th component manifolds is comparable to average squared separation distance among remaining component manifolds. Thus, we can expect joint classifier to outperform most of individual classifiers, but it is still possible that some individual classifiers will do better. Of course, if one knew in advance which classifiers were best, n one would only use data best classifiers. We expect that more typical situations include case where best classifier changes over time or where separation distances are nearly equal for all component manifolds, in which case condition in (17) is true for all. B. Experiments In this section, we apply rom projection-based fusion algorithm to perform binary classification. Suppose a number of syntic cameras, each with resolution, observe motion of a truck along a straight road. 4 This forms a 1-D manifold in image space pertaining to each camera; joint manifold is also a 1-D manifold in. Suppose now that we wish to classify between two types of trucks. Example images three camera views for two classes are shown in Fig. 4. The resolution of each image is pixels. In our experiment, we convert images to grayscale sum rom projections for three camera views. The sample camera views suggest that some views make it easier to distinguish between classes than ors. For instance, head-on view of two trucks is very similar for most shift parameters, while side view is more appropriate for discerning between two classes of trucks. The probability of error, which in this case is given by, for different manifold-based classification approaches as a function of signal-to-noise ratio (SNR) is shown in Fig. 4. It is clear that joint manifold approach performs better than majority voting is comparable in performance to best camera. While one might hope to be able to do even better than best camera, Theorem 5 suggests that in general this is only possible when no camera is significantly better than average camera. Moreover, in absence of prior information regarding how well each camera truly performs, best strategy for central processor would be to fuse data all cameras. Thus, joint manifold fusion 4 Our syntic images were generated using POVRAY ( org), an open-source ray tracing software package.

10 DAVENPORT et al.: JOINT MANIFOLDS FOR DATA FUSION 2589 Fig. 4. Sample images of two different trucks multiple camera views probability of error versus SNR for individual cameras, joint manifold, majority voting. The number of pixels in each camera image N = = Joint manifold-based classification outperforms majority voting performs nearly as well as best camera. proves to be more effective than high-level fusion algorithms like majority voting. This example highlights two scenarios when our proposed approach should prove useful. First, our method acts as a simple scheme for data fusion in case when most cameras do not yield particularly reliable data ( thus decision fusion algorithms like majority voting are ineffective.) Second, due to high dimensionality of data, transmitting images could be expensive in terms of communication bwidth. Our method ensures that communication cost is reduced to be proportional only to number of degrees of freedom of signal. V. JOINT MANIFOLD LEARNING In contrast to classification scenario described previously, in which we knew manifold structure a priori, we now consider manifold learning algorithms that attempt to learn manifold structure a set of samples of a manifold. This is accomplished by constructing a (possibly nonlinear) embedding of data into a subset of, where.if dimension of manifold is known, n is typically set to. Such algorithms provide a powerful framework for navigation, visualization interpolation of high-dimensional data. For instance, manifold learning can be employed in inference of articulation parameters (e.g., 3-D pose) a set of images of a moving object. In this section, we demonstrate that in a variety of settings, joint manifold is significantly easier to learn than individual component manifolds. This improvement is due to both kind of increased robustness to noise noted in Section IV to fact that, as was shown in Theorem 2, joint manifold can be significantly better-conditioned than component manifolds, meaning that it is easier to learn structure of joint manifold a finite sampling of points. A. Theory Several algorithms for manifold learning have been proposed, each giving rise to a nonlinear map with its own special properties advantages (e.g., Isomap [30], Locally Linear Embedding (LLE) [31], Hessian Eigenmaps [32], etc.) Of se approaches, we devote special attention here to Isomap algorithm, which assumes that point cloud consists of samples a data manifold that is (at least approximately) isometric to a convex subset of Euclidean space. In this case, re exists an isometric mapping a parameter space to manifold such that for all. In essence, Isomap attempts to discover inverse mapping. Isomap works in three stages: 1) Construct a graph that contains a vertex for each data point; an edge connects two vertices if Euclidean distance between corresponding data points is below a specified threshold. 2) Weight each edge in graph by computing Euclidean distance between corresponding data points. We n estimate geodesic distance between each pair of vertices as length of shortest path between corresponding vertices in graph. 3) Embed points in using multidimensional scaling (MDS), which attempts to embed points so that ir Euclidean distance approximates estimated geodesic distances. A crucial component of MDS algorithm is a suitable linear transformation of matrix of squared geodesic distances; rank- approximation of this new matrix yields best possible -dimensional coordinate structure of input sample points in a mean-squared sense. Furr results on performance of Isomap in terms of geometric properties of underlying manifold can be found in [33]. We examine performance of Isomap for learning joint manifold as compared to learning isometric component manifolds separately. We assume that we have noiseless samples. In order to judge quality of embedding learned by Isomap, we will observe that for any pair

11 2590 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 10, OCTOBER 2010 Fig. 5. (top) Articulation manifolds sharing a common 2-D parameter space 2. Images simulate viewing a translating disc J = 3 viewing angles. (bottom) 2-D embedding of individual joint manifolds learned via Isomap. of samples a manifold whose vertices are linked within graph, we have that (18) for some that will depend upon samples of graph. Isomap will perform well if largest value of that satisfies (18) for any pair of samples that are connected by an edge in graph is close to 1. Using this fact, we can compare performance of manifold learning using Isomap on samples joint manifold to using Isomap on samples a particular component manifold. The proof of this orem can again be found in [28]. Theorem 6: Let be a joint manifold isometric component manifolds. Let suppose that we are given a graph that contains one vertex for each sample obtained. For each, define as largest value such that for all pairs of points connected by an edge in that (19). Then we have (20) From Theorem 6 we see that, in many cases, joint manifold estimates of geodesic distances will be more accurate than estimates obtained using one of component manifolds. If for a particular component manifold we observe that, n we know that joint manifold leads to better estimates. Essentially, we may expect that joint manifold will lead to estimates that are better than average case across component manifolds. We now consider case where we have a dense sampling of manifolds so that, examine case where we obtain noisy samples. We will assume that noise is i.i.d. demonstrate that any distance calculation performed on serves as a better estimator of pairwise ( consequently, geodesic) distances between any two points than that performed on any component manifold using points. Again, proof of this orem can be found in [28]. Theorem 7: Let be a joint manifold isometric component manifolds. Let assume that for all. Assume that we acquire noisy observations, where are independent noise vectors with,, for. Then where. We observe that estimate of true distance suffers a small constant bias; this can be hled using a simple debiasing step. 5 Theorem 7 indicates that probability of large deviations in estimated distance decreases exponentially in number of component manifolds ; thus we should observe significant denoising even in case where is relatively small. B. Practice The oretical results derived previously assume that acquired data arises isometric component manifolds. As noted in Section III-A, barring controlled or syntic scenarios, this is very rarely case. In practice, isometric assumption breaks down due to two reasons: 1) cameras may be at different distances scene, nonidentical cameras may possess different dynamic ranges, or cameras may be of different modalities (such as visual versus infrared cameras or even 5 Manifold learning algorithms such as Isomap deal with biased estimates of distances by centering matrix of squared distances, i.e., removing mean of each row/column every element.

12 DAVENPORT et al.: JOINT MANIFOLDS FOR DATA FUSION 2591 Fig. 6. (top) Noisy captured images. SNR 3:5 db; (bottom) 2-D embeddings learned via Isomap noisy images. The joint manifold structure helps ameliorate effects of noise. visual plus audio data), thus component manifolds may be scaled differently; 2) due to occlusions or partially-overlapping fields of view, certain regions of some component manifolds may be ill-conditioned. In order to hle such nonidealities, we make two modifications to Isomap algorithm while performing joint manifold learning. Recall that that in order to find nearest-neighbor graph, Isomap must first calculate matrix of squared pairwise Euclidean distances. We denote this matrix for joint manifold for component manifold. Note that. Thus, if a particular component manifold is scaled differently than ors, by which we mean that with, n all entries of corresponding will be reweighted by, so that will have a disproportionate impact on. This corresponds to first nonideality described previously, can be alleviated by normalizing each by its Frobenius norm, which can be interpreted as scaling each manifold so that an Eulerian path through complete graph has unit length. The second nonideality can be partially addressed by attempting to adaptively detect correct for occlusion events. Consider case of large-scale occlusions, in which we make simplifying assumption that for each camera object of interest is eir entirely within camera s view or entirely occluded. In this case, nonoccluded component manifolds are still locally isometric to each or, i.e., re exists a neighborhood such that for all for all corresponding to nonoccluded component manifolds. Thus, if we knew which cameras were occluded for a pair of points, say, n we could simply ignore those cameras in computing rescale so that it is comparable with case when no cameras exhibit occlusions. More specifically, we let denote index set for nonoccluded component manifolds set. To do this automatically, we compare to a specific threshold to zero when it is below a certain value, i.e., we set for some parameter, since for component manifolds in which object of interest is occluded this distance will be relatively small. The parameter can be reasonably inferred data. is used by subsequent steps in Isomap to learn an improved low-dimensional embedding of high-dimensional acquired data. Note that while this approach does not rigorously hle boundary cases where objects are only partially occluded, our experimental results, shown in following, indicate that algorithms are robust to such cases. C. Experiments We provide a variety of results using both simulated real data that demonstrate significant gains obtained by exploiting joint manifold structure, both with without use of rom projections. The manifold learning results have been generated using Isomap [30]. For ease of presentation, all of our experiments are performed on 2-D image manifolds, which are isomorphic, to a closed rectangular subset of. Thus, ideally 2-D embedding of data should resemble a rectangular grid of points that correspond to samples of joint manifold in high dimensional space. 1) Manifolds Isometric to Euclidean Space: As a first example, we consider three different manifolds formed by pixel images of an ellipse with major axis minor axis translating in a 2-D plane, for ; an example point is shown in Fig. 5. The eccentricity of ellipse directly affects condition number of image articulation manifold; in fact, it can be shown that manifolds associated with more eccentric ellipses exhibit higher values for condition number. Consequently, we expect that it is harder to learn such manifolds. Fig. 5 shows that this is indeed case. We add a small amount of i.i.d. Gaussian noise to each image apply Isomap to both individual datasets as well as concatenated dataset. We observe that 2-D rectangular embedding is poorly learnt for each of component manifolds but improves visibly for joint manifold. 2) Gaussian Noise in Realistic Images: We now demonstrate how using joint manifolds can help ameliorate imaging artifacts such as Gaussian noise in a more realistic setting. We test our

13 2592 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 10, OCTOBER 2010 Fig. 7. (top) Sample images of 2 koalas moving along individual 1-D paths, yielding a 2-D manifold; (middle) 2-D embeddings of dataset learned via Isomap N =76800pixel images; (bottom) 2-D embeddings of dataset learned M = 2400 rom projections. Learning joint manifold yields a much improved 2-D embedding. proposed joint manifold learning approach on a set of syntic truck images. The data comprises a set of 540 views of a truck on a highway 3 vantage points. Each image is of size. The images are parametrized by 2-D location of truck on road; thus, each of image data sets can be modeled by a 2-D manifold. Sample views are shown in Fig. 4; for this experiment, we only use images Class 2. We convert images to grayscale, so that ambient dimension of data each camera lies in. Next, we add i.i.d. Gaussian noise to each image attempt to learn 2-D manifold. The noise level is quite high, as evidenced by sample images in Fig. 6. It is visually clear 2-D embedding results that learning performance improves markedly when data is modeled using a joint manifold, thus providing numerical evidence for Theorem 7. 3) Real Data Experiment Learning With Occlusions: We now test our methods on data a camera network; images are obtained a network of four Unibrain OEM Firewire board cameras. Each camera has resolution. The data comprises different views of independent motions of 2 toy koalas along individual 1-D paths, yielding a 2-D combined parameter space. This data suffers real-world artifacts such as fluctuations in illumination conditions variations in pose of koalas; furr, koalas occlude one anor in certain views or are absent certain views depending upon particular vantage point. Sample images 2-D embedding results are displayed in Fig. 7. We observe that best embedding is obtained by using modified version of Isomap for learning joint manifold. To test effectiveness of data fusion method described in Section III-B, we compute rom projections of each image sum m to obtain a romly projected version of joint data repeat previously shown experiment. The dimensionality of projected data is only 3% of original data; yet, we see very little degradation in performance, thus displaying effectiveness of rom projection-based fusion. 4) Real Data Experiment Unsupervised Target Tracking: As a practical application of manifold learning, we consider a situation where we are given a set of training data consisting of images of a target moving through a region along with a set of test images of target moving along a particular trajectory. We do not explicitly incorporate any known information regarding locations of cameras or parameter space describing target s motion. The training images comprise views of a coffee mug placed at different positions on an irregular rectangular grid. Example images each camera are shown in Fig. 8. For test data, we translate coffee mug so that its 2-D path traces out shape of letter R. We aim to recover this shape using both test training data. To solve this problem, we attempt to learn a 2-D embedding of joint manifold using modified version of Isomap detailed in Section V-B. The learned embedding for each camera is shown in Fig. 8. As is visually evident, learning data using any one camera yields very poor results; however learning joint manifold helps discern 2-D structure to a much better degree. In particular, R trajectory in test data is correctly recovered only by learning joint manifold. Finally, we repeat previously shown procedure using rom projections of each image, fuse data by summing measurement vectors. While recovered trajectory of anomalous (test) data suffers some degradation in visual quality, we observe comparable 2-D embedding results for individual joint manifolds as with original data set. Since dimensionality of projected data is merely 6% that of original data set, this would translate to significant savings in communication costs in a real-world camera network.

14 DAVENPORT et al.: JOINT MANIFOLDS FOR DATA FUSION 2593 Fig. 8. (top) Sample images of 2-D movement of a coffee mug; (middle) 2-D embeddings of dataset learned via Isomap N = pixel images; (bottom) 2-D embeddings of dataset learned via Isomap M = 4800 rom projections. The black dotted line corresponds to an R -shaped trajectory in physical space. Learning joint manifold yields a much improved 2-D embedding of training points, as well as R -shaped trajectory. VI. DISCUSSION Joint manifolds naturally capture structure present in data produced by camera networks. We have studied topological geometric properties of joint manifolds, have provided some basic examples that illustrate how y can improve performance of common signal processing algorithms. We have also introduced a simple framework for data fusion for camera networks that employs independent rom projections of each image, which are n accumulated to obtain an accurate low-dimensional representation of joint manifold. Our fusion scheme can be directly applied to data acquired by such devices. Furrmore, while we have focused primarily on camera networks in this paper, our framework can be used for fusion of signals acquired by many generic sensor networks, as well as multimodal joint audio/video data. ACKNOWLEDGMENT The authors would like to thank J. P. Slavinsky for his help in acquiring data for experimental results presented in this paper. REFERENCES [1] H. Lu, Geometric ory of images, Ph.D. dissertation, Univ. California, San Diego, [2] D. Donoho Grimes, Image manifolds which are isometric to Euclidean space, J. Math. Imag. Comput. Vis., vol. 23, no. 1, pp. 5 24, [3] M. Wakin, D. Donoho, H. Choi, R. Baraniuk, The multiscale structure of non-differentiable image manifolds, in Proc. Wavelets XI at SPIE Optics Photonics, San Diego, CA, Aug [4] M. Belkin P. Niyogi, Using manifold structure for partially labelled classification, in Proc. Advances in Neural Information Processing Systems, Vancouver, BC, Dec. 2003, vol. 15, pp [5] J. Costa A. Hero, Geodesic entropic graphs for dimension entropy estimation in manifold learning, IEEE Trans. Signal Process., vol. 52, no. 8, pp , Aug [6] M. Turk A. Pentl, Eigenfaces for recognition, J. Cogn. Neurosci., vol. 3, no. 1, pp , [7] G. Hinton, P. Dayan, M. Revow, Modelling manifolds of images of hwritten digits, IEEE Trans. Neural Netw., vol. 8, no. 1, pp , Jan [8] D. Broomhead M. Kirby, The whitney reduction network: A method for computing autoassociative graphs, Neural Comput., vol. 13, pp , [9] D. Donoho, Compressed sensing, IEEE Trans. Inf. Theory, vol. 52, no. 4, pp , Apr [10] E. Cès, J. Romberg, T. Tao, Robust uncertainty principles: Exact signal reconstruction highly incomplete frequency information, IEEE Trans. Inf. Theory, vol. 52, no. 2, pp , Feb [11] M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, R. Baraniuk, Single pixel imaging via compressive sampling, IEEE Signal Process. Mag., vol. 25, no. 2, pp , Mar [12] S. Lafon, Y. Keller, R. Coifman, Data fusion multicue data matching by diffusion maps, IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 11, pp , Nov [13] C. Wang S. Mahadevan, Manifold alignment using Procrustes analysis, in Proc. Int. Conf. Machine Learning (ICML), Helsinki, Finl, Jul. 2008, pp [14] W. Boothby, An Introduction to Differentiable Manifolds Riemannian Geometry. New York: Academic, [15] P. Niyogi, S. Smale, S. Weinberger, Finding homology of submanifolds with confidence rom samples, Univ. Chicago, 2004, Tech. Rep. TR [16] Y. Keller, R. Coifman, S. Lafon, S. Zucker, Audio-visual group recognition using diffusion maps, IEEE Trans. Signal Process., vol. 58, no. 1, pp , Jan [17] R. Baraniuk M. Wakin, Rom projections of smooth manifolds, Found. Comput. Math., vol. 9, no. 1, pp , [18] M. Duarte, M. Davenport, M. Wakin, J. Laska, D. Takhar, K. Kelly, R. Baraniuk, Multiscale rom projections for compressive classification, in Proc. IEEE Int. Conf. Image Processing (ICIP), San Antonio, TX, Sep. 2007, pp [19] C. Hegde, M. Wakin, R. Baraniuk, Rom projections for manifold learning, in Proc. Advances in Neural Information Processing Systems, Vancouver, BC, Dec. 2007, vol. 20, pp [20] W. Johnson J. Lindenstrauss, Extensions of Lipschitz mappings into a Hilbert space, in Proc. Conf. in Modern Analysis Probability, 1984, pp [21] M. Rabbat, J. Haupt, A. Singh, R. Nowak, Decentralized compression predistribution via romized gossiping, in Proc. Int. Work. Info. Proc. in Sensor Networks (IPSN), Apr. 2006, pp [22] W. Bajwa, J. Haupt, A. Sayeed, R. Nowak, Compressive wireless sensing, in Proc. Int. Workshop Inf. Proc. in Sensor Networks (IPSN), Apr. 2006, pp

15 2594 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 10, OCTOBER 2010 [23] V. Cevher, A. Sankaranarayanan, M. F. Duarte, D. Reddy, R. G. Baraniuk, R. Chellappa, Compressive sensing for background subtraction, in Proc. Eur. Conf. Comp. Vision (ECCV), Marseille, France, Oct. 2008, pp [24] S. Nayar, V. Branzoi, T. Boult, Programmable imaging using a digital micromirror array, in Proc. Eur. Conf. Comp. Vision (ECCV), Washington, DC, Jun. 2004, pp [25] U. Grener, A. Srivastava, M. I. Miller, Asymptotic performance analysis of Bayesian object recognition, IEEE Trans. Inf. Theory, vol. 46, no. 4, pp , Jul [26] U. Grener, M. I. Miller, A. Srivastava, Hilbert-Schmidt lower bounds for estimators on matrix Lie groups, IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 8, pp , Aug [27] M. Davenport, M. Duarte, M. Wakin, J. Laska, D. Takhar, K. Kelly, R. Baraniuk, The smashed filter for compressive classification target recognition, in Proc. SPIE Symp. Elec. Imaging: Comp. Imaging, [28] M. Davenport, C. Hegde, M. Duarte, R. Baraniuk, A oretical analysis of joint manifolds, ECE Dept., Rice Univ., 2009, Tech. Rep. TREE0901. [29] W. Hoeffding, Probability inequalities for sums of bounded rom variables, J. Amer. Statist. Assoc., vol. 58, no. 301, pp , [30] J. Tenenbaum, V. de Silva, J. Lford, A global geometric framework for nonlinear dimensionality reduction, Science, vol. 290, pp , [31] S. Roweis L. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science, vol. 290, pp , [32] D. Donoho C. Grimes, Hessian eigenmaps: Locally linear embedding techniques for high dimensional data, Proc. Nat. Acad. Sci., vol. 100, no. 10, pp , [33] M. Bernstein, V. de Silva, J. Langford, J. Tenenbaum, Graph approximations to geodesics on embedded manifolds, Stanford Univ., Stanford, CA, Mark A. Davenport (S 04 M 10) Mark A. Davenport received (S 04) B.S.E.E., received M.S., B.S.E.E., Ph.D. degrees in electrical M.S., computer Ph.D. engineering degrees in 2004, electrical 2007, computer 2010, respectively, B.A. engineering degree managerial in 2004, 2007, studies in 2004, 2010, all respectively, Rice University, Houston, TX. B.A. degree in managerial studies in 2004, He is currently an NSF all Mamatical Rice University, Sciences Houston, Post-Doctoral TX. Research Fellow in Department of HeStatistics currently at Stanford NSFUniversity, Mamatical Stanford, Sciences CA. His research interests include Post-Doctoral compressive Research sensing, Fellow nonlinear approximation, Department application of low-dimensional of Statistics at signal Stanford models University, to a variety Stanford, of problems CA. in signal processing machine His research learning. interests He isinclude also cofounder compressive an sensing, editor of Rejecta Mamatica. nonlinear approximation, application of lowdimensional Hershel M. signal Rich Invention models to Award a variety of Rice problems in 2007 Dr. Davenport shared for in signal his work processing on single-pixel machine camera learning. He compressive is also cofounder sensing. an editor of Rejecta Mamatica. Dr. Davenport shared Hershel M. Rich Invention Award Rice in 2007 for his work on compressive sensing single-pixel camera. Chinmay Hegde (S 06) received B. Tech. degree in electrical engineering Indian Institute of Technology, Madras, Chennai, India, in 2006, is currently working toward graduate degree in Department of Electrical Computer Engineering Chinmay at Rice University, Hegde (S 06) Houston, received TX. B. Tech. degree His research interests include electrical compressive engineering sensing, statistical, Indian Institute manifoldbased signal processing. Technology, Madras, Chennai, India, in 2006, is of currently working toward graduate degree in Department of Electrical Computer Engineering at Rice University, Houston, TX. His research interests include compressive sensing, statistical manifold-based signal processing. Marco F. Duarte (S 99 M 09) Marcoreceived F. Duarte (S 99 M 09) B.Sc. degreereceived in computer B.Sc. engineering (with distinction) degree in M.Sc. computer degreeengineering electrical (with engineering distinction) University of Wisconsin-Madison M.Sc. in degree 2002 in electrical 2004, respectively, engineering Ph.D. degree in electrical engineering University of Rice Wisconsin-Madison University, Houston, in 2002 TX, in , respectively, Ph.D. degree in electrical He is currently an NSF/IPAM engineering Mamatical Rice Sciences University, Post-Doctoral Houston, Research TX, in Fellow in Program of Applied Computational Mamatics at Princeton University, Princeton, NJ. HisHe research is currently interests an include NSF/IPAM machine Mamatical learning, compressive sensing, sensor networks, ences Postdoctoral optical image Research coding. Fellow in Pro- Sci- Dr. Duarte received gram Presidential of Applied Fellowship Computational TexasMamatics Instruments Distinguished Fellowshipat inprinceton 2004, as well University, as Princeton, Hershel M. NJ. Rich HisInvention research Award interests include 2007 formachine his worklearning, on compressive compressive sensing sensing, sensor singlenetworks, pixel camera, all optical image Rice University. coding. He is also a member of Tau Beta Pi SIAM. Dr. Duarte received Presidential Fellowship Texas Instruments Distinguished Fellowship in 2004, as well as Hershel M. Rich Invention Award in 2007 for his work on compressive sensing single pixel Richard camera, all G. Baraniuk Rice University. (M 93 SM 98 F 01) He is also received a member of B.Sc. Tau degree Beta Pi SIAM. University of Manitoba, Winnipeg, MB, Canada, in 1987, M.Sc. degree University of Wisconsin-Madison, in 1988, Ph.D. degree Richard G. Baraniuk (M 93 SM 98 F 01) received B.Sc. degree University of Man- University of Illinois at Urbana-Champaign, in 1992, all in electrical engineering. itoba, Winnipeg, MB, Canada, in 1987, M.Sc. After spending with Signal Processing Laboratory of Ecole degree University of Wisconsin-Madison, Normale Supérieure, Lyon, France, he joined Rice University, Houston, TX, in 1988, Ph.D. degree University where he is currently Victor E. Cameron Professor of Electrical Computer Engineering. He spent sabbaticals at Ecole Nationale Supérieure de Télé- of Illinois at Urbana-Champaign, in 1992, all in electrical engineering. communications in Paris in 2001 Ecole Fédérale Polytechnique de Lausanne After spending with Signal Processing Laboratory of Ecole Normale Suprieure, in Switzerl in His research interests lie in area of signal image processing. In 1999, he founded Connexions (cnx.org), a nonprofit publishing Lyon, France, he joined Rice University, Houston, project that invites authors, educators, learners worldwide to create, rip, TX, where he is currently Victor E. Cameron mix, burn free textbooks, courses, learning materials a global Professor of Electrical Computer Engineering. He spent sabbaticals at open-access repository. Ecole Nationale Suprieure de Tlcommunications in Paris in 2001 Ecole Prof. Baraniuk has been a Guest Editor of several special issues of IEEE Fdrale Polytechnique de Lausanne in Switzerl in His research SIGNAL PROCESSING MAGAZINE, IEEE JOURNAL OF SPECIAL TOPICS IN SIGNAL interests lie in area of signal image processing. In 1999, he founded PROCESSING, PROCEEDINGS OF THE Connexions (cnx.org), a nonprofit publishing IEEE project that has invites served authors, as technical educators, program chair learners or on worldwide technical to program create, rip, committee mix, for burn several free IEEE textbooks, workshops courses, conferences. learning materials He received a a NATO global Post-Doctoral open-access repository. fellowship NSERC Prof. in Baraniuk 1992, has National been a Young Guest Investigator Editor of several award special issues National of Science IEEE Foundation Signal Processing in 1994, Magazine, a Young Investigator IEEE Journal Award of Special Topics Office in of Signal Naval Research Processing, in 1995, Rosenbaum Proceedings Fellowship of IEEE Isaac has served Newton as Institute technical of Cambridge program chair University or on in 1998, technical C. Holmes program MacDonald committee National for several Outsting IEEE Teaching workshops Award conferences. Eta Kappa He Nu received in 1999, a NATO Charles postdoctoral Duncan Junior fellowship Faculty Achievement NSERC Award in 1992, Rice National in 2000, Young University Investigator of Illinois award ECE Young Alumni National Achievement Science Foundation Award in in 2000, 1994, a Young George Investigator R. Brown Award Award for Superior Office Teaching of Naval at Rice Research in 2001, in 1995, 2003, Rosenbaum 2006, Hershel Fellowship M. Rich Invention Isaac Award Newton Institute Rice of in Cambridge 2007, Wavelet University Pioneer in 1998, Award C. Holmes SPIE inmacdonald 2008, National Internet Outsting Pioneer Award Teaching Award Berkman Center Eta for Kappa Internet Nu in 1999, Society at Harvard Charles Law Duncan School Junior in Faculty HeAchievement was selected as Award one of Edutopia Rice inmagazine s 2000, Daring University Dozen of educators Illinois ECE in Young Connexions Alumni Achievement received Award Tech Museum in 2000, Laureate George Award R. Brown Award Tech Museum for Superior of Innovation Teachinginat2006. RiceHis in 2001, work with 2003, Kevin Kelly 2006, on Hershel Rice single-pixel M. Rich Invention compressive Award camera was Riceselected in 2007, by MIT Wavelet Technology PioneerReview AwardMagazine SPIE as in a TR , Top 10 Emerging Internet Pioneer Technology Award in He was Berkman coauthor Center on a paper for Internet with M. Crouse Society R. atnowak Harvard that Law wonschool IEEE insignal Processing He was selected Societyas Junior one of Paper Edutopia AwardMagazines 2001 Daring anordozen with V. educators Ribeiro in R Riedi Connexions that won received Passive Tech Active Museum Measurement Laureate(PAM) AwardWorkshop Tech Best Student MuseumPaper of Innovation Award in in HeHis was work elected witha Kevin FellowKelly of onaaas Rice in singlepixel compressive camera was selected by MIT Technology Review Magazine as a TR10 Top 10 Emerging Technology in He was coauthor on a paper with M. Crouse R. Nowak thatwon IEEE Signal Processing Society Junior Paper Award in 2001 anor with V. Ribeiro R. Riedi that won Passive Active Measurement (PAM) Workshop Best Student Paper Award in He was elected a Fellow of IEEE in 2001 a Fellow of AAAS in 2009.