Brian Noguchi CS 229 (Fall 05) Project Final Writeup A Hierarchical Application of ICA-based Feature Extraction to Image Classification Brian Noguchi

Size: px

Start display at page:

Download "Brian Noguchi CS 229 (Fall 05) Project Final Writeup A Hierarchical Application of ICA-based Feature Extraction to Image Classification Brian Noguchi"

Hugo Jennings
5 years ago
Views:

1 A Hierarchical Application of ICA-based Feature Etraction to Iage Classification Introduction Iage classification poses one of the greatest challenges in the achine vision and achine learning counities. For all its advances, achine vision cannot nearly achieve the caliber of huan vision. Huan beings iage classification syste is both fast and accurate. Siply put, it works, while achine vision has a ways to go before reaching the sae level. Given this doinance of huan vision over achine vision, it ight help to adopt soe ideas fro the huan visual syste in an attept to iprove achine vision. Fortunately, research has produced atheatical tools that iic parts of the brain s visual corte. Recent literature has shown that when you apply independent coponents analysis (ICA) to natural iages, the resulting independent coponents are siilar to the detection of edges in an iage. Fascinatingly, these independent coponents can be likened to neurons in the brain. Siple cell neurons in the visual regions, V and V, of the brain respond to certain features in an iage input such as oriented edges. Intuitively, the independent coponents of iages are tuned to the priitive features to which siple cells are ost responsive. Thus, if we wanted to iic the way the brain achieves iage recognition, we ight want to use ICA to first etract the ost salient priitive features of iages. One way to think about the result of ICA is a set of feature filters called independent coponent filters (ICFs), that when applied to an iage or portion of an iage, easure the presence of a feature (or equivalently independent coponent) in the iage. Thus, each filter is siilar to a neuron in the brain where every neuron is set to fire whenever it detects one specific feature. After the brain etracts the strongest features in an iage, it does further analysis of this inforation in order to arrive at a conclusion about what is in the iage. Soe of the neuroscience counity believes that the brain works hierarchically where each level uses essentially the sae algorith to etract higher and higher order levels of inforation. If we follow such a theory, we would be tepted to apply a second layer of ICA on top of the first layer. This is what Lindgren et al [] did, and they achieved soewhat proising results. For each second layer filter, they showed the top five iages that had the largest agnitude response to that filter. Interestingly, they found that soe filters were really tuned to certain natural iages. For eaple, one filter responded best to five iages that reseble clouds or possess cloud tetures, while another filter grouped dense forest scenery together. However, the authors udged their results on what they thought the iage clusters appeared to represent, and in fact used surveys of people to ustify their interpretation. In other words, the results, although proising, were not copletely and rigorously validated. Therefore, one of the tasks that I decided to undertake is what happens when we apply Lindgren et al s algorith to a ore structured and labeled data set of individual obects. In Section, I describe Lindgren et al s algorith. In Section, I present and discuss the results of applying the two-layer ICA algorith to a ore inforative iage dataset. Despite showing a lot of proise, the results for this data are very confusing and inconsistent. In a belief that I can still harness the proise of the double application of ICA, I then use supervised learning on the results of the second layer of ICA in an attept to learn how to recognize obects in iages. This supervised learning algorith is described in Section 3, and its results are presented in Section 4. Ultiately, the chosen supervised learning algorith does not do any better than randoly guessing what obect is in an iage. Therefore, we conclude that although the results

2 of Lindgren et al s algorith ay be harnessed for iage classification, the results are actually ore difficult to interpret than they originally appeared in their paper. Section : An Overview of the Two-Layer ICA Algorith [] Lindgren et al apply ICA twice in the following anner. The first application of ICA is relatively straightforward. Beginning with a set of N iages, they choose p rectangular subregions, called patches, fro each iage. Each iage becoes an observation in the dataset { i i,..., } where in this case N p. They then apply ICA on this dataset and consequently attain the uniing atri W such that s W where ( i ) s is the th source that is causing our observed th signal. Each row of the uniing atri W can be thought of as the row vector representing an independent coponent filter (ICF). In order to reduce the tie involved to calculate results, we apply principal coponents analysis (PCA) to the set of ICF s (independent coponent filters) in order to reduce the diension to d, so that we only have d filters. Net, Lindgren et al carry out the following steps in order to apply ICA a second tie on top of the first application. For each of the d filters and for each of the N iages, they convolute that filter with that entire iage. Altogether, this results in N d convolutions, each represented by a atri of values. For each convolution, the authors create a 0-diensional vector that represents a 0-bin histogra of responses of an iage s piels to a particular ICF. They use the L-nor to noralize this vector. Then for each iage, they concatenate the vectors of all filters, and noralize the resulting vector by the L nor. This construction will yield N colun vectors (i.e., one for each iage), of diension 0 d. Grouping all of these response vectors into a set { y i i,..., N}, we arrive at our new dataset to which we apply the second pass of ICA. Applying ICA to this dataset yields a second uniing atri W with each row vector corresponding to 0 d second level filters. Again, we apply PCA to the resulting set of second level ICFs to reduce the nuber of ICFs to d filters. The end result is two sets of filters (ICF s) Section : Application of the Two-Layer ICA Algorith to an Iage Database of Obects In order to see how Lindgren et al s algorith fares with an iage dataset of single obects, we applied the two-layer ICA algorith to subsets of obects fro the grayscale version of the Asterda Library of Obect Iages []. This dataset contains 000 obects with over 00 iages per obect. Moreover, each obect has iages fro different viewpoints and different illuination. Each iage had piel diensions of 9 44 and consisted of only one obect against a black background. We conducted tests of the algorith in the following anner. We ran separate eperients on 5, 0, and 50 obects. Each of these obect subsets was chosen randoly. Then for each subset of obects, we randoly chose 50% of the iages fro each obect s set of iages. We put one half in a training dataset and the other half in a testing dataset. To the training set, we applied the first pass of ICA, choosing iage patch diensions of. We then used PCA to choose the top 00 principal ICFs. Each of these filters was then convolved with each iage in the training set, and the proper filter response histogras were

3 recorded for each iage. Finally, after using the second pass of ICA, we used PCA on the resulting filters set to reduce the nuber of second layer ICFs. For each eperient, this put us in possession of two sets of ICFs, one set fro the first pass of ICA and the other set fro the second pass of ICA. Applying the second set of our ICFs to the dataset that we constructed fro the convolutions, we arrive at a set of independent coponents for each iage. These encode the responses of the iages to the higher level ICF s. An iage s response to the second filter, in this case, would be the dot product between the filter and the convolution histogra-derived vector for the iage. Hence, response of iage i to the response to a second level filter is the corresponding eleent in the independent coponent corresponding to that filter. (See figure) After working with the training set, we took our testing set and we convolved each test iage with each lower-level ICF derived fro our training set. Fro these convolutions, we constructed the appropriate ICF response histogra for each test iage and concatenate the into one vector for the iage. This gave us a new dataset to which we then applied our secondpass ICFs. This produced a new set of pseudo-independent coponents, which reflected the response of each iage in our second dataset to the second level filters. To visualize our results, we took each filter s top 36 responses and the worst 36 responses. Our results will be discussed in ters of this output forat. For the results involving the training set, we find that soe filters had all of their top 36 responses as iages fro one obect only (See Figure ). Siilarly soe filters had all of their worse 36 responses as iages fro one obect only. As can be seen in Figure, these filters appear to robustly identify an obect, despite etree changes in the obect orientation. When we analyze iages fro 50 obects and find 00 higher-level ICFs, we find a good nuber of the ICF s that are ICF s whose top 36 responses as the sae obect or whose worst 36 responses are the sae obect. In fact, there were 0 filters, each detecting only one type of obect in its top 36 responses; oreover, there were 8 filters, each detecting only one type of obect is its worst 36 responses. What is ore, we are not even counting the ICFs that alost had their top or botto 36 responses as the sae obect but were off by to 4 obects. Things really gets interesting, though, when you apply these 00 higher-level filters to the testing dataset. All of a sudden the nuber of these special discriinating ICF s ups fro 0 to 46 and fro 8 to 34 respectively. Moreover, a few obects in the test dataset see to appear to be the 36 greatest or sallest response iages for a disproportionate nuber of ICF s. This is odd on several counts. First of all, the testing dataset sees to be ore active in its interaction with the higher-level ICF s than the training dataset. Yet, the ICF s were derived fro the training dataset. A second oddity is that if one filter appeared to single out one obect in the training set, it did not single out the sae obect in the testing set. One would not have epected this to happen, since siilar iages should respond very siilarly when acted on by an ICF. In spite of these oddities, I think that it is still an aazing phenoenon that a lot of these filters appeared to single out one obect, even though given a different dataset of the sae obects it ight not single out the sae obect. One way to attept to resolve this issue is through supervised learning based on the higher-level independent coponents. Perhaps applying supervised learning ethods will uncover patterns in the iage responses that we soehow cannot huanly envision. 3

4 Section 3: Supervised Learning of Obects Based on Results fro the Two-Layer ICA Approach As we stated, our hope is to soehow find patterns of responses characteristic of each obect. That is, for each obect, we have a collection of responses to filters, one response per iage. If we can learn how likely a certain response to a collection of filters is given knowledge of what the obect is, then we can use Bayes rule to find the ost probable obect given the response of any unlabeled iage to a collection of filters. Hence, it would be intuitive to develop a generative supervised learning algorith. Thus, we decided to develop a odel siilar to a iture of Gaussians odel, ecept that the odel we now describe is for supervised learning and also has ore than one covariance atri. Let z be the rando variable representing what obect we label iage i. Assue that each z is drawn fro the sae ultinoial distribution. Moreover, let be the rando variable representing the vector of iage i s responses to the second-level ICFs. Assue that ( given we know the iage label z i ), then is drawn fro a Gaussian with ean vector µ and covariance atri Σ. Hence our odel is given by z i ~ Multinoial( φ) z ~ Gaussian( µ, Σ ) This is essentially a iture of Gaussians odel with the iportant eception that, here, all the labels do not necessarily share the sae covariance atri. The aiu likelihood estiates of our paraeters are found to be φ { z } Σ µ i i i { z i i { z ) ) { z }( µ )( µ ) { z Our goal is to find the obect that aiizes the conditional probability of the obect given the response-to-filters data, p( z ; φ, µ, Σ ). Thus, the ost likely obect that iage i should be is given by arg a p( ( i ) z z ; µ, Σ } ) p( z Section 4: Results of Supervised Learning Despite the belief that applying supervised learning would soehow uncover any tendencies toward a filter response each obect s iages, our results show otherwise. We found our aiu likelihood paraeters using the training dataset, and we calculated the ost likely obect labels in our testing dataset. We did this any ties for datasets consisting of obects iages. What we found was that, on average, we correctly identified an obect correctly approiately 50% of the tie. This is no better than rando. We thought that perhaps it was due to a sall training set, so we also tried again with a dataset consisting of 00 obects iages. Again, the results appear to be rando, with a successful identification rate of approiately %. ; φ) T 4

Conclusion Even though ICA appears to be a very proising tool for use in iage recognition, we found that its eact use beyond a first-level use of ICA appears cryptic.

5 Conclusion Even though ICA appears to be a very proising tool for use in iage recognition, we found that its eact use beyond a first-level use of ICA appears cryptic. The tests that we ran on the Asterda Library of Obect Iages were full of unusual results. Obviously, soe for of obect discriination is present. However, even application of a supervised learning odel did not appear to resolve the issue. In conclusion, I still think ICA should be eplored for use in iage recognition, but in what capacity and eactly how we can eploit that proise is still an unanswered question. [] J.T. Lindren and A. Hyvarinen. Learning high level independent coponents of iages through a spectral representation. Proceedings of the7th International Conference on Pattern Recognition (ICPR 04). [] J. M. Geusebroek, G. J. Burghouts, and A. W. M. Seulders, The Asterda library of obect iages, Int. J. Coput. Vision, 6(), 03-, January, 005. Figure : The iages that correspond to one ICF s top 36 responses. As you can see, the filter detects only one obect despite changes in the viewing angle. 5

TensorFlow and Keras-based Convolutional Neural Network in CAT Image Recognition Ang LI 1,*, Yi-xiang LI 2 and Xue-hui LI 3

2017 2nd International Conference on Coputational Modeling, Siulation and Applied Matheatics (CMSAM 2017) ISBN: 978-1-60595-499-8 TensorFlow and Keras-based Convolutional Neural Network in CAT Iage Recognition