Adapting K-Medians to Generate Normalized Cluster Centers

Transcription

1 Adapting -Medians to Generate Normalized Cluster Centers Benamin J. Anderson, Deborah S. Gross, David R. Musiant Anna M. Ritz, Thomas G. Smith, Leah E. Steinberg Carleton College {dgross, dmusian, ritza, smitht, Abstrat Many appliations of lustering require the use of normalized data, suh as text or mass spetra mining. The spherial -means algorithm [6], an adaptation of the traditional -means algorithm, is highly useful for data of this kind beause it produes normalized luster enters. The -medians lustering algorithm is also an important lustering tool beause of its wellknown resistane to outliers. -medians, however, is not trivially adapted to produe normalized luster enters. We introdue a new algorithm (alled MN), inspired by spherial -means, that integrates with - medians lustering to produe loally optimal normalized luster enters. We then show theoretially and experimentally that MN produes lusters of signifiantly higher quality than one would obtain via a simple saling of the luster enters produed from traditional -medians. eywords: -medians, lustering, normalized data. Introdution Clustering is one of the key tehniques used in data mining. In this paper we fous on lustering normalized datasets. Data is often normalized before lustering in order to remove unimportant distintions of sale. In lustering a orpus of text, for example, eah doument is often represented as a point where eah dimension represents a word s frequeny. When lustering data in this format, two douments of idential subet matter with different lengths should not be onsidered different. It therefore often makes sense to normalize text data before lustering. Single-partile mass spetrometry [] is another area where it is appropriate to normalize data. The hemial makeup of an aerosol partile is represented as two mass spetra, whih are plots of signal intensity versus the mass-to-harge ratio (m/z) for positive and negative ions produed by fragmenting the omponents of the aerosol partile. Thus, the presene of a peak indiates the presene of one or more ions ontaining the m/z value indiated (see Figure 5 for examples). As with text, we are generally not interested in the absolute magnitude of these peaks, but in the relative size of one peak when ompared with another. Cluster enters are often used to represent prototypial points, so it is usually desirable to normalize luster enters that arise from normalized data [5, 6]. The spherial -means algorithm [6] is well-suited to this task. For some appliations it is more desirable to use - norm distane (also known as Manhattan distane, denoted here as ) to measure the distane between points. This is beause the luster enter that minimizes -norm distane to all points within that luster is the median of that luster, and using the median instead of the mean tends to be more robust to outliers. The -medians algorithm [4, 9] is thus a powerful alternative. When attempting to use -medians on normalized data, however, unique hallenges arise in finding normalized loally optimal luster enters. We present the Manhattan Normalization (MN) algorithm, whih when integrated with -medians addresses these hallenges. There are two seemingly straightforward solutions to the problem of finding normalized luster enters with -medians. One ould apply traditional - medians and sale the luster enters at the end, or alternatively one ould use traditional -medians and sale the luster enters after eah iteration. Both of these ideas are heuristially based and lak theoretial guarantees of onvergene. We will show that MN integrated with -medians has guaranteed onvergene properties and performs better experimentally than either of these other approahes. We borrow many of our ideas from spherial -means, but we address the additional diffiulties introdued by the use of medians.

2 Setion 2 of this paper desribes relevant known lustering algorithms. Setion 3 desribes the hallenges that arise in normalized -medians lustering and our solutions to those hallenges. Experimental results and analysis are in Setion Clustering Review 2. Traditional -means The well-known -means algorithm [8] is used when one wishes to find luster enters that minimize the total of the squared 2-norm distane (also known as Eulidean distane, denoted here as 2 ) from eah point to its losest luster enter. Sine finding globally optimal luster enters is an NP-hard problem, -means may be used to find a loal solution. In order to run -means, one hooses the number of lusters to find and an initial set of luster enters. There are many different approahes for hoosing initial luster enters [3], but all of our work here holds regardless of whih tehnique is used. The -means framework an then be desribed as follows: -means Algorithm. Let x,x 2,...,x n be a set of points. We wish to partition them into disoint * * * lusters π, π2,..., π that minimize the obetive funtion: 2 Q({ π } = ) = x () = x π For eah luster, is the enter of eah luster, 2 whih is defined as the point for whih x is 2 x π minimized. This has been proven to be aomplished by hoosing to be the entroid of luster, defined as = x (2) n x π where n is the number of points in luster [8]. We therefore wish to find lusters that solve the following minimization problem: * { π } = arg min Q ({ π } ) (3) = = { π } = To do this, start with an arbitrary partitioning of the () π. Define the luster enters assoiated data { } = () with this partitioning as { } =. Define the index of iteration t =. Then follow these steps:. For eah point, find the luster enter with losest Eulidean distane. This yields a new partitioning { x x x i 2 2 } π = : i,, (4) =, where ties between lusters are resolved by random assignment to one of the optimal enters. Note that this is guaranteed not to inrease the obetive Q, sine eah point is assigned to its losest enter. by 2. Compute the new set of luster enters { } = omputing the mean (entroid) of eah luster. Sine the entroid is the point that minimizes the total distanes from all points to it, this step is also guaranteed not to inrease the obetive Q. 3. If a stopping riterion is met, report { π } final partitioning and { } = = as the as the final luster enters. Otherwise, inrement t by, and go to step above. A variety of stopping onditions are available. One ommon ondition is to stop when the differene between suessive values of the obetive Q is less than a small tolerane. -means is ultimately a loal optimization algorithm for minimizing lustering error, where lustering error is defined as the total squared Eulidean distane from eah point to its losest enter. The obetive Q never inreases from one iteration to the next. Sine -means is applied to a finite number of points, the algorithm must therefore terminate. 2.2 Spherial -means Many appliations for lustering normalized data require normalized luster enters. The spherial - means algorithm by Dhillon and Modha [6] addresses this need. The spherial -means literature uses the osine similarity metri. We find it more onvenient for this paper to use the squared Eulidean distane metri for spherial -means, but it is easily shown that for normalized data both of these metris yield preisely the same results. Spherial -means produes luster enters of magnitude by normalizing the luster enters after eah iteration. In other words, an extra step is added to the traditional -means algorithm as follows: Spherial -means algorithm. Start with a partitioning of the data as in the traditional -means algorithm. Initialize t =.. For eah point, find the losest luster enter as measured via squared Eulidean distane.

3 2. Compute the new set of luster enters { } = omputing the mean (entroid) of eah luster. 3. Normalize eah luster enter by saling it so that it has a magnitude of. In other words, redefine : =, =,, (5) 2 4. Terminate if the stopping ondition is met. Inrement t and go to step otherwise. Note that step 3 may atually inrease the lustering error following step 2, sine step 2 finds the luster enters that optimize luster error regardless of luster enter magnitude. Step 3 modifies the optimal luster enters so that they have a magnitude of at the expense of inreasing lustering error. However, Dhillon and Modha show [6] that if step 2 and step 3 are onsidered together as one operation, this proedure finds the optimal enter of magnitude for eah luster. The property that luster enters have a magnitude of is thus preserved from one iteration to the next, while the algorithm ensures that luster error does not inrease. Spherial -means is therefore guaranteed to onverge to a solution of loally optimal luster enters, eah with magnitude of medians We now review the -medians algorithm [4, 9], whih is used when one wishes to minimize the total - norm distane from eah point to its nearest luster enter. -medians is quite similar to -means, and its differenes from -means are defined as follows: -medians algorithm. Sine we now work with - norm distane instead of squared Eulidean distane, our obetive is stated as: π = = = x π by Q({ } ) x (6) We start with a partitioning of the data as in - means. Initialize t =.. For eah point, find the losest luster enter as measured via -norm distane. 2. Compute the new set of luster enters { } = omputing the median of the luster. In other words, for eah dimension ompute the median value for that dimension over all points in the luster. We use the median beause the median is the point that minimizes the total -norm distane from all points to it [4]. 3. Terminate if the stopping ondition is met. Inrement t and go to step otherwise. by In a similar fashion to -means, steps and 2 of - medians are guaranteed not to inrease the obetive Q. 3. Normalized -medians luster enters There are a variety of tradeoffs in hoosing between -medians and -means. We onsider the disussion of the hoie of -medians vs. -means (-medians is slower but more robust to outliers, et.) outside the sope of this paper. Our purpose is to enable the appropriate use of -medians on normalized data. We mentioned in Setion some simple hanges to -medians that might seem to appropriately adapt it for obtaining normalized luster enters from normalized data. We disuss these ideas, point out their flaws, and then move on to disuss our solution. 3. Simple Approahes Simple approah #: normalize by saling at eah iteration. A straightforward adaptation of spherial -means is problemati. The onept seems to be easy enough: during eah iteration, after new luster enters have been determined, normalize them via saling using the appropriate distane metri. In other words, redefine luster enters as follows: : =, =, (7) There is a key problem with this approah. Unlike spherial -means, saled luster enters are not the points of magnitude that minimize total -norm distane from eah point to its luster enter. Example: Consider the -norm normalized points x = (/5, /5, 3/5), x 2 = (/5, /5, 3/5), x 3 = (/5, 3/5, /5), x 4 = (/5, 3/5, /5), and x 5 = (3/5,/5,/5). The median of these points is (/5, /5, /5). If we simply proportionately sale this median to have a magnitude of, we end up with a luster enter of (/3, /3, /3). If we measure -norm distane from eah point to this enter, eah point is a distane of 8/5 from this enter, resulting in a total distane of 4/5. Suppose that we instead hoose the normalized point (/5, 3/5, /5) to be our luster enter. This enter yields a total -norm distane of only 36/5. The saled median is therefore learly not the enter that minimizes total -norm distane from all points to it. Simple approah #2: normalize by (method of hoie) when -medians stabilizes. This approah is heuristi in nature, though it may yield positive results. If the normalization tehnique to be used is simple saling, however, problems arise as disussed above. Like Simple Approah #, Simple Approah #2

4 to show that an in fat be less than or greater than (as was done in Setion 3.). We also assume through the remainder of this algorithm and in the theorem that follows that, =,...,d. This is true for many appliations of normalized data. If < for a partiular, transform Figure : Graphial representation of our MN algorithm. Vertial lines are dimensions, horizontal lines are values (thik horizontal lines are the same value ourring twie), and irles are urrent luster enter loations. Sliding the seond irle upward inurs less error than sliding any other irle would sine more values are above it. does not provide desirable theoretial guarantees that spherial -means does. By performing regular - medians at eah iteration, the obetive does not inrease but the enters found at eah iteration do not satisfy desired onstraints (magnitude of ). After - medians stabilizes, when the luster enters are normalized, the obetive is likely to inrease steeply. 3.2 Our Solution: The MN Algorithm The struture of the spherial -means algorithm is sound. In order to obtain luster enters of magnitude, one should produe normalized luster enters iteratively throughout the algorithm suh that the obetive error metri ontinues to derease. Simple approah # tried to address this, but was flawed. The missing link needed here is an algorithm to solve the following task: given a set of points assigned to a luster, find a enter of magnitude (in a -norm sense) that minimizes the total -norm distane from all points to this enter. Here is our algorithm for doing so. Manhattan Normalization (MN) algorithm. Let x,x 2,...,x n be a set of points where x i =, i=,...,n. We wish to find a point, where =, that minimizes: n n d x = x (8) i i i= i= = where d is the number of dimensions, indiates the th omponent of luster enter, and x i represents the th omponent of point x i.. Initialize to be the median of x,x 2,...,x n. minimizes the obetive above, but it is not neessarily true that =. (If =, we terminate the algorithm.) Note that examples an be easily generated this dimension by negating both and all values x i (i=,...,n). On ompletion of the algorithm, negate again. This makes the rest of the algorithm easier to state (no speial ases for negatives), yet has no effet on its orretness. The algorithm is symmetri with respet to whether < or >, so we desribe the < ase and indiate the > ase in brakets. 2. For eah dimension, ount the number of values x i that are stritly greater than [less than]. Denote these ounts as z, =,...,d. 3. Let m be the dimension for whih z is maximal. If more than one dimension z has the same maximal value, hoose one arbitrarily. 4. Redefine m to be the smallest [largest] value x im (i=,...,n) greater than [less than] m. If m <, set m =. 5. If =, terminate the algorithm. If < m >, go to step 2. Otherwise, redefine m as ( ) ( ) m + and terminate. The idea behind the algorithm an be larified via Figure. At eah iteration, the dimension that we hange is the one that has a maximal number of values in the diretion that we want the enter to move. To see this, onsider sliding any of the irles in Figure upward a small distane ε, whih would have the effet of inreasing by ε. Furthermore, the obetive error metri (sum of -norm distanes from all points to the luster enter) is inreased by ε for eah tik on or below the irle moved, and dereased by ε for eah tik above the irle. Therefore, we slide the irle with the most tiks above it in order to inrease while inurring as little error as possible. We repeat this entire proess until =. With this intuition in mind, we prove the following theorem. Theorem: Given a set of points x,x 2,...,x n where x i =, i=,...,n, the MN algorithm finds a point ( =) that minimizes the total -norm error from all points to it. Formally, is the solution to the following optimization problem:

5 n n d arg min xi = i= i= = st.. = xi (9) Proof: We know that the initial value of as hosen in Step of the algorithm minimizes the following alternative problem [4]: arg min n xi () i= If =, then the theorem is trivially true. Suppose that <. Steps 2 and 3 hoose a dimension m of to modify. Suppose that m. Step 4 redefines the mth omponent of. We denote here the new value of as ĉ. We then observe that ĉ must be a solution for the following optimization problem: n n d arg min xi = xi i= i= = () st.. = ˆ To see this, reall that ĉ was obtained by adding ˆm m to the m th omponent of. This adds to the obetive an additional error of ˆm m times the number of values x im, i=,...,n less than or equal to m. Likewise, it subtrats from the obetive an error of ˆm m times the number of values x im, i=,...,n greater than m. Sine m was hosen to be the dimension with maximal values greater than m, and sine the total number of values in eah dimension is the same, any inrease at all in any other dimension of value less than or equal to ˆm m annot result in a lesser inrease of the obetive. This argument is easily adapted for the > and m < ases through appropriate reversals (positive / negative, less than / greater than, et.). We omit these other ases for spae saving purposes. Now that we have established the MN algorithm, we an integrate it with -medians: MN iterative -medians algorithm. We start with a partitioning of the data. Initialize t =.. For eah point, find the luster enter with losest -norm distane. 2. Compute the new set of luster enters { } = omputing the median of the luster. 3. Normalize eah luster so that it has a -norm magnitude of by using the MN algorithm. 4. Terminate if the stopping ondition is met. Inrement t and go to step otherwise. by If one wishes to use -medians on normalized data and yield normalized luster enters, MN iterative - medians will find luster enters at eah iteration with error no greater than that from the previous iteration. 3.3 MN Algorithm Performane Issues -medians is learly not as fast as -means due to its median omputations. Sine our MN algorithm ours after a median is alulated, any pre-existing algorithm for alulating medians quikly an be used. The MN algorithm itself, in its worst ase, ould require an iteration for eah value above the median in eah dimension. This is yields an upper bound of nd iterations. At the beginning of the algorithm, there is an initial step where the number of values greater than or less than the initial enter must be ounted for eah dimension. This also requires nd alulations (d dimensions, n points for eah), but this is only neessary one. Eah suessive iteration then requires determination of whih dimension has the greatest number of values above the urrent enter. This ould be handled via a priority queue where the priority is the number of values above the urrent enter. With d entries in the priority queue, this would result in log2 d alulations per iteration to update it. This yields a omplexity of Ond ( log d ) eah time that MN is used, i.e. at eah maor -medians iteration. For a sizeable dataset, then, MN forms a relatively negligible portion of alulation time. While there are many optimized approahes for alulating medians under ertain irumstanes, a straightforward modified quiksort algorithm has a omplexity of On ( log n) on average. Sine eah maor iteration of the -medians algorithm requires that a median be alulated in eah dimension within eah luster, this has a omplexity of Ond ( log n ). (In the ase where the n points are distributed evenly among the lusters, we an refine this omplexity to n n O d log. Sine is the number of lusters and is not expeted to be dramatially large, this omplexity an again be simplified to Ond ( log n ).) Therefore, for a large dataset where n d, these Ond ( log n ) alulations required by the median algorithm will ompletely dominate the Ond ( log d) alulations required by the MN algorithm. It should also be pointed out that when the data is all non-negative, as our atmospheri data is, equation (8) an be formulated as a linear program (LP). An LP

6 solver ould therefore be used as an alternative to our MN algorithm. To test this, we used CLP [7], a high quality open-soure LP solver. Beause the MN algorithm is designed to attak this problem diretly and CLP is a generi LP solver, CLP took dramatially longer than MN. Finally, we point out that our algorithm does not laim to have ompetitive running time with -means. It is well known that -means is onsiderably faster than -medians. The purpose of this paper is to demonstrate that if one wishes to use -medians beause of its outlier-resistant properties, it an be so adapted by using the MN algorithm. 4. Experiments In order to test the effetiveness of the MN algorithm in improving traditional -medians, we ompare four different versions of -medians:. MN at eah iteration. This is our MN iterative - medians algorithm as desribed in Setion MN only at the end. This is Simple Approah #2 (Setion 3.) with MN used as the normalization algorithm at the end. 3. Saling at eah iteration. This is Simple Approah # (Setion 3.): at eah iteration of the - medians algorithm, -norm saling is used to normalize the median. 4. Saling only at the end. This is Simple Approah #2 with -norm saling (Setion 3.) used as the normalization algorithm at the end. We also ompare these four algorithms with - means as a sanity hek. For all five tehniques, we hoose initial enters via the heuristi tehnique of hoosing the first point as the first enter, then hoosing eah suessive enter to be the point in the dataset whose distane to its losest enter is greatest. We terminate iteration of the algorithm when the error metri hanges by no more than.. For the two approahes that normalize only at the end, note that the error undergoes a sudden inrease after the last iteration. We test our lustering algorithms against three different data sets. We first disuss results from text douments that are lustered by word frequeny. We then move on to two mass spetral datasets, both representing aerosol partile data. The first of these two datasets onsists of a syntheti dataset generated by adding noise to seven atual partile mass spetra. The seond dataset orresponds to atual aerosol data taken from St. Louis in February of 24. For eah of these three datasets, we report our experiments using a single value for (number of lusters) for brevity. Our goal is to examine differenes in lustering error and onvergene properties, and we point out that the theoretial disussion above on the merits of these algorithms is independent of number of lusters. We typially hoose to be our best estimate as to the number of lusters atually in the data, sine this enables an easy san of the resulting lusters for orretness. Clustering error in these experiments is defined as the average Manhattan distane from eah point to its losest enter. 4. Text Douments Our text dataset was generated from a set of 55 sonnets from 5 different authors (Rossetti, Spenser, Browning, Sidney, and Shakespeare), where eah sonnet was represented as a single point by ounting word frequenies within that text [2]. We preproessed our data with the Porter stemming algorithm []. We lustered with = 5 for all algorithms. We ompare the lustering error from our four - medians variants in Figure 2. Both of the iterative normalization methods have the same general urve, but MN iterative normalization has lower error than saled iterative normalization. The saled iterative normalization error inreases and dereases slightly as it stabilizes, while MN iterative always dereases as explained in Setion 3.2. For the two ases where normalization is performed only at the end, the errors are idential until the very end of the algorithm beause these two lustering methods are exatly the same until the last pass. Notie, however, that after normalization saling gives a higher error than MN. Figure 2 also shows that for this dataset, normalizing with either tehnique at the end yields only slightly worse errors than normalization at eah iteration. This suggests that one an use this shortut of normalizing only at the end if a slight degradation in luster auray and a rapid inrease in lustering error at the end are aeptable. Figures 5-9 (grouped together at the end of the paper) represent the breakdown of eah luster by author for the four -medians algorithms and the spherial -means algorithm. Sine we know the author of eah sonnet, we an then measure how homogenous eah luster is with respet to the five authors. Figures 7 and 8, whih represent the lusters generated from the two algorithms that use saled normalization, show fairly poor results. Cluster 2 in Figure 7 and luster in Figure 8 eah ontain the bulk of the sonnets by all authors. MN iterative (Figure 5), on the other hand, does a muh better ob lustering by

7 Error author: lusters, 3, 4, and 5 are mostly homogenous. These results are onsiderably better than saled normalization. Additionally, though MN at the end has worse lusters than iterative MN (Figure 6 only has three deent lusters; 3, 4, and 5), it still performs better than saled normalization. Finally, when omparing iterative MN with spherial -means (Figure 9), we see that it performs at least as well. Note that we are unable to diretly ompare the lustering error from spherial -means to the error from the -medians algorithms in Figure 2 beause the distane metris are different. 4.2 Syntheti Partiles MN at eah iteration Saled at eah iteration MN only at the end Saled only at the end Iteration Figure 2: Error per iteration for the four - medians algorithms on the text data. Note that the algorithms involving saling onsistently perform worse than the algorithms using MN. The sale for the vertial axis has been hosen to render the results as learly as possible. We generated a syntheti dataset of mass spetra by starting with spetra from seven atual aerosol partiles. Based on these seven partiles, we reated 2 artifiial partiles by adding noise to the spetra of the seven partiles []. Sine there are seven known partiles, we lustered using = 7. Looking at the error versus the number of passes, we see the same trends as in the text data (Figure 3). MN outperforms saled normalization when performed either iteratively or at the end. There is one important differene to note between Figures 2 and 3. The final errors for iterative MN and MN at the end are idential in Figure 3. (Tehnially, iterative MN is 2.46 x -4 higher than MN at the end, but this is negligible due to possible rounding errors.) In Figure 2, however, iterative MN has visibly lower error. This again shows that our MN algorithm is a onsiderably better normalization tehnique than simple saling, and the user an hoose between using Error Iteration it at eah iteration or at the end depending on needs. The improvement that MN shows over simple saling is not as dramati in Figure 3 as it is in Figure 2. Note that we have hosen a sale for the vertial axis to render the results as learly as possible. Although we have shown theoretially that MN will always perform better than (or equal to) simple saling, how muh better MN performs depends on the partiular dataset. It is apparently the ase that for this syntheti dataset, simple saling is less error-prone than it is for our sonnets dataset. Nonetheless, in both examples, MN performs better. This is to be expeted, sine MN has been proven to be loally optimal. Figures -3 (grouped together at the end of the paper) show the luster distributions from these four algorithms. All four graphs show that eah luster is mostly homogeneous. Figure 4 shows the results from spherial -means lustering on this data. Three of the lusters in this graph (lusters, 3, and 5) show onsiderably less homogeneity. 4.3 St. Louis Dataset MN at eah iteration Saled at eah iteration MN only at the end Saled only at the end Figure 3: Error per iteration for the four - medians algorithms on the syntheti partiles. We finally present results from lustering a dataset of mass spetra orresponding to 2966 partiles olleted during February 24 in East St. Louis, Illinois at the EPA SuperSite loation. We used =9 whih seemed to provide easily interpretable results. Figure 4 shows error versus the number of passes for the four -medians algorithms, indiating behavior onsistent with the previous datasets. We have no knowledge of what the orret lusters are for this dataset. We did, however, examine the luster enters that resulted from these four - medians algorithms (author Prof. Gross is an atmospheri sientist austomed to examining suh plots). We observed that the MN iterative -medians algorithm piked up more peaks due to negative ions

8 than any of the other three -medians algorithms did. This set of luster enters is therefore more likely to be orret than the other three sets. The MN iterative - medians algorithm also showed onsiderably more peaks at loations with high m/z values (mass-toharge ratios) than any of the other algorithms did, whih also added to its redibility. Interestingly, the only other one of these four algorithms to show a luster enter with peaks at high m/z values was the saled iterative algorithm. We present two sample luster enters to illustrate these findings (Figure 5). 5. Conlusions and Future Work We have presented the MN algorithm, whih allows suessful use of -medians on normalized data when normalized luster enters are desired. This algorithm an be used during eah iteration of -medians or simply at the end of the algorithm, depending on whether one wants to emphasize non-inreasing lustering error or speed. We provided theoretial and experimental evidene that our algorithm is orret. Finally, to demonstrate experimentally the viability of our tehnique, we provided some brief omparisons with spherial -means. On our datasets, -medians ombined with MN yields lusters that are omparable to or better than those produed by spherial -means. Most of the salability issues for our algorithm are limited by the state of the art in saling traditional - medians. There is nonetheless room for future work in onsidering how to sale our algorithm, partiularly when the number of dimensions is high. Aknowledgements. This researh is supported by NSF ITR grant IIS and by Carleton College. 6. Referenes [] B. J. Anderson, D. R. Musiant, A. M. Ritz, A. Ault, D. S. Gross, M. Yuen, and M. Gaelli, "User-Friendly Error MN at eah iteration Saled at eah iteration MN only at the end Saled only at the end Iteration Figure 4: Error per iteration for the four - medians algorithms on the St. Louis data. Clustering for Atmospheri Data Analysis," Carleton College, Northfield, MN, Tehnial Report 5-a, 25. [2] E. Blomquist, Sonnet Central, 23, [3] P. S. Bradley and U. M. Fayyad, "Refining Initial Points for -Means Clustering," presented at Pro. 5th International Conf. on Mahine Learning, 998. [4] P. S. Bradley, O. L. Mangasarian, and W. N. Street, "Clustering via Conave Minimization," in Advanes in Neural Information Proessing Systems, vol. 9, M. C. Mozer, M. I. Jordan, and T. Petshe, Eds. Cambridge, MA: MIT Press, 997, pp [5] I. S. Dhillon, Y. Guan, and J. Fan, "Effiient Clustering of Very Large Doument Colletions," in Data Mining for Sientifi and Engineering Appliations, 2, pp [6] I. S. Dhillon and D. S. Modha, "Conept Deompositions for Large Sparse Text Data using Clustering," Mahine Learning, vol. 42, pp , 2. [7] J. Forrest, D. d. l. Nuez, and R. Lougee-Heimer, "CLP: COIN Linear Program Code,".2.2 ed, 25. [8] T. Hastie, R. Tibshirani, and J. H. Friedman, The Elements of Statistial Learning: Springer, 2. [9] A.. Jain and R. C. Dubes, Algorithms for Clustering Data: Prentie-Hall, 98. [] M. F. Porter, "An Algorithm for Suffix Stripping," Program, vol. 4, pp. 3-37, 98. [] D. T. Suess and. A. Prather, "Mass Spetrometry of Aerosols," Chemial Reviews, vol. 99, pp , 999.

9 # of Douments Figure 5: -medians with MN at eah iteration # of Douments Figure 6: -medians with MN only at the end # of Douments Figure 7: -medians saled at eah iteration # of Douments Figure 8: -medians saled only at the end # of Douments Figure 9: Spherial -means Rossetti Spenser # of Douments Browning Sidney Shakespeare I. Text Douments

10 # of Partiles Figure : -medians with MN at eah iteration # of Partiles Figure : -medians with MN only at the end # of Partiles Figure 2: -medians saled at eah iteration # of Partiles Figure 3: -medians saled only at the end # of Partiles Figure 4: Spherial -means Ambient Metals Ambient Brake Dust 7-3 Ambient 4 General 7 Ambient Smoke Clust er s Elemental Carbon Organi Carbon PAH II. Syntheti Partiles

11 a 27 b m/z m/z Figure 5: St. Louis luster samples. Corresponding aerosol partile mass spetra luster enters from -medians with MN at eah iteration (a) and -medians saled at eah iteration (b). Of the four -medians algorithms, only these two produed luster enters suh as these with peaks in the high m/z range. -medians with MN at eah iteration shows more of these peaks. Additionally, -medians with MN at eah iteration produes more luster enters with peaks in negative spetra. These harateristis are more representative of this data. III. St. Louis Data