Descriptive Data Mining Modeling in Telecom Systems

Transcription

1 Descriptive Data Miig Modelig i Telecom Systems Ivo Pejaović, Zora Sočir, Damir Medved 2 Faculty of Electrical Egieerig ad Computig, Uiversity of Zagreb Usa 3, HR-0000 Zagreb, Croatia Tel: ; ; Fax: s: ivo.pejaovic@fer.hr, zora.socir@fer.hr 2 Croatia Telecom Ic. Hebragova 32-34, HR-000 Zagreb, Croatia Tel: ; Fax: damir.medved@ht.hr Abstract: Croatia Telecom possesses huge amouts of data which has bee collected for various purposes. The pilot project was deployed to get ew isights about the distributio of errors ad iterfereces i Croatia Telecom s etwor. This paper describes the usage of descriptive data miig modelig for gettig valuable isights from the observatioal data, the results of the aalysis ad possible beefits from these results. The special emphasis was give to the usage of probabilistic model-based clusterig ad o EM algorithm which is used for estimatio of parameters of the model.. INTRODUCTION Telecom operators are amog the biggest geerators of data ad the adoptio of ew ways of explorig ad aalyzig data i telecom systems is very importat. These ew ways of aalyzig captured ad stored data should provide ew isights about various problems which ca be foud i telecom operators busiess. Data miig aalysis begis to play a sigificat role i the aalysis of telecom s data ad there are umerous emergig data miig applicatios i telecommuicatios []. Data miig ca be defied as the aalysis of (ofte very large) observatioal data sets to fid ususpected relatioships ad to summarize the data i ovel ways that are both uderstadable ad useful to the data ower. Over the time, umerous data miig techiques have bee developed for aalyzig data which ca be grouped as follows: exploratory data aalysis (EDA), descriptive modelig, predictive modelig, fidig patters ad rules, retrieval by cotet [2]. Croatia Telecom has bee collectig various data for a log time ad for various purposes. Today it possesses huge amouts of data which has bee origially collected for purposes other tha data miig aalysis. This paper describes pilot project of usig selected umber of data miig techiques for the aalysis of the telecom s data ad possible beefits of their usage. The importace of this project is that this is oe of the first data miig projects i Croatia Telecom which ca pave the way to the other similar projects. The data that was used for the aalysis was the data about the errors ad iterfereces i Croatia Telecom s etwor. The goal wated to be achieved was gettig ew isights about the distributio of errors ad iterfereces i the part of the telecom s etwor. Due to the ature of the aalyzed data ad the goal wated to be achieved we used descriptive data miig modelig ad EDA techiques for fidig iterestig patters ad atural groupigs withi the data. The paper describes the aalysis of the observatioal telecom s data based o probabilistic model-based clusterig usig mixture models ad especially, models based o EM (Expectatio-Maximizatio) algorithm. This paper is structured as follows: Sectio 2 describes descriptive modelig ad EDA techiques, Sectio 3 describes the idea of mixture models ad EM algorithm that is used for modelig, Sectio 4 describes the data used for the aalysis, Sectio 5 describes the aalysis ad results of the aalysis ad coclusio i Sectio 6 is followed by the referece list. 2. DESCRIPTIVE MODELING AND EDA The goal of descriptive modelig is to build a model which describes all of the data. Examples of such descriptios iclude models for the overall probability distributio of the data (desity estimatio), partitioig of the p-dimesioal space ito groups (cluster aalysis ad segmetatio) ad models describig the relatioships betwee variables (depedecy modelig) [2]. I cluster aalysis the aim is to discover atural groups i data, for example, i telecom operators databases or scietific databases. I cluster aalysis ad segmetatio there is o right umber of clusters or segmets. It is usually chose by researcher o the basis of some objective or subjective criterio.

2 Figure The map of the aalyzed area Exploratory data aalysis (EDA) is the ame for group of techiques where the goal is simply to explore data without ay clear idea what we are looig for. Typically, EDA techiques are iteractive ad visual. There are may effective graphical display methods for relatively small, lowdimesioal data sets. As the dimesioality (umber of variables, p) icreases, it becomes much more difficult to visualize the cloud of poits i p-space. For dimesios above 3 or 4 EDA techiques are ot very useful. The examples of EDA techiques are various histograms, scatterplots, cotour plots etc. 3. EM ALGORITHM FOR MIXTURE MODELS A multivariate radom variable X is a set X,,X p of p radom variables ad x = {x,,x p } deotes a set of values for X. The geeral form of a mixture distributio for multivariate x is give with [2]: K f ( x) = π f ( x; θ ) () = Where π is the probability that a observatio will come from the -th compoet, K is the umber of compoets, f (x;θ ) is the distributio of the -th compoet ad θ is the vector of parameters describig the -th compoet. The mixture models provide a geeral framewor for clusterig i a probabilistic cotext. I the literature this is referred to as probabilistic model-based clusterig sice there is a assumed probability model for each compoet cluster. I this framewor it is assumed that data come from a multivariate fiite mixture model that is described with () ad every cluster is described by oe compoet of a mixture model. Roughly speaig, the geeral procedure is as follows: give a observatioal data set of observatios D = {x(),,x()}, determie how may clusters K we wat to fit to the data, choose parametric models for each of this K clusters (for example multivariate Normal distributios, expoetial distributios etc.) ad the use EM algorithm to determie the compoet parameters θ ad compoet probabilities π from the data. Oce mixture decompositio has bee foud, the data ca be assiged to the clusters by assigig each poit to the cluster from which it is most liely to have come. The properties of the EM algorithm are described i details i [2], [3], [4], [5], [6], [7]. EM algorithm is a iterative algorithm for estimatig the parameters of the parametric models as is the case with mixture models. It cosists of two steps ad geerally ca be described as follows: E-step calculatio of the expected cluster probabilities. This is the first step ad it is called Expectatio. M-step calculatio of the distributio parameters through maximizatio of the lielihood of the distributios give the data. This step is called Maximizatio. Cosiderig the observatioal data set D, idepedetly sampled from the same distributio f(x θ) where θ is a set of model parameters, we ca defie the lielihood fuctio L(θ D) as the probability that data would have arise, for a give value of θ, regarded as a fuctio of θ [2]:

3 L( θ D) = L( θ x(),..., x( )) = = p( x(),..., x( ) θ) = i= f ( x( i) θ) The lielihood fuctio is used as a measure of how well some parametric model fits the data. The lielihood computatio is simply the multiplicatio of the sum of the probabilities for each of the istaces. For example, with two clusters A ad B cotaiig istaces from D with cluster probabilities π A ad π B the computatio of L(θ D) is: (2) Therefore, a error or iterferece is represeted as set [X,Y,MSMLO]. The preparatio of data for modelig icluded removig of oise ad icosistecies from the data. The data cotaied 0.2% of oise ad wrog iputs, ad they were dropt from the data set for buildig a model. The whole process of preparig the data for the aalysis is described i [9]. ( θ D) = [ πap( x A) + πbp( x B) ] [ πap( x2 A) + πbp( x2 B) ]... [ π p( x A) + π p( x B) ] L A B (3) 4. UNDERSTANDING THE DATA The aalysis was doe o the data about errors ad iterfereces i the part of Croatia Telecom s etwor that covers area of TK ceter Zagreb (Telecommuicatio ceter Zagreb). The aalyzed data was collected by Croatia Telecom from December 999 to the ed of May The data set cotaied iformatio about more tha registrated errors ad iterfereces. The collected data cotaied various attributes that describe errors ad iterfereces etered i the database. Amog these various attributes we were especially iterested i the attributes that describe geographical locatio of the errors ad iterfereces ad the attribute that describes the ature (or id) of the errors ad iterfereces. Figure shows the map of the whole aalyzed area. The spots o the map represet places of registered iterfereces ad errors. The attributes that describe geographical locatio of the errors are Gauss-Kruger coordiates. These attributes are labeled with X ad Y ad represet distace i meters related to the origi for this zoe (the fifth zoe) of Gauss-Kruger projectio system [8]. The attribute X correspods to the logitude ad attribute Y correspods to the latitude. The values of these attributes are represeted as real umbers. Figures 2 ad 3 show the distributio of values for the attributes X ad Y respectively. The attribute that describes the ature of errors ad iterfereces is labeled with MSMLO. It is categorical attribute ad may have 66 differet values that describe the ature or id of a error or iterferece. For example, it may be a error o the cable of specific id (optical, shielded, ushielded), o PCM (Pulse Code Modulatio), o telephoe lie etc. Figure 2 - Distributio of values for attribute X Figure 3 - Distributio of values for attribute Y 5. THE ANALYSIS The aalysis was aimed at discoverig atural groups withi collected data. Cosiderig the distributio of the coordiates of errors that is show i Figures 2 ad 3, it ca be cocluded that it ca be best described as the mixture model with multivariate Normal distributios. Actually, durig the aalysis we have tried few differet models based o differet clusterig algorithms (for example -meas [2], Oracle s o-cluster [0] etc.) ad it has bee show that the model based o mixture model with multivariate Normal distributio fits the observed data the best [9]. For modelig it has bee used the implemetatio of EM algorithm cotaied i Wea ope-source data miig tool []. This implemetatio of EM algorithm hadles both umerical ad categorical attributes. Supposed distributios of umerical attributes are mixture Normal distributios.

4 Figure 4 Distributio of clusters After the preparig process it has bee used modelig with EM algorithm for fidig the model s parameters. The model has bee built o the values of set [X,Y,MSMLO]. So, resultig clusters from model have three dimesios. Dimesios X ad Y determie geographical locatio of the clusters while dimesio MSMLO gives distributio of the errors ad iterfereces. The fial umber of clusters was chose comparig the values of the lielihood fuctio (2) for the differet umber of clusters K. The lielihood fuctio reached the maximum for the umber of 9 clusters. The parameters that had eeded to be estimated were meas, compoet (cluster) probabilities ad stadard deviatios. The values of these parameters ca be foud i [9]. Every cluster correspods to oe compoet of the give mixture model. Figure 4 shows geographic represetatio of the clusters (oly geographic dimesios of the clusters are show). Every cluster is represeted with spots of a differet color ad every cluster has its umber. The clusters have various size ad shape, probability (that is proportioal to the umber of errors ad iterfereces covered by them) ad stadard deviatio. For example cluster 3, which covers the cetral part of Zagreb tow, has the highest cluster probability 58%, which meas that it covers 58% of all registered errors etc. [9]. Distributio of the attribute MSMLO is very differet for each of these clusters eve for very ear clusters (i geographical sese). Comparig the distributios of the attribute MSMLO for differet clusters very iterestig patters ca be oticed. For example, Figures 5 ad 6 show the distributios of the attribute MSMLO of the clusters that cover area of Velia Gorica, clusters 0 ad 7 i Figure 5. Figure 5 Distributio of attribute MSMLO for cluster 0 Although these two clusters are very ear to each other, we ca see sigificat differeces i distributio of errors ad iterfereces i these two clusters. For example, the secod ad third bar from the left i Figure 5 represet frequecies of errors ad iterfereces o two ids of cables. Together, they tae more tha 30% of all errors ad iterfereces i the cluster 0. I Figure 6 it ca be oticed that frequecy of errors o the cables i the cluster 7 taes oly 5% of all errors ad iterfereces. It could be very iterestig to see what is the reaso for that. Deeper uderstadig ad fidig out the reasos of these patters is ot the tas of descriptive modelig ad it is usually o the domai experts (i this case, egieers resposible for etwor maagemet ad etwor cotrol).

5 Figure 6 Distributio of attribute MSMLO for cluster 7 The same observatios ca be made for ay pair of clusters ad ay type of errors or iterfereces. Furthermore, owig these distributios of clusters, aalysis ca be repeated o the part of the aalyzed area that is covered with oly few of the clusters. These results give more detailed picture of what is goig o i the part of the aalyzed area that caot be see i the aalysis of the whole area. For example, if we were iterested i what is goig o i Zagreb tow the aalysis should be repeated o the area covered with clusters ad 3, which cover the regio of Zagreb tow ad surroudig places. The umber of clusters i that area ca be also estimated cosiderig the values of the lielihood fuctio for the estimated parameters of the mixture model ad umber of clusters. By doig that we ca mae a hierarchy of clusters each havig its view o the problem. Clusters o the top of hierarchy have more geeral view of what is goig o tha clusters below them. The overall probability of the occurrece of the errors ad iterfereces is govered by may factors: desity of the telecom ifrastructure, id of the equipmet, umber of users, exteral iterfereces etc. So it is quite possible that some areas will have very differet distributios of errors ad iterfereces due to some specific factor (or factors). Explorig these distributios could result i discoverig valuable iformatio. Fidig areas with the uusual distributios of errors ad iterfereces may lead to further ivestigatio of the reasos their existece. This ca be accomplished by ivestigatio o the groud ad/or usig some other techiques of data miig aalysis (for example predictive modelig). Usig predictive modelig techiques requires more detailed data about telecom ifrastructure i specific area ad exteral iterfereces. The fial result ca be the discovery of reasos of some types of errors ad iterfereces. Oce the reasos of errors ad iterfereces are ow, some of them ca be removed. 6. CONCLUSION I descriptive modelig the validatio of the results is always o the side of the domai experts. Discovered patters ad atural groupigs withi the aalyzed data reveals a lot iformatio about distributios of errors ad iterfereces of specific id i the areas related to the specific clusters. Some of the distributios that have bee foud may already be ow to the domai experts but some of them may ot. Discoverig these patters i the aalyzed data ca improve the quality of service that operator offers to its customers (for example, by elimiatig some reasos of the iterfereces o the telephoe lies etc.). Descriptive data miig modelig techiques have bee used to mae the model which describes the data. EDA techiques have bee used to visualize the results of modelig. Usig data miig techiques offers the possibility of fidig the ew ad valuable iformatio that may otherwise be lost. This iformatio ca be used for various purposes i telecommuicatios: improvig the quality of service, Busiess Itelligece systems, CRM (Customer Relatioship Maagemet) applicatios [2] etc. REFERENCES [] Jaiwei Ha, Russ B. Altma, Vipi Kumar, Heii Maila, Daryl Pregibo: Emergig Scietific Applicatios i Data Miig, Commuicatios of ACM 45, 8 (August 2002), [2] David Had, Heii Maila, Padhraic Smyth: Priciples of Data Miig, MIT Press, 200. [3] A.P.Dempster, N.M. Laird, ad D.B. Rubi: Maximumlielihood from icomplete data via the em algorithm, J. Royal Statist. Soc. Ser. B., 39, 977. [4] R. Reder ad H. Waler: Mixture desities, maximum lielihood ad the em algorithm, SIAM Review, 26(2), 984. [5] Z. Ghahramami ad M. Jorda: Learig from icomplete data, Techical Report AI Lab Memo No. 509, CBCL Paper No. 08, MIT AI Lab, August 995. [6] M. Jorda ad R. Jacobs: Hierarchical mixtures of experts ad the em algorithm, Neural Computatio, 6:8 24, 994. [7] C.F.J. Wu: O the covergece properties of the em algorithm, The Aals of Statistics, ():95 03, 983. [8] URL: fd50projectios_tutorial.htm [9] Pejaovic, Ivo: Data Miig Methods i Busiess Itelligece Systems (i Croatia), Master thesis, Faculty of Electrical Egieerig ad Computig, Zagreb, [0] URL: [] URL: [2] Michael J. A. Berry, Gordo S. Lioff: Masterig Data Miig The Art ad Siece of Customer Relatioship Maagemet, Joh Wiley & Sos Ic., 2000.