Exploiting Enriched Contextual Information for Mobile App Classification

Transcription

1 Exploiting Enrihed Contextual Information for Mobile App Classifiation Hengshu Zhu 1 Huanhuan Cao 2 Enhong Chen 1 Hui Xiong 3 Jilei Tian 2 1 University of Siene and Tehnology of China 2 Nokia Researh Center 3 Rutgers University 1 {zhs, heneh}@ust.edu.n 2 {happia.ao, jilei.tian}@nokia.om 3 hxiong@rutgers.edu ABSTRACT A key step for the mobile app usage analysis is to lassify apps into some predefined ategories. However, it is a nontrivial task to effetively lassify mobile apps due to the limited ontextual information available for the analysis. To this end, in this paper, we propose an approah to first enrih the ontextual information of mobile apps by exploiting the additional Web knowledge from the Web searh engine. Then, inspired by the observation that different types of mobile apps may be relevant to different real-world ontexts, we also extrat some ontextual features for mobile apps from the ontext-rih devie logs of mobile users. Finally, we ombine all the enrihed ontextual information into a Maximum Entropy model for training a mobile app lassifier. The experimental results based on 443 mobile users devie logs learly show that our approah outperforms two state-of-the-art benhmark methods with a signifiant margin. Categories and Subjet Desriptors I.5.2 [Pattern Reognition]: Design Methodology-Feature evaluation and seletion, Classifier design and evaluation Keywords Automati mobile app lassifiation, Web knowledge, Realworld ontexts. 1. INTRODUCTION With the wide spread use of mobile devies in reent years, a huge number of mobile apps have been developed for mobile users. Indeed, mobile apps play an important role in the daily lives of mobile users. Intuitively, the study of the use of mobile apps an help to understand the user preferenes, and thus motivates many intelligent personalized servies, suh as app reommendation and target advertising [4, 6, 13]. However, the information diretly from mobile apps is usually very limited and ambiguous. For example, a user s Permission to make digital or hard opies of all or part of this work for personal or lassroom use is granted without fee provided that opies are not made or distributed for profit or ommerial advantage and that opies bear this notie and the full itation on the first page. To opy otherwise, to republish, to post on servers or to redistribute to lists, requires prior speifi permission and/or a fee. CIKM 12, Otober 29 November 2, 2012, Maui, HI, USA. Copyright 2012 ACM /12/10...$ preferene model may not fully understand the information the user usually plays Angry Birds unless the mobile app Angry Birds is reognized as a predefined app ategory Game/Stategy Game. Indeed, due to the large number and high inreasing speed of mobile apps, it is expeted to have an effetive and automati approah for mobile app lassifiation. Indeed, mobile app lassifiation is not a trivial task whih is still under development. The major hallenge is that there are not many effetive and expliit features available for lassifiation models due to the limited ontextual information of apps available for the analysis. To be speifi, there is limited ontextual information about mobile apps in their names, and the only available expliit features of mobile apps are the semanti words ontained in their names. However, these words are usually too short and sparse to reflet the relevane between mobile apps and partiular ategories. To this end, in this paper, we propose to leverage both Web knowledge and real-world ontexts for enrihing the ontextual information of apps, thus an improve the performane of mobile app lassifiation. Aording to some state-of-the-art works on short text lassifiation [3, 9, 10, 12], an effetive approah to enrihing the original few and sparse textual features is leveraging Web knowledge. Inspired with those works, we propose to take advantage of a Web searh engine to obtain some snippets to desribe a given mobile app for enrihing the textual features of the app. However, sometimes it may be diffiult to obtain suffiient Web knowledge for new or rarely used mobile apps. In this ase, the relevant real-world ontexts of mobile apps may be useful. Some observations from the reent studies [4, 6, 13] indiate that the app usage of a mobile user is usually ontext-aware. For example, business apps are likely used under the ontext like Loation: Work Plae, Profile: Meeting, while games are usually played under the ontext like Loation: Home, Is a holiday?: Yes. Compared with Web knowledge, the relevant real-world ontexts of new or rarely used mobile apps may be more available sine they an be obtained from the ontext-rih devie logs of the users who used them in mobile devies. Therefore, we also propose to leverage the relevant real-world ontexts of mobile apps to improve the performane of app lassifiation. To be speifi, the ontributions of this paper are summarized as follows. First, automati mobile app lassifiation is a novel problem whih is still under development. To the best of our knowledge, we are the first to leverage both Web knowledge 1617

2 and relevant real-world ontexts to enrih the limited ontextual information of mobile apps for solving this problem. Seond, we study and extrat several effetive features from both Web knowledge and real-world ontexts. Then, we propose to exploit the Maximum Entropy model (Max- Ent) [8] for ombining the effetive features to train a very effetive mobile app lassifier. Finally, to evaluate the proposed approah, we ondut extensive experiments on the ontext-rih mobile devie logs olleted from 443 mobile users, whih ontain 680 unique mobile apps and more than 8.8 million app usage reords. The experimental results learly show that our approah outperforms two state-of-the-art benhmark approahes with a signifiant margin. 2. OVERVIEW In this setion, we first introdue several related notions and give an overview of our mobile app lassifiation approah. Then we introdue the training model whih is used in our approah for app lassifiation. 2.1 Preliminary App Taxonomy. To reognize the semanti meanings of apps, we an lassify eah app to one or more ategories aording a predefined app taxonomy. To be speifi, an app taxonomy Υ is a tree of ategories where eah node orresponds to a predefined app ategory. The semanti meaning of eah app an be defined by the ategory labels along the path from the root to the orresponding nodes. Searh Snippets. In our approah, we propose to leverage the Web knowledge to enrih the textual information of apps. To be speifi, we first submit eah app name to a Web searh engine (e.g., Google or other app searh engines), and then obtain the searh snippets as the additional textual information of the orresponding app. A searh snippet is the abstrat of the web page whih are returned as relevant to the submitted searh query. The textual information in searh snippets is brief but an effetively summarize the orresponding web pages. Context Log. Smart mobile devies an apture the historial ontext data and the orresponding app usage reords of users through ontext-rih devie logs, or ontext logs for short. For example, Table 1 shows a toy ontext log whih ontains several ontext reords, and eah ontext reord onsists of a timestamp, the most detailed ontextual information at that time, and the orresponding user app usage reord aptured by the mobile devie. The ontextual information at a time point is represented by several ontextual features (e.g., Day name, Time range, and Loation) and their orresponding values (e.g., Saturday, AM8:00-9:00, and Home), whih an be annotated as ontextual feature-value pairs. Moreover, app usage reords an be empty (denoted as Null ) beause users do not always use apps. In Table 1, raw loation related data in ontext data, suh as GPS oordinates or ell IDs, have been transformed into semanti loations suh as Home and Work Plae by a loation mining approah (e.g., [7]). The basi idea of suh kinds of approah is to find the lusters of user positions and reognize their semanti meanings through a time pattern analysis. Given a target taxonomy Υ, an app a and a systemspeified parameter K, our approah inorporates the features extrated from both the relevant Web knowledge and ontextual information of a to lassify a into a ranked list of K ategories a 1, a 2,..., a K, among N leaf ategories { 1, 2,..., N } of Υ. 2.2 Training Model for App Classifiation Many lassifiation approahes, suh as Naive Bayes, SVMs and Maximum Entropy (MaxEnt) an be taken advantage of in our framework for app lassifiation. Among them, in this paper we propose to leverage MaxEnt for training a mobile app lassifier aording to the disussion in [9]. When we utilize the MaxEnt model to train app lassifiers, the most important work is to selet effetive feature funtions. Intuitively, given an app a and its ategory label, the basi features that an reflet the relevane between a and are the words ontained in the name of a. Suppose that the name of app a onsists of a set of words {wi a }, eah w a i an be onsidered as a relevant boolean feature to the ategory label. That is, the ourrene of a word w an be denoted as f(w) = 1 while vise versa we denote f(w) = 0. The weights of these features an be learned in the training proess of the MaxEnt model. A ritial problem of this type of features is that the lengths of app names are usually too short and the ontained words are very sparse. As a result, it is diffiult to train an effetive lassifier by only taking advantage of the words in app names. Moreover, the available training data are usually with limited size and may not over a suffiiently large set of words for refleting the relevane between apps and ategory labels. Therefore, a new app whose partial, or all words in name do not appear in the training data will not obtain lassifiation results if the lassifier is only based on the words in app names. To this end, we should also take onsideration of other effetive features whih an apture the relevane between apps and ategory labels. In the following two setions, we introdue the features extrated from both the relevant Web knowledge and ontextual information for training an app lassifier. 3. EXTRACTING WEB KNOWLEDGE BA- SED TEXTUAL FEATURES In this setion, we introdue how to leverage Web knowledge for extrating additional textual features of mobile apps. To be speifi, we investigate two suh kinds of textual features to apture the relevane between apps and the orresponding ategory labels, namely, expliit feedbak of Vetor Spae Models and impliit feedbak of semanti topis. 3.1 Expliit Feedbak of Vetor Spae Models This type of features exploits the top M results (i.e., searh snippets) returned by a Web searh engine by leveraging the expliit feedbak of Vetor Spae Models (VSM) [11]. Given an app a and its ategory label, we first submit a s name to a Web searh engine (e.g., Google API in our experiments). Then, for eah of the top M results, we map it to a ategory label in the app taxonomy Υ by building a Vetor Spae Model. There are three steps in the proess of mapping snippets to ategories by VSM. First, for eah app ategory, we integrate all top M snippets returned by a Web searh engine for some pre-seleted apps labeled with as a ategory profile d. Partiularly, we remove all stop words (e.g., of, the ) in d and normalize verb and adjetives (e.g., plays play, better good ). Seond, we build a 1618

3 Table 1: A toy ontext log from real-world data set. Timestamp Context App usage reords t 1 {(Day name: Monday),(Time range: AM8:00-9:00),(Profile: General),(Loation: Home)} Angry Birds t 2 {(Day name: Monday),(Time range: AM8:00-9:00),(Profile: General),(Loation: Moving)} Null t 3 {(Day name: Monday),(Time range: AM8:00-9:00),(Profile: General),(Loation: Moving)} Twitter... t 55 {(Day name: Monday),(Time range: AM8:00-9:00),(Profile: General),(Loation: Moving)} UC Web t 56 {(Day name: Monday),(Time range: AM8:00-9:00),(Profile: General),(Loation: Moving)} Null t 57 {(Day name: Monday),(Time range: AM8:00-9:00),(Profile: General),(Loation: Moving)} Musi Player normalized words vetor w = dim[n] for eah app ategory, where n indiates the number of all unique normalized words in all ategory profiles, dim[i] = freq i, (1 i n) i freq i, and freq i, indiates the frequeny of the i-th word in the ategory profile d. Finally, for eah snippet s returned for app a, we remove the words whih do not appear in any ategory profile and build its normalized word vetor w a,s in a similar way. By alulating the Cosine distane between w a,s and w, we map s to the ategory where = arg min Distane( w a,s, w ). Through the Vetor Spae Model, we an alulate a general label onfidene sore introdued in [3]: GConf(, a) = M,a M, (1) where M,a indiates the number of returned related searh snippets of a whose ategory labels are after mapping. Intuitively, the Gonf sore reflets the onfidene that a is labeled as gained from Web knowledge. The larger the sore, the higher the onfidene. However, sometimes Gonf sore may not aurately validate the relevane between and a due to the noise ategory labels ontained in the mapping list. In pratie, we find the more unique ategory labels ontained in the mapping list, the more unertainty for lassifiation. Therefore, we further define another sore named general label entropy to measure the unertainty of app lassifiation, whih an be alulated as follows: GEnt(, a) = i P ( i ) log P ( i ), (2) where P ( i ) = Gonf( i, a). 3.2 Impliit Feedbak of Semanti Topis Although the expliit feedbak of VSM an apture the relevane between app and ategory label in terms of the ourrenes of words, it does not take advantage of the latent semanti meaning behind words and may not work well in some ases. For example, in VSM, the following words Game, Play and Funny are treated as totally different measures to alulate the distane between word vetors. However, these words indeed have latent semanti relationships beause they an be ategorized into the same semanti topi Entertainment. Aording to [9], the latent semanti topis an improve the performane of short&sparse text lassifiation. Therefore, we study the textual features whih onsider the impliit feedbak of semanti topis. To be speifi, we propose to leverage the widely used Latent Dirihlet Alloation model (LDA) [1] for learning latent semanti topis. After learning latent topis, we extrat the feature based on the impliit feedbak of semanti topis as follows. Given an app a, we first leverage a Web searh engine to obtain the top M relevant snippets and remove the words whih do not appear in any ategory profile. Then, we map eah snippet s to a ategory by alulating the KL-divergene between their topi distributions as follows: D KL (P (z s) P (z )) = k P (z k s)ln P (z k s) P (z k ), (3) where P (z k ) = P (z k d ) and P (z k s) P (z) P (w s z) an be obtained through the LDA training proess. The ategory with the smallest KL-divergene will be seleted as the label of s, i.e., = arg min D KL (P (z s) P (z )). Finally, we alulate the topi onfidene sore for a given ategory as follows: T Conf(a, ) = T a, M, (4) where T a, indiates the number of returned snippets for a with the ategory label. Intuitively, the T onf sore reflets the onfidene that a is labeled as with respet to latent semanti topis. The larger the sore, the higher the onfidene. Moreover, we also alulate the topi based general label entropy in similar way aording to Equation EXTRACTING REAL-WORLD CONTEX- TUAL FEATURES In this setion, we introdue how to extrat effetive ontextual features of mobile apps from real-world ontext logs. To be speifi, we study two types of ontextual features, namely, pseudo feedbak from ontext vetors and frequent ontext patterns. 4.1 Pseudo Feedbak of Context Vetors The first type of ontextual feature onsiders the feedbak of ontext vetors. We assume that the usage of a partiular ategory of apps is relevant to some ontextual feature-value pairs. To be speifi, given the apps in the Game/Strategy Game ategory, they may be relevant to the ontextual feature-pairs (Day period: Evening) and (Loation: Home), respetively. Based on this assumption, similar to the VSM-based approah introdued in Setion 3.1, we an build a ontext vetor for eah app ategory. After building the ontext vetors for app ategories, we an take the feedbak of the pseudo ategory based on ontext similarity as a ontextual feature. To be speifi, given an app a and a ategory label, we first build the ontext vetor of a denoted as Ω a and then alulate the Cosine distane between Ω a and all app ategories ontext vetors. Finally, we rank ategory labels in asending order aording to their Cosine distane to Ω a. Partiularly, we define the pseudo ategory by = arg min Distane( Ω a, Ω ) and alulate the ategory 1619

4 rank distane by: CRDistane(a, ) = Rk() Rk( ) = Rk() 1, (5) where Rk() denotes the rank of ategory obtained by omparing vetor distanes to a. Intuitively, the CRDistane [0, N ), where N indiates the number of leaf nodes in the app taxonomy Υ. Obviously, the smaller the distane, the more likely the ategory label is the orret label. 4.2 Frequent Context Patterns When leveraging the pseudo feedbak of ontext vetors, we regard eah unique ontext feature-value pair as an independent measure for the relevane between ontext and the usage of a partiular ategory of apps. However, reently some researhers pointed out that some ontext featurevalue pairs are mutually related rather than separate elements and their o-ourrenes are relevant to app usage as well [2]. Along this line, we study a ontextual feature to apture the relevane between the o-ourrene of ontextual feature-value pairs and app usage. Speifially, we take advantage of the frequent ontext patterns for apps as follows. It is not a trivial work to mine these ontext patterns. As pointed out by Cao et al. [2], the amounts of ontext data and app usage reords are usually extremely unbalaned, whih makes it diffiult to mine suh ontext patterns through traditional assoiation rule mining approahes. Fortunately, some researhers have studied this problem and proposed some novel algorithms for mining suh ontext patterns. For example, Cao et al. [2] proposed a novel algorithm for mining suh ontext patterns, whih are referred to as behavior patterns in their work, by utilizing different ways of alulating supports and onfidenes. In an inremental work of [2], Li et al [5] proposed a more effiient algorithm named BP-Growth for this problem. In this paper, we leverage the BP-Growth algorithm for mining frequent ontext patterns. The basi idea of the algorithm is partitioning the original ontext logs into smaller sub-ontext logs for reduing the mining spae and mining frequent ontext patterns in these sub-ontext logs. After mining ontext patterns, we regard the ourrenes of relevant frequent ontext patterns as boolean features for determining a proper ategory label for a in a similar way of leveraging the words in app names. 5. EXPERIMENTAL RESULTS In this setion, we evaluate our approah through systemati empirial omparisons with two state-of-the-arts baselines on a real-world data set. 5.1 Experimental Set Up and Data Set The data set used in the experiments is olleted from many volunteers by a major manufaturer of smart mobile devies. The data set onsists of 443 ontext logs from different mobile users and total 8,852,187 ontext reords, whih ontains rih ontextual information and the usage reords of total 680 unique apps spanning for from several weeks to several months. In the experiments, we manually define a two-level app taxonomy based on the taxonomy of Nokia Store, whih ontains 9 level-1 ategories and 27 level-2 ategories. Table 2 shows the details of our predefined app taxonomy. Table 2: The predefined two-level taxonomy in our experiments. Level-1 Categories Internet Business Communiation Game Multimedia Navigation SNS System Referene Level-2 Categories Web Browser, Others Offie Tools, Seurity, Others Call, Mail&SMS, Others Ation, Strategy, Others Audio, Video, Others City Guides, Maps, Others IM, Blog&Forum, Others Management, Performane, Others News, Utility, Reading, Others We invited three human labelers who are familiar with smart mobile devies and apps to manually label the total 680 apps with the 27 level-2 ategory labels. For eah app, eah labeler gave the most appropriate ategory label by his (or her) own usage experiene (all the apps in the experiments an be downloaded through Nokia Store). The final label of eah app was voted by three labelers. Partiularly, for more than 95% apps, the three labelers gave the same labels. 5.2 Benhmark Methods and Evaluation Metris In this paper, we adopt two state-of-the-arts baselines to evaluate the performane of our approah. To the best of our knowledge, there is only one relevant approah has been reported in reent years, whih an be diretly leveraged for automati app lassifiation. Therefore, we leverage this approah as the first baseline. Word Vetor based App Classifier (WVAC) is introdued in [7], whih is adapted from the web query lassifiation approah proposed in [3] for app usage reord normalization. The seond baseline is originally developed for short text lassifiation, whih an be extended for lassifying apps. Hidden Topi based App Classifiation (HTAC) is introdued in [9], whose main idea is to learn hidden topis for enrihing original short&sparse texts. To be speifi, this approah adds semanti topis as additional textual features and integrate them with words for lassifying short&sparse texts. To redue the unertainty of splitting the data into training and test data, in the experiments we utilize a ten-fold ross validation to evaluate eah lassifiation approah. To evaluate the lassifiation performane of eah approah, we leverage three metris Overall Preision@K, Overall Reall@K and Overall F 1 Sore as introdued in [3]. 5.3 Overall Results and Analysis In order to study the ontribution of Web knowledge based textual features and ontextual features in our approah, we ompare four MaxEnt models with different features, namely, ME (MaxEnt with words), ME-T (MaxEnt with words + Web knowledge based Textual features), ME-C (Max- Ent with words + Contextual Features) and ME-T-C (Max- Ent with words + Web knowledge based Textual features + Contextual Features). Beause we treat the words in app names as basi features, all models take advantage of this kind of features by default. In our experiments, we hoose Google as our Web searh engine to obtain the relevant snippets of apps, and set the number of searh results M to be 10, whih equals to the number of searh results in one searh page. Eah searh 1620

5 ditional features in the MaxEnt model an ahieve the best performane. (a) () (b) Figure 1: The average overall (a) (b) and () F 1 sore of eah lassifiation approah with different K in the ten-fold ross validation. snippet is normalized by Stop-Words Remover 1 and Porter Stemmer 2. The number of latent topi K for both our approah and baseline HTAC are set to 20 aording to the estimation approah introdued in [14]. Two parameters α and β for training LDA model are set to be 50/K and 0.1. The settings of ontext pattern mining approah BP-Growth are similar to [5]. To avoid over fitting in the training proess of MaxEnt model, we also use Gaussian prior for parameter Λ similar to [8]. Moreover, both our approah and the baselines are implemented by standard C++ and the experiments are onduted on a 2.8GHZ 2 Dual-ore CPU, 2G main memory PC. Figure 1 (a), (b) and () ompares the average overall preision@k, reall@k and F 1 of two baseline methods WVAC, HTAC and our approahes with different features, namely, ME, ME-T, ME-C and ME-T-C in ten rounds of tests. From the above experiments, we an draw the onlusions as follows: 1) First, all other approahes outperform ME, whih implies the textual information in app names is insuffiient for lassifying apps effetively and leveraging additional features an improve the lassifiation performane dramatially. 2) Seond, the MaxEnt model with Web knowledge based textual features ME-T outperforms the two baselines WVAC and HTAC, whih indiates that the ombination of multiple Web knowledge based textual features and basi app name based features is more effetive than single Web knowledge based textual features for app lassifiation. 3) Third, the MaxEnt model with ontextual features ME-C also outperforms two baselines, whih validates the effetiveness of relevant ontexts for improving the app lassifiation performane. 4) Finally, the MaxEnt model whih ombines the Web knowledge based textual features and ontextual features outperforms both ME-T and ME-C, whih indiates the integration of two kinds of ad lager/mogul/porter-stemmer 6. CONCLUDING REMARKS In this paper, we proposed a novel approah for lassifying mobile apps by leveraging both Web knowledge and relevant real-world ontext. To be speifi, we first extrated several Web knowledge based textual features by taking advantage of a Web searh engine. Then, we also leveraged real-world ontext logs whih reord the usage of apps and orresponding ontexts to extrat relevant ontextual features. Finally, we integrated both types of features into a widely used Max- Ent model for training an app lassifier. The experiments on a real-world data set olleted from mobile users learly show that our approah outperforms two state-of-the-arts baselines. Aknowledgement. This work is supported by grants from Natural Siene Foundation of China (No ), Key Program of National Natural Siene Foundation of China (No ), National Major Speial Siene & Tehnology Projets (No. 2011ZX ), Researh Fund for the Dotoral Program of Higher Eduation of China (No ) and Nokia. 7. REFERENCES [1] D. M. Blei, A. Y. Ng, and M. I. Jordan. Lantent dirihlet alloation. Journal of Mahine Learning Researh, pages , [2] H. Cao, T. Bao, Q. Yang, E. Chen, and J. Tian. An effetive approah for mining mobile user habits. In CIKM 10, pages , [3] H. Cao, D. H. Hu, D. Shen, D. Jiang, J.-T. Sun, E. Chen, and Q. Yang. Context-aware query lassifiation. In SIGIR 09, pages 3 10, [4] M. Kahng, S. Lee, and S.-g. Lee. Ranking in ontext-aware reommender systems. In WWW 11, pages 65 66, [5] X. Li, H. Cao, H. Xiong, E. Chen, and J. Tian. Bp-growth: Searhing strategies for effiient behavior pattern mining. In MDM 12, [6] Q. Liu, Y. Ge, Z. Li, E. Chen, and H. Xiong. Personalized travel pakage reommendation. In ICDM 11, pages [7] H. Ma, H. Cao, Q. Yang, E. Chen, and J. Tian. A habit mining approah for disovering similar mobile users. In WWW 12, [8] K. Nigam. Using maximum entropy for text lassifiation. In In IJCAI-99 Workshop on Mahine Learning for Information Filtering, pages 61 67, [9] X.-H. Phan, C.-T. Nguyen, D.-T. Le, L.-M. Nguyen, S. Horiguhi, and Q.-T. Ha. A hidden topi-based framework toward building appliations with short web douments. IEEE Transations on Knowledge and Data Engineering, 23: , [10] M. Sahami and T. D. Heilman. A web-based kernel funtion for measuring the similarity of short text snippets. In WWW 06, pages , [11] G. Salton, A. Wong, and C. S. Yang. A vetor spae model for automati indexing. Commun. ACM, 18: , [12] D. Shen, J.-T. Sun, Q. Yang, and Z. Chen. Building bridges for web query lassifiation. In SIGIR 06, pages , [13] K. Yu, B. Zhang, H. Zhu, H. Cao, and J. Tian. Towards personalized ontext-aware reommendation by mining ontext logs through topi models. In PAKDD 12, [14] H. Zhu, H. Cao, H. Xiong, E. Chen, and J. Tian. Towards expert finding by leveraging relevant ategories in authority ranking. In CIKM 11, pages ,