Geo-activity Recommendations by using Improved Feature Combination

Transcription

1 Geo-activity Recoendations by using Iproved Feature Cobination Masoud Sattari Middle East Technical University Ankara, Turkey Murat Manguoglu Middle East Technical University Ankara, Turkey Isail H. Toroslu Middle East Technical University Ankara, Turkey Panagiotis Syeonidis Aristotle University Thessaloniki, Greece Pinar Senkul Middle East Technical University Ankara, Turkey Yannis Manolopoulos Aristotle University Thessaloniki, Greece ABSTRACT In this paper, we propose a new odel to integrate additional data, which is obtained fro geospatial resources other than original data set in order to iprove Location/Activity recoendations. The data set that is used in this work is a GPS trajectory of soe users, which is gathered over 2 years. In order to have ore accurate predictions and recoendations, we present a odel that injects additional inforation to the ain data set and we ai to apply a atheatical ethod on the erged data. On the erged data set, singular value decoposition technique is applied to extract latent relations. Several tests have been conducted, and the results of our proposed ethod are copared with a siilar work for the sae data set. Author Keywords Geospatial Recoendation, Matrix Factorization, Feature Cobination ACM Classification Keywords H.2.8 [Database Manageent] Database Applications data ining General Ters Algoriths, Experientation, Perforance INTRODUCTION With booing technology of sart obile phones and satellite-assisted positioning systes, new deands for applications in this field are eerging. One of these requireents that ost of the users are interested in is activity recoendation based on location (GPS) data, which is specially used to guide tourists and unfailiar individuals in tourist-attracting cities. Therefore, locationbased social networks (LBSN) and location-based recoendations have eerged as interesting research topics involving several diensions [9, 0, ]. In this work, we propose a ethod to iprove location-based recoendation by injecting additional data into the original data set. The additional data coes fro an Copyright is held by the author/owner(s). UbiCop 2, Septeber 5-8, 202, Pittsburgh, USA. ACM /2/09. external resource into the geospatial activity-location recoendation syste. Our work has actually been inspired fro a siilar earlier work of [8]. We also use the sae data set, including three atrices, naely location-activity atrix, location-feature atrix and activity-activity atrix. The forer atrix iplies preference ratings of people on a specific activity in a specific location. Its entries actually correspond to the frequency of perforing an activity in that location for all users. The second one contains available features of locations and finally, the last one represents relationships aong different activities. As expected, the first data set, naely location-activity atrix, is very sparse and we want to deterine the values of its unknown entries by utilizing the inforation in the other two atrices. Generally speaking, erging a data fro one resource (Activity- Activity and Location-Feature) with the data fro another resource (Location-Activity) is described as Feature Cobination [, 7] in the literature. Both [8] and this work apply feature cobination to cobine these three atrices and construct an integrated atrix. We apply Singular Value Decoposition (SVD) to uncover the latent relations within data. Basic difference and contribution of our work lies in the way integrated atrix is used for prediction and the application of SVD. At the end, we show the effectiveness of our approach by coparing it with the atrix copletion approach of [8]. Our experients show that, our ethod has higher prediction accuracy than the one in [8]. The rest of the paper has been organized as follows. In Section 2, we discuss related work and consider pros and cons of the. Section 3 explains the data odel and we introduce our ethod in Section 4. Evaluation ethods and results of our experients are available in Section 5. Finally, in Section 6 we have conclusion part of the paper. RELATED WORK Nowadays, ost of the recoender systes are used in e- coerce to help individuals in decision aking [4]. Since

2 social networks like Facebook and Google+ 2 have attracted illions of people, several algoriths have been designed to recoend friendship requests and advertiseents based on the geographical position of users [2]. Different recoender systes techniques have been proposed in literature and researchers have tried to cobine the to build better hybrid odels that ake use of these techniques [7]. Also, there are applications that track people and get their feedback to build a GPS based recoendation [8]. Soe works [9] add another attribute like user and odel the data with a 3 diensional tensor in order to have ore targeted recoendations using collaborative filtering techniques. Beside this, they use a odel-based ethod that benefits fro achine learning techniques, to predict issing values in entioned tensor. Siilarly, the work in [0] presents a obile recoendation syste that, in addition to applying tensor factorization to whole data set, incorporates 2 other algoriths to predict issing values using partial data set. The ai of our work, as well as the work of [8], can be described as a recoender syste that recoends activities for a location. In [8], this is handled by copleting the whole activity-location atrix. Thus, when there is a deand for aking a recoendation, easily the entry corresponding to that recoendation request can be reached in the copleted atrix and the recoendation can be ade. However, if that atrix is very large this approach ay not be feasible due to two reasons: even with a very effective ethod, atrix copletion ay take a lot of tie, and there could be a need for a very large storage to store the result. Specially, if the recoendation requests correspond to a sall portion of the whole atrix and they sparsely arrive, then responding to those requests on the fly ay be a better approach. In [8], the odel that is used to predict unknown values is called Collective Matrix Factorization, which was proposed by Singh [6]. Collective Matrix Factorization begins with construction of an objective function. Then, this function is converted to an optiization proble, whereas this task is done iteratively by a nuerical ethod that is known as Gradient Descent [5]. Note that this ethod finds local inia and does not guarantee to find the global inia. Practically, the ore local inia are near to global inia, the better the odel estiates the unknown values. Our work ais to tackle the proble of generating recoendations when they are needed. Therefore, diensionality reduction techniques appear as appropriate approach. In order to be able to deal with huge atrices, we use Singular Value Decoposition (SVD) [3] to generate low rank atrices, while injecting the additional inforation coing fro two other data resources into the ain activity-location atrix. Therefore, the ain contribution of our work is cobining ore than one atrix into a single structure. Then, to ake a recoendation we use only the ain data part of the low rank approxiation atrices that have been generated with SVD. DATA SET CHARACTERISTICS The data set that is used in this paper is gathered by Microsoft Research Asia. Pre-processed data set is available online and can be downloaded fro Microsoft Research Asia website 3. The data set is collected fro a web-based application over 2.5 years so that, each user is equipped with a GPS installed tool (GPS Navigator, sart phone) during visiting Beijing city in China. For each location that is visited, users ay insert coents about that place and possible 5 activities that are done in that location (food and drink, shopping, ovie and shows, sports and exercise, touris and auseent) thus, these coents with location info can be used to create a atrix called Location- Activity atrix, whose rows are locations and coluns are activities. This atrix consists of 67 locations and 5 activities that each entry of it denotes the frequency of perforing an activity for all users in that location. To ake it ore inforative, location-feature data extracted fro Point of Interest (POI) database, which is based on city s yellow pages, is also added to our odel. Usually in big cities for any given location area in the city this database gives the type and nuber of activities like cineas, restaurants, shopping centers, sport coplexes and so on. Gathered data can be odeled as a Location-Feature atrix whose entries are nonnegative integer values that for a given location fro available 67 locations shows the frequency of each 3 available features and vice versa [8]. As explained in [8], there is a statistical relation between activities. For exaple if a user goes to cinea s/he ay go to restaurant to eat food too. This is the kind of relationship that is aied to be captured as Activity Correlation. This inforation is also available in this data set as a 5-by-5 atrix, which is naed as Activity-Activity atrix whose entries are real values in interval [,-] and show the correlation between activities represented as the rows and coluns. The ai is ainly to ake activity recoendation to the users based on their spatial location. Since, the Location- Activity atrix is very sparse, it akes sense to use the additional inforation captured in Location-Feature and Activity-Activity atrices in order to ake ore accurate activity recoendations. That was the ain idea behind the work of [8]. Siply, the ai is to ake use of Location

3 Feature and Activity-Activity atrices to predict the issing entries of the Location-Activity atrix. OVERVIEW OF THE PROPOSED METHOD In this section we describe our proposed ethod to deterine the values of unknown entries of the Location- Activity sparse atrix. In the following sub-sections, we explain the procedure of the erging atrices, construction of the low-rank atrices and Activity-Location recoendation routine. We also describe the whole process through an exaple thoroughly. Feature Cobination and Low-Rank Matrix Construction As discussed in Section 3, we have erged ain atrix X (Location-Activity) with two additional atrices Y (Location-Feature) and Z (Activity-Activity) to construct an integrated atrix T. In order to preserve the structure of a atrix we have injected zero values in the rest of the null entries of T. ( ) ( ) () Then, we siply apply a well-known technique that is Singular Value Decoposition (SVD), to reveal the latent seantic indexing of data. The idea in SVD is to decopose a atrix T into three atrices U, S and V such that, U is left singular atrix, V is the right singular atrix and finally S contains singular values. In general, SVD is used for diensionality reduction so that, with selecting the top k values of S and k coluns of U and and by ultiplying the respectively, we can represent atrix T with reduced atrix which has rank of k ( ( )). For siplicity, in the rest of paper we show the reduced rank coponents of T with, and. Activity Recoendation Decoposing original atrix T reveals an interesting characteristic of and. Actually, using for a given activity we can easily find siilar locations that this activity is also done and using, for a given location we can find siilar activities that are done in that location. Moreover, cobining these two ay even lead to better results. In this step, to predict a frequency rating for a given location i and activity j, we search for siilar rows of i in and also for siilar coluns of j in. In order to have an accurate estiate for frequency rating, instead of selecting one siilar neighborhood, we pick the ost siilar rows in and also the ost siilar n coluns in. The following equations define these operations. (2) (3) ( ( )) (4) ( ( )) (5) After that, the average of and is used as the predicted rating value for an activity at a location. Notice that, since the row count of is ore than the nuber of actual locations (due to erge operation) to find siilar locations in we tri it so that, its rows corresponds to locations (l) only. This is done by selecting the first l rows of for the siilarity search. Siilarly, to find siilar activities in we tri it so that, its coluns corresponds to the activities (a) only. It is perfored by selecting last a coluns of for the siilarity search. Thus, equations (4) and (5) are applied to only these rows and coluns. In order to see influence of siilarity etrics, we have applied both Euclidean distance and Cosine siilarity to get siilarity atrices. On the basis of our experients, we have observed that, Cosine siilarity yields ore accurate results than Euclidean distance. Euclidean distance is defined as follows where, denotes the Euclidean nor of x. ( ) Exaple In this section we deonstrate a saple exaple of our ethod with a siplified data set. As we have already seen, is Location-Activity atrix, is Location-Feature atrix and is Activity-Activity correlation atrix. (6) (7) (8) (9) At the beginning, we cobine X, Y and Z atrices based on () to construct the integrated atrix T. In order to illustrate how our ethod works and how accurately it deterines soe issing values, we select 3 nonzero entries fro X randoly and change their values to 0. The selected entries are x, x 22 and x 33 which are bolded in T.

4 (0) index of ost siilar rows in descending order fro left to right. Siilarly, each colun in Si_index(V T ) shows the index of ost siilar coluns in descending order fro top to down. According to (2), SVD ethod is applied to T which yields atrices, and as follow: ( ) (9) ( ) (20) () (2) (3) Since we are interested in specific part of U and, we tri the so that, they eet the sae indices of X in T. In order to reduce the integrated atrix to rank 2, first 2 coluns of U, S and first 2 rows of are selected as shown in (4), (5) and (6) correspondingly. (4) (5) (6) To predict the value of in X we search for siilar rows of i in and siilar coluns of j in. Reeber that to copute siilarity atrix, we ade use of Cosine siilarity between vectors. Related siilarity atrices are presented in (7) and (8). ( ) (7) ( ) (8) Using these siilarity atrices we copute Si_index(U) and Si_index(V T ). Each row in Si_index (U) shows the As a final step, we are to predict the value of given x ij fro Location-Activity atrix X. We find the ean value of top (in this exaple is chosen to 3) siilar nonzero rows of i and ean value of top n (in this exaple n is chosen to ) siilar nonzero coluns of j incorporating (9) and (20). The estiated value of x ij is ean value of these 2 ters. Prediction steps for x, x 22 and x 33 are as follow. Top 3 siilar rows of row are 2, 5 and 3 with corresponding values of 0, 2 and 0 in colun. Since values of rows 2 and 3 are zero we put the aside and select next siilar rows which in this case is row 4 only. ( ) (2) According to first colun of (8), colun 2 is the ost siilar colun to colun with value of 3 in row. Thus, the predicted value for x is ean value of this value and the value that is calculated in (2). ( ) (22) In x 22, top 3 nonzero siilar rows of row 2 are, 3 and 4 with values of 3, 53 and 4 in colun 2. ( ) (23) Also, colun3 is the ost siilar nonzero colun of colun 2 with value of 7 in row 2. ( ) (24) Procedure for x 33 is sae as the previous entries thus, we just show the values. ( ) (25) ( ) (26) In order to show better coparison of predicted values with original values and siilar work in [8], we organized the at Table. Predicted Work in [8] Original x x x Table. Predicted value vs. work in [8] and original value

5 MAE EXPERIMENTS In this section we explain the evaluation ethod and afterwards, present experiental results of our work. Finally we copare the results with siilar works. Evaluation Method In order to easure the accuracy of our approach, wellknown k-fold cross-validation ethod is used to partition the whole data set into a training data set and a validation data set. To be able to apply this validation technique we have assued that our data set is accurate. Then, we have set ( ) of nonzero entries of the Location-Activity atrix to zero. Afterwards, we have applied our approach on the data set to predict the values of those entries. The k results fro the folds then can be averaged (or otherwise cobined) to produce a single estiation. In order to copare the results nuerically, we have used Root Mean Squared Error () and Mean Absolute Error (MAE) which both easure difference between observed values (original values) and estiated values. ( ) (27) ( ) ( ) (28) Where, O and E are observed value and estiated value correspondingly. We have copared our results with a state-of-the-art work that is studied by Zheng et al. in [8]. Experiental Results In this section we show the experiental results obtained fro the proposed odel for prediction without applying abstraction ethod and with abstraction ethod. Evaluation without Abstraction As we discussed in previous subsection, in order to prepare training and validation data, we have used k-fold crossvalidation. As it is used usually, we put k = 0 that is, in each fold of execution, we select 0% of nonzero entries in location- activity atrix randoly and put the as validation set, reaining part is the training set and it is used to construct the prediction odel Round Proposed ethod Work in [8] Figure. MAE values for proposed work vs. work in [8] As discussed in Section 3, by using additional data, we ipleent the proposed ethod to predict the rating value of all entries in location-activity atrix. Since for validation data we have both observed value and predicted value, we can calculate and MAE for this fold. Therefore, in each loop we acquire a value for and MAE. At the end of last loop we su up results for all folds and calculate the ean value of corresponding error ters. Obviously, all steps of 0-fold cross-validation were also applied to recoendation ethod in [8]. Each colun in Figure shows the ean value of MAE for 0 folds. In order to have a coprehensive view of results we run the application for 0 ties and put the ean value of MAE in a new colun. Figure 2 shows the sae coparison of for our proposed ethod and work in [8]. Like MAE, we run the application for 0 ties that each colun shows ean value of for 0 folds Round Proposed ethod Work in [8] Figure 2. values for proposed work vs. work in [8] Evaluation with Abstraction As entioned in [8], there are always places that are rated ore than other places hence, they tend to have an abnorally greater rating values coparing to other places that are visited frequently but, does not receive uch ratings. Beside this, coent adding is an optional case that users are requested to do it and they usually do not provide so any useful coents. In this data set which has 2,765 GPS trajectories 62 users have participated and a total nuber of 530 coents are collected and based on the previous discussion, ost of the are related to soe faous places and final Location-Activity atrix becoes very sparse. An explicit drawback of this sparseness is that, the interval between axiu and iniu values is largely extended and oreover, the available frequencies are distributed around specific points within the interval. Having such a large interval aong the values of Location- Activity atrix causes two probles. One is related with the interpretation of these values. Soe values are very large, in the range of a few hundreds, and on the other hand, there are also so any values less than 00. It is obvious that the forer group corresponds to strong recoendation. However, without knowing the whole distribution, it is not very easy to deterine whether the values in the latter group correspond to weak recoendation or natural. The second proble is

6 MAE related with the error calculation. When the range of values is very large and they are not clustered, their ipact in error calculations will be related to their actual values. If there is an activity in a location with frequency 250, and if a syste predicts a value such as 300, then this will contribute soe error. However, both of these values actually correspond to strong recoendation. Thus, rather than using these actual values, it would be better if abstract and discrete values are used in a sall range. To alleviate the ipact of this proble on final evaluations, we propose an abstraction ethod and calculate error ters to this ethod respectively. In this abstraction ethod, we partition the nonzero values of Location-Activity atrix to 5 clusters using k-eans clustering algorith. In this algorith, 5 rando points are selected as initial ean values randoly within the ratings values then, each point (rating value in this case) is assigned to a cluster according to shortest value of Euclidean distance fro each ean values. In this step for each cluster, ean value is calculated and assigning points to each cluster is perfored iteratively, until no point assigned to a new cluster. In this abstraction ethod, instead of working on rating values we have exained the cluster that each value belongs to. Note that, the clusters are ranked in a single diension. Instead of coputing error between original value and predicted value, we find the distance between the clusters that original value belongs to and the cluster that predicted value falls into it. As in previous ethod, we start this ethod by applying 0-fold cross-validation but in each fold, instead of keeping the predicted value, we find the cluster for this value. Also, for the siilar work in [8], each predicted values fall into one of the clusters. In the last step, we use cluster nubers to calculate the error ters. In our experients, we have clustered values to 5 clusters. In order to clarify, this procedure is deonstrated for the exaple in previous section. The nonzero values of atrix X in (7) are clustered to 3 clusters using k-eans and corresponding clusters are: { } { } { } (29) According to (29), validation data x =, x 22 =7 and x 33 =53 belongs to C, C and C2 and regarding to Table the predicted values for the are 9.5, 8.5 and 43.6 which indicates that new clusters for predicted values are C, C and C2. As an exaple, let s assue that for a given validation data x ij the original cluster is C, our predicted value falls into cluster C3 and the predicted value in work [8] falls into C2 thus, the MAE error can be calculated consequently as for proposed ethod and for siilar work in [8]. Figure 3 shows the results of MAE when we apply the abstraction technique. Siilar to the previous ethod each colun shows ean value of MAE for 0 folds. We run application for 0 ties to show a coprehensive view of results Round Figure 3. MAE values for proposed work vs. work in [8] with applying abstraction Finally, Figure 4 shows the ean value of for 0 folds in one colun. For the sae reason the application is run for 0 ties and each colun shows ean value of in each tie of execution Round Proposed ethod Work in [8] Proposed ethod Work in [8] Figure 4. values for proposed work vs. work in [8] with applying abstraction Paraeter Optiization Since each data set has its own characteristics, it is not possible to have one set of fixed paraeters that works for all. In our work, we have paraeterized possible nuber of top siilar locations and top siilar activities. Due to the fact that value of paraeters and n, which denote the ost siilar values of rows and coluns affect the errors, we have probed different values of these paraeters to find the optial values that reduce the. For each of and n we choose values fro to 5 thus, we have obtained 25 cobinations of the that can be organized in 5 diagras. We have also ipleented it for without abstraction and with abstraction as discussed in previous subsections.

7 n= n= n= n=2 (a) n= (b) n=2 (a) n= (b) n= n= n=3 (c) n=3 (c) n= n= n= n= n=5 (d) n=4 (e) n=5 Figure 5. vs. paraeter without abstraction As illustrated in Figure 5, values of = and n=3 when results are not abstracted lead to iniu. Siilarly, Figure. 6 shows that =4 and n=4 are the best values when the abstraction is applied, that gives the iniu. As shown in Figure 5, in this data set for a fixed n, increases up to =3, and then it starts to decrease. However, for even larger values of it does not reach to the level for =. Therefore, = sees like the ost appropriate choice. Although for different values of n slightly different results are obtained, except for they are quite close to each other. Moreover, aong the n=3 gives the sallest values. On the other hand, when the abstraction is used, for a fixed n, shows rather fluctuated behavior for increasing. In all the cases we have tested, =4 gives the sallest value, and siilar to the previous case except for, for all other n values results are very close to each other, and n=4 is the sallest aong the. Considering the size of Location-Activity atrix (67 by 5), searching for siilar entries ore than 5 would not be feasible. It is not possible to interpret why these values produce the best values, but, it is quite clear that it is directly dependent to the data set (the atrix). (d) n=4 (e) n=5 Figure 6. vs. paraeter with abstraction CONCLUSION In this paper, a new approach has been proposed for cobining additional inforation into ain sparse data set for aking recoendation. This idea has been applied to geo-spatial data set for activity-location recoendation syste. Activity correlation data and location feature data has also been added into the syste in order to iprove the accuracy for predicting the issing values of the sparse location-activity data set. The sae proble has already been investigated in [8]. Unlike that work, which aied to coplete the whole atrix, we have used low-rank approxiation approach of SVD in order to reply recoendation requests when they arrive. Since the original atrix has been reduced to saller atrices, its eory requireent and construction ties are uch less than the ethod of 7. Furtherore, we use a different ethod for prediction, which ais to ake prediction only cell-wise. This leads to further tie efficiency. Indeed, both approaches have their own pros and cons. In large data sets with low recoendation requests our ethod is ore applicable. Moreover, through soe experients, we have also shown that the accuracy values of our approach are better than the one obtained in [8]. ACKNOWLEDGMENTS This work has been partially funded by the Greek GSRT (project nuber 0TUR/4-3-3) and the Turkish TUBITAK (project nuber 09E282) national agencies as part of Greek-Turkey bilateral scientific cooperation.

8 REFERENCES. Burke, R. D. Hybrid web recoender systes. In P. Brusilovsky, A. Kobsa, and W. Nejdl, editors, The Adaptive Web, Methods and Strategies of Web Personalization, volue 432 of Lecture Notes in Coputer Science, Springer, Huang, Q. and Liu, Y. On geo-social network services. In Proc. of the 7th IEEE Int. Conf. on Geoinforatics, Fairfax (VA, USA), Lax, P. Linear Algebra and its Applications, Wiley, Linden, G., Sith, B. and York, J. Aazon.co recoendations: Ite-to-ite collaborative filtering. IEEE Internet Coputing, 7(), Nocedal, J. and Wright, S. J. Nuerical Optiization. Springer, Singh, A. P. and Gordon, G. J. Relational learning via collective atrix factorization. In KDD 08: Proc. of the 4th ACM SIGKDD Intl. Conf. on Knowledge Discovery & Data Mining, New York, NY, USA: ACM Spiegel, S., Kunegis, J. and Li, F. Hydra: A Hybrid Recoender Syste [Cross-Linked Rating and Content Inforation]. In CNIKM '09: Proceeding of the st ACM international workshop on Coplex networks eet inforation; knowledge anageent (2009), Zheng, V. W., Zheng, Y., Xie, X. and Yang, Q. Collaborative location and activity recoendations with GPS history data. In WWW 0: Proc. of the 9 th International World Wide Web Conference. New York, NY, USA: ACM. 9. Zheng, V. W., Cao, B., Zheng, Y., Xie, X. and Yang, Q. Collaborative Filtering Meets Mobile Recoendation: A User-centered Approach. In AAAI 200. ACM, Zheng, V. W., Zheng, Y., Xie, X. and Yang, Q. Towards obile intelligence: Learning fro GPS history data for collaborative recoendation. In Artificial Intelligence Journal (AIJ), (202) Zheng, Y. Location-based social networks: Users. In Coputing with Spatial Trajectories, Zheng, Y. and Zhou, X. Eds. Springer, 20