User-Oriented Context Suggestion

Transcription

1 User-Oriented Context Suggestion Yong Zheng Center for Web Intelligence DePaul University Chicago, Illinois, USA Bamshad Mobasher Center for Web Intelligence DePaul University Chicago, Illinois, USA Robin Burke Center for Web Intelligence DePaul University Chicago, Illinois, USA ABSTRACT Recommender systems have been used in many domains to assist users decision making by providing item recommendations and thereby reducing information overload. Contextaware recommender systems go further, incorporating the variability of users preferences across contexts, and suggesting items that are appropriate in different contexts. In this paper, we present a novel recommendation task, Context Suggestion, whereby the system recommends contexts in which items may be selected. We introduce the motivations behind the notion of context suggestion and discuss several potential solutions. In particular, we focus specifically on user-oriented context suggestion which involves recommending appropriate contexts based on a user s profile. We propose extensions of well-known context-aware recommendation algorithms such as tensor factorization and deviation-based contextual modeling and adapt them as methods to recommend contexts instead of items. In our empirical evaluation, we compare the proposed solutions to several baseline algorithms using four real-world data sets. Categories and Subject Descriptors H.3.3 [Information Search and Retrieval]: Information filtering Keywords Recommendation; Context; Context Suggestion, Contextaware Recommendation 1. INTRODUCTION AND MOTIVATION Recommender systems (RS) are effective at alleviating information overload by tailoring recommendations to users personal preferences. Context-aware recommender systems Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. UMAP 16, July 13-17, 2016, Halifax, NS, Canada Copyright 2016 ACM /16/07... $ (CARS) have the additional goal of adapting those recommendations to specific contexts in which users will select or use recommended items. CARS take contextual factors into account in modeling user profiles and in generating recommendations. In CARS, context is generally defined as any information that can be used to characterize the situation of entities [10]. We believe the context information usually refers to the dynamic attributes which may change when the same activity (e.g., watching a movie, listening to a music, etc) is performed repeatedly [24], such as the scenarios of the activities (such as time, location, companion) and dynamic factors from users (such as emotional states). In rating-based application domains, such as those involving movie or book ratings, the standard formulation of the recommendation problem begins with a two dimensional matrix of ratings, organized by user and item: Users Items Ratings. The key insight of context-aware recommender systems is that users preferences for items may be also a function of the context in which those items are encountered. Incorporating contexts requires that we estimate user preferences using a multidimensional rating function R: Users Items Contexts Ratings [1]. Both traditional RS and CARS are designed to provide item recommendations. However, incorporating the notion of context into the recommendation process, enables such systems to also recommend contexts, themselves, to users. There are many situations where recommending an appropriate context may be important and beneficial to users. For example, users may need context suggestions in a movie domain, including When, Where and with Whom to watch a specific movie. In a music streaming application, the best context in which a particular song should be played (such as the type of activity or time of the day) may be useful information in selecting the song. Similarly, in the travel domain, the desire to visit a particular destination may vary depending on the season. The goal of Context Suggestion is to recommend a list of appropriate contexts to users in order to optimize their experience when consuming the items (e.g., watching a movie, listening to music, enjoying a trip). There are at least three ways that context suggestion can be useful: Recommending the right context at the appropriate time may lead to a better user experience. User experience refers to a person s emotions and attitudes about using a particular product, system or service. It

2 is not always sufficient to suggest a set of items to a user based on his or her preferences. The right item should be delivered Motivations to the right person at the right time and in the right place. As shown by Figure 1, users may have different experiences if they are going to watch 1). To maximize theuser movie experience Life of PI (UX) with different companions in UX refers to a person's different emotions locations. and attitudes Knowingabout the context using a particular in which this product, system or movie service. can be enjoyed the most may affect the user s decision in selecting it during a particular interaction. Figure 2: Context Suggestion in Google Music Partners at Cinema Family at Home Partners at Swimming Pool Figure 1: Different User Experience in Movie domain Context suggestion can derive the context acquisition 27 process used in context-aware systems. Context information can be collected explicitly (such as through user surveys) or implicitly (such as through Web or mobile activity and usage logs). These approaches have well-known disadvantages: user surveys require additional human efforts, and usage logs only present limited context information, such as time and location. Alternatively, through context suggestion, a predefined list of contexts can be ranked and recommended to users, and the user s taste in different contexts can be inferred from their interactions with the suggested contexts. For example, in Google Music (see Figure 2), the system is able to suggest a list of pre-defined contexts (e.g., workout, study, hanging out) to the user. Note that there is a mutually reinforcing relationship between the notions of context acquisition and context suggestion. There are many predefined context categories in Google Music, but only a limited number of them can be presented to a user. A context recommender is needed to rank the list of contexts based on a variety of factors, including possibly, the user s past behavior and interests. On the other hand, the acquired context information from users can in turn be used for model learning in the future. Besides, the context suggestion in this example also shares something in common with the topic of activity recommendation [8]. We interpret the activities to be recommended as users intents which fall into the notion of contexts in our case. In addition, not only the activities, but also the attributes of the activity, such as time, location and companion can be suggested, which reveals another difference between context suggestion and activity recommendation. Context suggestions can also be viewed as the process for enabling context-aware recommendation. As the example of Google Music suggests, the user s choice on the contexts can be viewed as an explicit query to the system effectively providing a constraint on the choice of items that can be recommended. Therefore, music recommendation in the selected context can be formulated as a context-aware recommendation task. This approach allows systems to integrate context suggestion and context-aware recommendation in order to further assist user s decision making. In this paper, we focus specifically on user-oriented context suggestion which involves recommending appropriate contexts based on a user s profile. We propose extensions of well-known context-aware recommendation algorithms such as tensor factorization and deviation-based contextual modeling and adapt them as methods to recommend contexts instead of items. We introduce the related work in Section 2 and the specific problem setting in Section 3. Our evaluation methodologies are presented in Section 4, followed by the experimental results in Section 5, comparing the proposed solutions to several baseline algorithms using four real-world data sets. 2. RELATED WORK A Context-aware recommender system (CARS) [1] aims to adapt to users preference models to different contextual situations. The context information may include time, location, companion, emotions, occasion, and so forth. In recent years, context-aware recommendation has proved effective in many applications, and several context-aware recommendation algorithms have been developed, including algorithms based on collaborative filtering [26], contextaware matrix factorization [6] and tensor factorization [12]. By contrast, context suggestion recommends the appropriate contexts to users in order to improve their experience with a given set of items. The idea of recommending context was first mentioned in the tutorial [2] on contextaware recommender systems by Adomavicius and Tuzhilin in 2008, where context recommendation was presented as a potentially new recommendation opportunity. Since then, there have been few research efforts exploring the notion of context suggestion. Table 1: Uses of Context Suggestion Context Suggestion Context Suggestion As Explanations Bundle Suggestion Inputs user item user, item user item user, item user, item Outputs a list of contexts a list of contexts a list of contexts items + contexts users + contexts contexts + items contexts + users

3 In earlier work, we have discussed different uses of context suggestion [23] which are summarized in Table 1. We briefly describe these uses below. context suggestion, for the purpose of recommending an appropriate context, always has a list of contexts as output, but the inputs could be a single user, a single item, or a user-item pair resulting in different recommendation tasks. For example, we may suggest an appropriate time (day, season, etc.) for a user to go on a vacation. Or, we may suggest the best season for tourists to visit Alaska, in which case the item (i.e., Alaska) is the input. If in addition, the suggestions are customized to the user based on his or her interests, then both user and item are considered inputs. The first attempt to solve this problem was made by Baltrunas, et al. [5], where the goal was to predict the best contexts for users listening to a specific music track. They proposed different K- nearest neighbor classifiers to predict the contexts. In our previous work, we comprehensively explored the use of multi-label classification techniques as the basis for personalized context suggestions [28]. Context suggestion as explanations provides a finer-grained way to recommend a combination of appropriate contexts together with a list of users or items. For example, the system may recommend the user to listen to a list of songs during a workout. The suggested contexts can be viewed as the explanations for why the system recommends this item to the user. Context suggestions could be used to recommend both contexts and items in the form of a bundle suggestion. For example, when a user is browsing items on an e-commerce Web site, the system may suggest an appropriate occasion (e.g., Mother s Day, birthdays, etc.) for users to gift a specific item. Similarly, the system can simultaneously provide other recommended items which are appropriate as gifts for the same specific occasions. These applications can be derived from two basic recommendation tasks. One task is UI-Oriented Context Suggestion which recommends contexts given a user-item pair and it has been explored by our previous work [28]. For example, what is the suggested time and location if a user is going to watch Life of PI. Another recommendation task is User-Oriented Context Suggestion in which the suggested context for each user is used as a way to constrain the space of recommended items. This latter task is similar to what s used in Google Music (see Figure 2), Pandora, and Youtube. Currently, Google Music and Pandora only use time as a constraint to rank contexts. For example, Wake up is only recommended to users in early mornings, Focus is suggested to the users during the working period (such as 9AM to 5PM), and Relaxing at home is recommended after work (such as 6 PM to 11 PM). In this paper, we present and analyze finer-grained methodologies for useroriented context suggestion going beyond simple solutions using only one factor as constraint. Another possible task could be Item-Oriented Context Suggestion, but we specifically explore User-Oriented Context Suggestion in this paper, since user personalization is expected to be more significant in this task. 3. PROBLEM SETTING For user-oriented context suggestion, we use the same problem setting as commonly used for the context-aware item recommendation tasks. Accordingly, we use similar data sets used in training in context-aware recommendation models. A sample of a context-aware movie ratings data set is shown in the table below. Table 2: Sample of Context-aware Movie Data Set User Item Rating Time Location U1 T1 3 Weekend Home U1 T2 4 Weekend Cinema U1 T1 5 Weekday Cinema U2 T2 3 Weekday Cinema U2 T3 4 Weekday Home U2 T4 5 Weekend Home We use contextual dimension to denote a contextual variable, such as Time or Location. The term contextual condition refers to a specific value in a dimension, e.g. Weekday and Weekend are two conditions for Time. In the table above, there are six rating profiles given by two users on four movies within different contextual situations. In user-oriented context suggestion, the system recommends a list of appropriate contextual conditions to each user, similar to the Google Music example earlier. Without user s explicit preferences on context conditions, it is difficult to evaluate the quality and accuracy of our suggested contexts. There are two alternative solutions: one is to use the usage frequency to indicate how a user likes or dislikes a context condition. In Table 2, user U1 may prefer to watch a movie on the weekend and in the cinema rather than at home or on the weekday, since, U1 has two rating profiles at weekend and in cinema, respectively. Note that we ignore the combinations of contexts and directly recommend individual context conditions in the task of useroriented context suggestion. However, most context-aware data sets are collected from user surveys, and subjects may be asked to rate items in pre-defined contextual situations, resulting in potentially unreliable data for evaluation. Table 3: User-Context Rating Data Matrix Weekend Weekday Home Cinema U U Another solution is to use the user s average item rating in each context to represent user s preferences on contexts. In other words, the multidimensional context-aware data matrix shown in Table 2 can be converted to a 2-dimensional user-context rating matrix (UC rating matrix) as shown in Table 3, where the value in each cell is obtained by the average rating given by each user on multiple items in each specific context condition, and the item information is

4 eliminated from the rating matrix. Note that using users average rating in each context may also result in unreliable evaluations, if the number of ratings by a user on a context condition is limited. We plan to collect a real-world data with users explicit preferences on context in our future work. Given the fact that we do not have such a data with users explicit preferences on contexts, we use the 2- dimensional UC rating matrix as the ground truth in our experimental evaluations. 4. EXPERIMENTAL APPROACHES We split a context-aware data set (e.g., data matrix shown in Table 2) into training and testing data sets with multidimensional context information. We then convert these data sets into 2-dimensional training and testing matrices (e.g., UC rating matrix shown by Table 3). Next, we propose to build different recommenders to suggest contexts first using the original multidimensional training set, and then using the derived 2-dimensional testing set for evaluation purpose. Namely, the rating data in the 2-dimensional test set is used as the ground truth for evaluation purposes. In the remaining part of this section, we present several baseline algorithms for context suggestion and then introduce two novel approaches based on traditional context-aware recommendation models. 4.1 Baselines As noted previously, we use user s average rating in context conditions to represent the ground truth. Therefore, the most straightforward methodology is to reuse this strategy to build recommenders for context suggestion. Note that these baseline approaches will only use the derived 2- dimensional training set (based on the original multidimensional training set) to build models and perform the evaluations based on the derived 2-dimensional testing set. More specifically, we build three approaches as baselines for our evaluations: Context Average The predicted score for a user u on context condition c can be obtained by the average rating given in c from the 2-dimensional training data set (i.e., UC rating matrix). This is a non-personalized method which produces the same ranking of contexts to each user based on how popular the contexts are as inferred from the UC rating matrix. Of course, this approach may not be very effective if the number of rating profiles in a particular context condition is limited User-Context Average A finer-grained approach is to create a user-personalized recommender based on user s average item ratings in each context. In other words, the rating given by a user u on context condition c in the 2-dimensional training data set can be directly used as the predicted score for u on context c in the testing data set. If u did not rate items in c in the training set, we are able to downgrade the prediction in other ways: we use the average rating in c, and adopt the average rating by u as an alternative way if c is a cold-start condition. This approach suffers from data sparsity too the prediction may introduce biases if the number of rating profiles given by u in context c is limited Recommenders Based on UC Rating Matrix To alleviate the data sparsity problem in the two approaches above, we can apply a traditional recommender to the 2- dimensional training set (i.e., UC rating matrix). In this paper, we use the biased matrix factorization introduced by Koren et al. [13], using the prediction function in Equation 1. r uc = µ + b u + b c + p T u q c (1) r uc is our predicted score for user u s preference on c. It is broken down into four components: global average rating µ, user bias b u, context bias b c, and user-context interaction p T u q c which presents the dot product of a user vector p u and a context vector q c. The algorithm assigns N latent factors for each user and context vector. p u can describe how u likes these factors, while q c denotes how c obtains these factors. Note that the three approaches introduced above are built upon the 2-dimensional training set which is represented by a UC rating matrix. Next, we present our approaches to the context suggestion problem. 4.2 By Contextual Rating Deviations In addition to building a recommender based on the 2- dimen-sional UC rating matrix, we can reuse the outputs of context-aware recommendation algorithms built on the multidimensional context-aware data sets. More specifically, we introduce the contextual rating deviations (CRDs) which can be obtained by deviation-based contextual modeling approaches in context-aware recommendation. CRDs represent how a user s rating is different in each context condition. The idea of context rating deviation has been incorporated into matrix factorization [6] and sparse linear method [29] as the underlying approach to building contextaware recommendation algorithms. In context-aware matrix factorization (CAMF) for example, the predictions for two variants CAMF C and CAMF CU can be described by Equation 2 and 3 respectively. N r uic1 c 2...c N = µ + b u + b i + CRD(c j) + p T u q i (2) r uic1 c 2...c N = µ + j=1 N CRD(c j, u) + b i + p T u q i (3) j=1 Assume there are N context dimensions, and c j (j = 1, 2,..., N) represents the context conditions in the j th context dimension. In CAMF C, the prediction score for u on item i in context c 1c 2...c N, r uic1 c 2...c N, is composed of five components: global average rating, user bias, item bias, user-item interaction and the aggregated context rating deviations CRD(c j). The intuition behind extending this approach to the problem of context suggestion is that we can apply the CAMF C model to the multidimensional training set and obtain the deviation values of CRD(c j) for various context conditions. The different CRD values can tell us how users prefer

5 each context condition generally. For example, if CRD in weekend is 1 and CRD in weekday is 0.3, this deviation might indicate that users prefer to watch movies on weekends rather than during the weekday. Therefore, the ranked list of context conditions based on those CRD values can be viewed as the list of contexts suggested to users. Note, the outputs by CAMF C provide a non-personalized context suggestion. By contrast, CAMF CU is able to learn a CRD value for each user in each context condition which is denoted by CRD(c j, u) in Equation 3. Thus, we can generate a userpersonalized context suggestion based on the CRD values for each user in each context condition. The drawback in this model is that we cannot obtain the CRD(c j, u) if u did not rate items in context c j. In this case, we learn a CAMF C model first, and use CRD(c j) as a substitution. The CRD values can also be obtained by the deviationbased contextual sparse linear method (CSLIM) [29, 30]: CSLIM C and CSLIM CU. In a summary, we reuse the outputs in context-aware recommendation algorithms to rank the context conditions for user-oriented context suggestion. In this way, the quality of context suggestion may rely on the performance of selected deviation-based contextual modeling approaches, i.e., how good the CRDs are that have been learned in using the specific modeling approach. 4.3 By UI-Oriented Context Suggestion Recall that the task in UI-oriented context suggestion is to recommend a list of contexts given a user-item pair. Table 5: Predictions By UI-Oriented Context Suggestion User Item Weekend Weekday Home Cinema U1 T U2 T U1 T U2 T In UI-oriented context suggestion, we are able to learn a predictive model based on the multidimensional training set and make predictions based on each user-item pair in the multidimensional testing set. An example of such predictions is depicted in Table 5 where the numerical values denote the predicted score for each context condition given by a user on a specific item. Table 6: User-Context Predictions Weekend Weekday Home Cinema U U We are able to aggregate the outputs in Table 5 to obtain the user s average ratings on each context condition as shown in Table 6. Recall that we generate the outputs in Table 5 based on the user-item pairs in the testing set for evaluation purposes. In other words, we first use a solution in UIoriented context suggestion to provide context predictions for each pair of users and items in the testing set, and then aggregate a list of context suggestions for each user. The aggregation is simplely a process of averaging a user s predicted score on each context conditions over all the items he or she rated in the testing set. This approach could be quite useful in practice in a variety of situations. For example, we may promote a list of items to a user, and then use this approach to generate a list of suggested contexts for that user. In earlier work, we evaluated different multilabel classification (MLC) [20] techniques as the solution for UI-oriented context suggestion [28], and found that Label Powerset () method [21] is the best performing MLC approach for this task. considers each unique set of context conditions as one of the classes of a new single-label classification task, and tries to learn the best unique set of contexts as the prediction by learning from features. Simply, we use user and item ids, as well as a binary value as features, where the binary value is used to indicate relevance, and it can be obtained by setting a rating threshold in the data. However, it works effectively at the price of computational costs if there are many rating profiles or several context conditions. In this paper, we explore pairwise interaction tensor factorization () as a new solution for UI-oriented context suggestion. [19] was originally developed as a solution for personalized tag recommendation. Here, we view the UIoriented context suggestion as a task of tag recommendation, where each context condition is considered as an individual tag to be recommended. We create three dimensions in the : user, item and context. All the context conditions are put into a single dimension so that can be applied directly to provide context suggestions for each user-item pair. Then, we are able to aggregate all the predictions to generate a list of context suggestions for each user, as shown in Table 6. The quality of context suggestion generated in this way may rely on the performance of solutions for UIoriented context suggestion. 5. EVALUATIONS AND RESULTS In this section, we present our data sets, evaluation protocols, experimental results and our findings. 5.1 Data Sets As noted before, we try to reuse the data sets in the contextaware recommendation task. Note that the number of data sets is very limited in this domain, and most of these data sets were collected from surveys, which results in small and sparse data sets. We choose data sets in four domains: restaurant, music, movie and mobile applications, as shown in Table 4. Restaurant data [18] is a data set collected from survey. Subjects gave ratings to the popular restaurants in Tijuana, Mexico by considering two contextual variables: time and location. Music data [4] was collected from InCarMusic which is an Android mobile application offering music recommendations to the passengers of a car. Users are requested to enter ratings for some items using a web application. There are 8 contextual factors and 34 contextual conditions in total.

6 Table 4: Descriptions of Multidimensional Context-aware Data Sets Restaurant Music LDOS-CoMoDa Frappe # of users # of items # of ratings 2,309 3, ,580 rating scale Raw frequency rating sparsity 9.62E E E E-04 # of context dimensions # of context conditions context dimensions Time, Location DrivingStyle, Landscape, Mood, NaturalPhenomena, RoadType, Sleepiness, TrafficConditions, Weather time, location, dayofweek, mood, dominantemo, endemo Time of the day, Day of the week, Location LDOS-CoMoDa data [14] is a publicly available contextaware movie data collected from surveys. There are originally 12 context dimensions which captured users various situations, and we use a subset of contexts which are proved to be influential ones in this data [17], including time, location, day of the week, and three emotional variables. Frappe data [3] comes from the mobile usage in the app named as Frappe which is a context-aware app discovery tool that will recommend the right apps for the right moment. We used 3 context dimensions for experimental evaluations, including time of the day, day of the week and location. This data captures the usage frequencies of an app by each user within 2 months. We employ a log transformation on the raw frequency numbers. And this data is the only large data set for public research in context-aware recommendation. The rating sparsity in Table 4 is obtained by Equation 4. D i denotes the size of each dimension, such as the number of users, the number of items, or the number of conditions in a specific context dimension. The music and LDOS-CoMoDa rating data are highly sparse since there are more context conditions than the other two data sets. Number of total ratings Sparsity = (4) i Di 5.2 Evaluation Protocols We apply a 5-fold cross validation strategy on these four data sets, i.e., each fold of data is viewed as a testing set, and the other four folds are used as training set. As mentioned in Section 3, for each fold evaluation, we have four groups of data: multidimensional training and testing sets, and the derived 2-dimensional training and testing sets (i.e., UC rating matrices). We use three baseline approaches described in Section 4.1: Context Average, User-context Average and biased matrix factorization which are denoted by, and respectively in Figure 3. In addition, we also evaluate the solutions based on deviation-based contextual modeling introduced in Section 4.2. More specifically, we evaluate four approaches: CAMF C and CSLIM C which provides a non-personalized context suggestion based on contextual rating deviations in each context condition, CAMF CU and CSLIM CU which suggest personalized contexts based on contextual rating deviations by each user on a specific context condition. Furthermore, we also add the solution based on (see Section 4.3) as another comparison. Note that we only use the relevant or positive profiles in. More specifically, in the 2-dimensional UC rating matrix, we only retain the rating profiles when its rating is no less than a threshold when we perform the algorithm. We set this threshold as 3 for the three rating-based data sets, and the mean of logged frequency value (i.e., 0.981) in the Frappe data. Furthermore, we add multilabel classification as another baseline to be compared with approach. We use label powerset () as the MLC algorithm and choose Random Forest [15] as the classification algorithm. We use the same rating threshold adopted in to segment the rating profiles to relevant and irrelevant ones. For CAMF and CSLIM algorithms, we use an open-source context-aware recommendation library, CARSKit [31] 1, to obtain the contextual rating deviations. For, we use the toolkit 2 provided by the authors [19]. In addition, we present data statistics (see Table 7) based on the 2-dimensional training sets (i.e., UC rating matrix) in order to get insights about how reliable our baseline approaches may work on these data. The rating density is the number of total ratings divided by the number of users and the number of context conditions. The rating density by context is the average number of ratings given in each condition divided by the number of total ratings. We can see that the rating density in restaurant data is 93.4% which tells that most users have given ratings to each context condition in this data. Therefore, the predictions by on this data may be more reliable than the ones in other data sets. Accordingly, the rating density by context in music and LDOS-CoMoDa data is only around 3.0% which may indicate the predictions by in these data is not as reliable as others. We evaluate the proposed approaches based on top-n recommendation. Five and ten are the most popular choices for the value of N. We use 5 since the number of context conditions is limited in the data sets. The choice of N may also depend on the specific applications. For example, a smaller number, e.g., 2 or 4, may be appropriate to be adopted in mobile applications, since the display space on mobile screen is limited. We choose two series of evaluation 1 CARSKit, 2,

7 Restaurant Restaurant U Precision@5 Recall@5 U NDCG Frappe MRR Frappe U Precision@ U Precision@5 U MRR LDOS-CoMoDa NDCG Recall@5 LDOS-CoMoDa Recall@5 U NDCG Music MRR Music U Precision@5 Recall@5 U NDCG MRR Figure 3: User-Oriented Context Suggestion: Experimental Evaluations Table 7: Data Description of UC Rating Matrix # of users # of conditions # of ratings rating density rating density by context Restaurant Music % % LDOSCoMoDa % 14.3% 2.9% 3.0% Frappe % user u has a gain gui from being recommended an item i, the average Discounted Cumulative Gain (DCG) for a list of J items is defined as shown in Equation 5. DCG = N J gui j 1 XX N u=1 j=1 max(1, logb j)) (5) 8.3% NDCG is the normalized version of DCG given by Equation 6, where DCG is the maximum possible DCG. metrics: relevance metrics, including precision and recall for top-5 context suggestion, and ranking metrics, including Normalized Discounted Cumulative Gain (NDCG) [11] and Mean Reciprocal Rank (MRR) [22]. More specifically, precision is calculated as the ratio of relevant items selected to the number of items recommended, and recall presents the probability that a relevant item will be selected. NDCG is a measure from information retrieval, where positions are discounted logarithmically. Assuming each N DCG = DCG DCG (6) The MRR is the average of the Reciprocal Rank (RR) across all the recommendation lists for individual users. RR measures how early in the list (i.e. how highly ranked) is the first relevant recommended item. MRR can be calculated base on Equation 7, where L denotes the relevant items

8 in the testing set for each user, and Rank i indicates the position of the i th relevant item in the recommendation list. MRR = 1 L L i=1 1 Rank i (7) 5.3 Experimental Results The experimental results can be described by Figure 3. The figures on the left side present precision and recall results for top-5 context suggestion, and the figures on the right side present results based on NDCG and MRR. Among the three baseline approaches, generally works better than and in precision, recall and MRR. Its advantage is more significant on the Music data than the ones in other data sets., as a non-personalized solution, only works sightly better than in the Frappe data in terms of precision, recall and MRR. Note that the rating density by context is only 2.9%, 3.0% and 8.3% (see Table 7) in the Music, LDOS-CoMoDa and Frappe data respectively, which may introduce biases in these baseline approaches, especially the and. Generally, is able to outperform the based on the results shown in the figure, which indicates that personalization is required in the user-oriented context suggestion task. Among the deviation-based contextual modeling approaches, we can see that CSLIM algorithms are able to obtain more accurate results in contextual rating deviations than the CAMF algorithms. That, in turn, results in better context suggestions in terms of both relevant and ranking metrics. This is not surprising since our previous work [29] has demonstrated that CSLIM outperforms CAMF in the task of context-aware recommendation. We also observe that user-personalized approaches work better than the non-personalized ones, even if the advantage is not that significant in some data sets. But we do find some evidences, for example, CSLIM CU performs better than CSLIM C in Frappe data in terms of precision and recall. CSLIM CU works better than CSLIM C in LDOS-CoMoDa data in terms of MRR, and CAMF CU works better than CAMF C in the music data in both relevance and ranking metrics. This finding confirms that user-personalization is required in the task of context suggestion. In addition, we also reuse the outputs in the UI-oriented context suggestion task to aggregate a list of suggested contexts to each user. is the one using label powerset as the multilabel classification solution in this case, and is the one we proposed in this paper which views context suggestion task being analogy to tag recommendations. We can see that generally outperforms the baseline approaches, especially the simple approach based on matrix factorization, which shows consistent results with our previous work [25]. The approach also works better than deviation-based approaches in the music and LDOS- CoMoDa data in terms of precision and ranking metrics. It also shows better ranking results in the restaurant data. is shown to be the best performing algorithm among all the examined approaches in both relevance and ranking metrics. And runs much more effectively than, especially when it comes to larger data set, such as the Frappe data. It is reasonable to see these results, since is shown to work better than the approach in the UIoriented context suggestion task. One possible explanation to interpret why and algorithms work better than deviation-based approaches is that they predict a score for each context condition towards a pair of user and item, and then aggregate the score across all the user-item pairs, which is exactly how we generate the ground truth in Section 3. As a summary, deviation-based approaches are able to outperform the baselines, and CSLIM is a better way to learn contextual rating deviations than CAMF. It is not surprising since SLIM [16] is an algorithm specifically designed for top-n recommendation and it was demonstrated to outperform the several state-of-the-art algorithms in traditional recommender systems. Note SLIM is not a learning-to-rank algorithm which directly optimizes ranking metrics. Meanwhile, reusing the outputs by UI-oriented context suggestion is viewed as the most effective approach. is able to work better than the deviation-based approach in terms of ranking metrics in some data sets. is proved to be the best performing solution among all these examined algorithms in both relevance and ranking metrics. Its advantage is significant in terms of improvement compared with other approaches, and it is viewed as an effective and efficient solution finally. 6. CONCLUSIONS AND FUTURE WORK In this paper, we have explored several solutions for useroriented context suggestion which aims to recommend a list of appropriate contexts to a given user. Based on our experimental evaluations, we find that the Pairwise Interactive Tensor Factorization adapted to context suggestion is the best solution for learning to rank contextual conditions. We also explored approaches based on contextual rating deviations and found that the extension of the Contextual Sparse Linear Method (CSLIM) is able to generate more accurate contextual rating deviations than the approaches based on Context-Aware Matrix Factorization. As introduced previously, Google Music currently uses time as constraint to filter or rank the context conditions. This strategy can also be further applied as a post-processing on the context suggestions by our approaches. The approaches explored in this work are general enough to be applied in any scenario, and they can be combined with other constraints beyond the temporal factors used in Google Music. Context suggestion is still a novel and promising research direction. The main challenge in current stage are the ground truth and evaluations. In our future work, we plan to collect preferences on contexts by user surveys or studies. Furthermore, we will explore the extensions of other context-aware recommendation algorithms as the the basis for context suggestion. For example, the contextaware splitting approaches [7, 27] could be reused to explore context suggestion. Similarity of contexts [9, 32, 33] can also be adopted as one resource to rank the recommendations. In addition, we will explore learning-to-rank algorithms to rank contexts by optimizing the ranking metrics directly.

9 7. REFERENCES [1] G. Adomavicius, B. Mobasher, F. Ricci, and A. Tuzhilin. Context-aware recommender systems. AI Magazine, 32(3):67 80, [2] G. Adomavicius and A. Tuzhilin. Tutorial on Context-aware Recommender Systems, the 2nd ACM Conference on Recommender Systems. tutorial.pdf. [3] L. Baltrunas, K. Church, A. Karatzoglou, and N. Oliver. Frappe: Understanding the usage and perception of mobile app recommendations in-the-wild. CoRR, abs/ , [4] L. Baltrunas, M. Kaminskas, B. Ludwig, O. Moling, F. Ricci, A. Aydin, K.-H. Lüke, and R. Schwaiger. Incarmusic: Context-aware music recommendations in a car. In E-Commerce and Web Technologies, pages Springer, [5] L. Baltrunas, M. Kaminskas, F. Ricci, R. Lior, B. Shapira, and K.-H. Luke. Best usage context predictions for music tracks. In The 2nd Workshop on Context-aware Recommender Systems, [6] L. Baltrunas, B. Ludwig, and F. Ricci. Matrix factorization techniques for context aware recommendation. In Proceedings of the fifth ACM conference on Recommender systems, pages ACM, [7] L. Baltrunas and F. Ricci. Experimental evaluation of context-dependent collaborative filtering using item splitting. User Modeling and User-Adapted Interaction, 24(1-2):7 34, [8] V. Bellotti, B. Begole, E. H. Chi, N. Ducheneaut, J. Fang, E. Isaacs, T. King, M. W. Newman, K. Partridge, B. Price, et al. Activity-based serendipitous recommendations with the magitti mobile leisure guide. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages ACM, [9] V. Codina, F. Ricci, and L. Ceccaroni. Exploiting the semantic similarity of contextual situations for pre-filtering recommendation. In User Modeling, Adaptation, and Personalization, pages Springer, [10] A. K. Dey. Understanding and using context. Personal and ubiquitous computing, 5(1):4 7, [11] K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS), 20(4): , [12] A. Karatzoglou, X. Amatriain, L. Baltrunas, and N. Oliver. Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative filtering. In Proceedings of the fourth ACM conference on Recommender systems, pages ACM, [13] Y. Koren, R. Bell, and C. Volinsky. Matrix factorization techniques for recommender systems. IEEE Computer, 42(8):30 37, [14] A. Košir, A. Odic, M. Kunaver, M. Tkalcic, and J. F. Tasic. Database for contextual personalization. ELEKTROTEHNISKI VESTNIK, 78(5): , [15] A. Liaw and M. Wiener. Classification and regression by randomforest. R news, 2(3):18 22, [16] X. Ning and G. Karypis. SLIM: Sparse linear methods for top-n recommender systems. In IEEE 11th International Conference on Data Mining, pages IEEE, [17] A. Odic, M. Tkalcic, J. F. Tasic, and A. Košir. Relevant context in a movie recommender system: UsersŠ opinion vs. statistical detection. ACM RecSys Workshop on Context-aware Recommender Systems, [18] X. Ramirez-Garcia and M. Garca-Valdez. Post-filtering for a restaurant context-aware recommender system. In Recent Advances on Hybrid Approaches for Designing Intelligent Systems, volume 547 of Studies in Computational Intelligence, pages Springer International Publishing, [19] S. Rendle and L. Schmidt-Thieme. Pairwise interaction tensor factorization for personalized tag recommendation. In Proceedings of the third ACM international conference on Web search and data mining, pages ACM, [20] G. Tsoumakas, I. Katakis, and I. Vlahavas. Mining multi-label data. In Data mining and knowledge discovery handbook, pages Springer, [21] G. Tsoumakas and I. Vlahavas. Random k-labelsets: An ensemble method for multilabel classification. In Machine learning: ECML 2007, pages Springer, [22] E. M. Voorhees et al. The trec-8 question answering track report. In Trec, volume 99, pages 77 82, [23] Y. Zheng. Context suggestion: Solutions and challenges. In Proceedings of the 15th IEEE International Conference on Data Mining Workshops, pages IEEE, [24] Y. Zheng. A revisit to the identification of contexts in recommender systems. In Proceedings of the 20th ACM Conference on Intelligent User Interfaces Companion, pages ACM, [25] Y. Zheng. Context-driven mobile apps management and recommendation. In Proceedings of the 31st Annual ACM Symposium on Applied Computing, pages ACM, [26] Y. Zheng, R. Burke, and B. Mobasher. Recommendation with differential context weighting. In User Modeling, Adaptation, and Personalization, pages [27] Y. Zheng, R. Burke, and B. Mobasher. Splitting approaches for context-aware recommendation: An empirical study. In Proceedings of the 29th Annual ACM Symposium on Applied Computing, pages ACM, [28] Y. Zheng, B. Mobasher, and R. Burke. Context recommendation using multi-label classification. In Proceedings of the 13th IEEE/WIC/ACM International Conference on Web Intelligence, pages IEEE/WIC/ACM, [29] Y. Zheng, B. Mobasher, and R. Burke. CSLIM: Contextual SLIM recommendation algorithms. In

10 Proceedings of the 8th ACM Conference on Recommender Systems, pages ACM, [30] Y. Zheng, B. Mobasher, and R. Burke. Deviation-based contextual SLIM recommenders. In Proceedings of the 23rd ACM Conference on Information and Knowledge Management, pages ACM, [31] Y. Zheng, B. Mobasher, and R. Burke. CARSKit: A java-based context-aware recommendation engine. In Proceedings of the 15th IEEE International Conference on Data Mining Workshops, pages IEEE, [32] Y. Zheng, B. Mobasher, and R. Burke. Integrating context similarity with sparse linear recommendation model. In User Modeling, Adaptation, and Personalization, volume 9146 of Lecture Notes in Computer Science, pages Springer Berlin Heidelberg, [33] Y. Zheng, B. Mobasher, and R. Burke. Similarity-based context-aware recommendation. In Web Information Systems Engineering, volume 9418 of Lecture Notes in Computer Science, pages Springer Berlin Heidelberg, 2015.