Indoor Location-based Recommender System. Zhongduo Lin

Indoor Location-based Recommender System by Zhongduo Lin A thesis submitted in conformity with the requirements for the degree of Master of Applied Science Graduate Department of Electrical and Computer Engineering University of Toronto Copyright c 2013 by Zhongduo Lin

Abstract Indoor Location-based Recommender System Zhongduo Lin Master of Applied Science Graduate Department of Electrical and Computer Engineering University of Toronto 2013 WiFi-based indoor localization is emerging as a new positioning technology. In this work, we present our efforts to find the best recommender system based on the indoor location tracks collected from the Bow Valley shopping mall for one week. The time a user spends in a shop is considered as an implicit preference and different mapping algorithms are proposed to map the time to a more realistic rating value. A new distribution error metric is proposed to examine the mapping algorithms. Eleven different recommender systems are built and evaluated in terms of accuracy and execution time. The Slope-One recommender system with a logarithmic mapping algorithm is finally selected with a score of 1.292, distribution error of 0.178 and execution time of 0.39 seconds for ten runs. ii

Acknowledgements I would like to thank my supervisor, Prof. Paul Chow, for his great guidance throughout the past two years. Thank you for allowing me to explore what I like and giving me full support when I didn t do well in my work. I also want to acknowledge all the support from my group members, who are always willing to help me whenever needed. Thank all my colleagues in CISCO Systems, especially my mentor Vince Mammoliti. Thank you for being such a great friend and philosopher. I greatly appreciate the encouragement from all my friends in the past two years. Thank you for cheering me up when I am down. iii

Contents 1 Introduction 1 1.1 Contribution.................................. 3 1.2 Thesis Organization.............................. 3 2 Recommender System Overview 5 2.1 Recommender System Model......................... 5 2.2 Recommender System Classifications.................... 7 2.2.1 Categories based on solutions.................... 7 2.2.2 Categories based on information collecting methods........ 9 2.2.3 Categories based on evaluation methods.............. 11 2.3 Collaborative Filtering Algorithms..................... 13 2.3.1 User-based CF algorithm....................... 14 2.3.2 Item-based CF algorithm....................... 15 2.3.3 Slope-One algorithm......................... 16 2.3.4 Similarity metrics........................... 18 3 Related Work 20 4 Collecting Data 25 4.1 Indoor Localization with WiFi........................ 25 4.2 Data Collection Architecture......................... 27 iv

4.3 The Bow Valley Square Data Set...................... 29 4.3.1 Definition of terms.......................... 29 4.3.2 Building test beds........................... 30 4.3.3 Accuracy analysis........................... 33 4.3.4 Preprocessing algorithms....................... 34 4.3.5 Statistical analysis.......................... 34 5 Building the Recommender Systems 38 5.1 System Overview............................... 38 5.1.1 Interaction between MSE and device................ 40 5.1.2 Interaction between MSE and server................ 40 5.1.3 Interaction between server and device................ 41 5.2 Mahout.................................... 42 5.3 Implicit Preference Mapping Functions................... 44 5.3.1 Mapping functions.......................... 45 5.3.2 Evaluation metrics.......................... 48 6 Methodology and Evaluation 51 6.1 Methodology................................. 51 6.1.1 Experimental platform........................ 51 6.1.2 Test cases............................... 52 6.1.3 Evaluation methods.......................... 52 6.2 Evaluation................................... 53 6.2.1 Evaluation of non-scaling mapping functions........... 53 6.2.2 Evaluation of mapping functions with scaling........... 55 6.2.3 Execution time evaluation...................... 57 7 Conclusion 59 v

8 Future Work 60 Bibliography 62 vi

List of Tables 2.1 Term definitions for different RSs...................... 6 2.2 A simple example of a utility matrix.................... 6 2.3 Classification of recommender systems research [1]............. 10 2.4 A simple example of average difference and root-mean-square difference. [2] 12 5.1 GroupLens 1M breakdown.......................... 50 6.1 Comparison of RSs with non-scaling mapping functions.......... 54 6.2 Comparison of RSs with scaling mapping functions............ 56 6.3 Distribution error for scaling mapping functions.............. 56 6.4 Comparison in terms of execution time................... 58 vii

List of Figures 4.1 Typical wireless controller to AP deployment for location [3]....... 27 4.2 MSE High Level Architecture [3]...................... 28 4.3 Interaction with the MSE API [3]...................... 32 4.4 Device occurrence distribution........................ 35 4.5 Number of points per path.......................... 36 4.6 Path duration distribution.......................... 37 5.1 Service System Overview........................... 39 5.2 Mahout Framework.............................. 43 5.3 Implicit recommender system overview................... 46 6.1 User-based RS with non-scaling mapping functions............ 54 6.2 User-based RS with scaling mapping functions............... 55 viii

Chapter 1 Introduction In the 2002 movie, Minority Report [4], Tom Cruise walks through a shopping mall and is targeted with personalized advertising. This is a highly sophisticated form of a Location- Based Service that provides a service based on the location of a user. The advertising Cruise receives also recommends products, such as cars and beer. This is a Recommender System (RS) that utilizes the location-based service to make a recommendation based on the user s location. This thesis develops the first WiFi-based Recommender System as a platform for further exploration of such systems. With the development of sensing and automated means of perceiving the physical environment, it is possible to collect much more implicit context with our everyday electronic devices such as smart phones, or personal digital assistants (PDA). Among these contexts, Ljungstrand [5] predicted that location-based services (LBSs) [6] will be the most common form of context-aware computing [7]. Traditionally, LBSs were designed to support outdoor applications such as navigation and fleet management. While the Global Positioning System (GPS) [8] has achieved a great success and popularity all over the world in the recent decade, indoor LBSs are a marketing tool that have the potential to increase business profit. The growing interest in this technology can be demonstrated by the recent actions of large technology companies. 1

Chapter 1. Introduction 2 1. March 24th, 2013, Apple acquired WiFiSlam [9], an indoor GPS startup that enables a smart phone to pinpoint its location. 2. September 26th, 2012, Cisco announced the acquisition of ThinkSmart Technologies [10], a startup that analyses indoor location information based on Cisco s wireless networking infrastructure. 3. April 27th, 2010, Google [11] submitted a white paper to several national data protection authorities on vehicle-based collection of WiFi data for use in Google location-based services. Often used in conjunction with a LBS is a Recommender System (RS). Recommender systems [12, 13, 14] are a subclass of information filtering systems that seek to predict the rating or preference that a user would give to an item (such as music, books, or movies) or social element (e.g. people or groups) they had not yet considered, using a model built from the characteristics of an item (content-based approaches) or the user s social environment (collaborative filtering approaches). It has become an extremely common context aware service in recent years. One main area of research has been focused on building recommender systems based on user trajectories. Application domains include mobile social networking, tourism guides, urban computing and information retrieval. While there are a few research papers published recently about location-based recommender systems using GPS tracks, there are even fewer for indoor LBSs due to the lack of a standard for indoor positioning, which will be discussed in Chapter 4. Compared to GPS, WiFi-based indoor positioning systems introduce different noise due to the instability of WiFi signals, and present unique characteristics that will be discussed in Section 4.1.

Chapter 1. Introduction 3 1.1 Contribution This work details our efforts to find the best recommender system implementation based on the real-world mobile tracks in a shopping mall, aiming to recommend the shops in the mall that will interest the customers according to their track history. Over ten thousand customers are tracked for a week in the Bow Valley Square shopping mall [15]. This data is used to develop a recommender system to predict the preferences a user will give to all the shops based on the data set and recommend the top shops to the user. The main contributions of this work are: 1. The first time building of a location-based recommender system using WiFi positioning technology; 2. A comparison between different recommender systems using WiFi positioning; 3. The proposal of different mapping functions for implicit information and a new evaluation metric for the mapping functions Though this work only focuses on a specific data set, it is expected that the methods developed in this thesis will work on other applications using the same positioning infrastructure. For example, CISCO s acquisition, ThinkSmart Technology, has demonstrated success on other applications such as airport planning and museum tour guide systems using the same positioning system. 1.2 Thesis Organization The remainder of this thesis is organized as follows. Chapter 2 provides an overview of the state-of-the-art technologies used in recommender systems and the different categories of recommender systems. This overview will help with understanding the basic data collection methods in Chapter 4 and evaluation methodology in Chapter 6. A list of

Chapter 1. Introduction 4 related work, including outdoor location-based recommender systems and generic services based on indoor location, is presented in Chapter 3. Chapter 4 details our data collecting method and a simple analysis of the data set. Chapter 5 presents the whole process of building the recommender systems. Our methodology and evaluation results are detailed in Chapter 6. Chapter 7 provides the conclusion and Chapter 8 describes future work.

Chapter 2 Recommender System Overview The interest in recommender systems has been high among industry and academia since the mid-1990s when the first papers on collaborative filtering were published and proved to be useful [16, 17, 18]. Then there was a significant boost to research into recommender systems when NetFlix offered a prize of $1,000,000 to the first person or team to beat their own recommender algorithm, CineMatch, by 10% [2]. This chapter will briefly introduce the basic ideas, concepts and techniques for general recommender systems. A more detailed and comprehensive overview on recommender systems can be found in [19]. 2.1 Recommender System Model There are three subjects in the data sets used to build a recommender system: user, item and preference. Since the Amazon book recommender system [20] is more familiar to the audience, it is taken as an analogy to help illustrate these three terms along with the shop recommender system (Shop RS) developed in this thesis,. Table 2.1 shows a comparison between the Amazon Book RS and Shop RS in terms of the definitions of the three subjects. Generally, the users refer to the customers who make the decisions and are the ones that the recommender systems aim to recommend items to. The items refer to different objects available for the users to choose. They can 5

Chapter 2. Recommender System Overview 6 Table 2.1: Term definitions for different RSs Amazon Book RS Shop RS user customers on the online system customers in the shopping mall item books on the system shops in the shopping mall preference ratings given by users to items time users spend in certain shops Table 2.2: A simple example of a utility matrix User ID CIBC Copy Centre Rise Bakery Cafe X-Press Second Cup 6 315 931 56 340 1023 80 135 80 648 2156 be books for the Amazon Book RS, places for a tourism recommender system, etc. The preference refers to the degree that a user likes a certain item. The most popular way to represent the preference is by the explicit rating values ranging from 1 to 5, which is used by the Amazon online system [20] and BestBuy [21]. They can also be implied by user behaviours like clicks or time spent in a certain web page, which will be discussed later in Section 2.2. The data set used in a recommender system is always represented by a matrix that is referred to as a utility matrix. A simple example for Shop RS is illustrated in Table 2.2. The users are represented by their user IDs such as 56 in the leftmost column, while the items are represented by the names of the shops. The preference values in the example are the actual number of seconds the user spent in the shops. Notice that most values in the preference field are left blank, which indicates that the users have never entered the corresponding shops. In reality, since a row in the utility matrix will include all the preferences to each shop in the shopping mall, the matrix will be much more sparse than that in the example, with the typical user only visiting a tiny fraction of all the available shops.

Chapter 2. Recommender System Overview 7 2.2 Recommender System Classifications There are many different ways to classify and identify a recommender system based on certain attributes or techniques used. Similar to [13], in this section we focus on the three major attributes that apply to most recommender systems, and can be used to identify the recommender system developed in this thesis the recommendation solutions adopted, the information collection methods and the evaluation methods. 2.2.1 Categories based on solutions A blank preference can be estimated in different ways using methods from machine learning, approximation theory and various heuristics. A recommender system solution refers to the way the recommender system is built. Following the classifications in [22], the recommender systems can be classified into three categories based on their approach: 1. Content-based recommender: The RS recommends items similar to the ones the user preferred in the past, 2. Collaborative recommender: The RS recommends items that similar users preferred in the past, 3. Hybrid recommender: A combined approach using both content-based recommender and collaborative recommender. In a content-based recommender, a record or collection of records representing important characteristics of an item, which is often referred to as an item profile, must be constructed to classify different items. For example, in a music recommender system, a profile can include features such as the composer, the singer, the genre and the year. There are various techniques to automatically extract characteristics from items, among which the most widely used one is Term Frequency/Inverse Document Frequency (TF-IDF) that is used to specify keyword weights in text-based items. The idea of

Chapter 2. Recommender System Overview 8 content-based recommender systems is based on the fact that people tend to like items that are similar to their preference. Assuming that some users like a song from Avril Lavigne, it is likely that they will like other songs from her. While this approach works quite well in practice, it suffers from several drawbacks: 1. Limited content analysis: Not only is it time consuming when some features, such as music genre, have to be entered manually, but it is not sufficient to reflect the different quality or popularity among items with the same features. 2. Domain-specialization: It is almost impossible to decide a set of general features for all the items. For example, a book can be well described by its page count, author and publisher, while none of these features can be applied to a bookmark. However, one can easily see the connection between a book and a bookmark. These drawbacks prevent researchers from developing a general framework for contentbased recommender systems [2]. An alternative way is to use collaborative methods, which is enjoying a high interest among researchers and is also the approach of this thesis. Instead of depending on the features of items to determine their similarity, it leverages the similarity of the user ratings for co-rated items. More details about the collaborative approach will be discussed in Section 2.3. The main limitation of this approach is the sparsity of the user ratings. In any recommender system, the number of ratings already obtained is usually very small compared to the number of ratings that have to be predicted. For example, there may be millions of books in the Amazon book recommender system, while the average number of books that a user rates may be below five. To overcome the limitations of content-based and collaborative recommender systems, a hybrid approach is proposed by combining collaborative and content-based recommender techniques. Almost all modern recommender systems can be classified into this category since they more or less utilize techniques from both content-based and col-

Chapter 2. Recommender System Overview 9 laborative methods. Depending on the different ways to combine these two methods, the hybrid recommender systems can be further classified as: (1) combining totally separate recommender systems, (2) adding the characteristics of one to the other, (3) constructing a general unifying model that incorporates the characteristics of both recommender systems. Since this is not the focus of this thesis, the discussion will end here. More details can be found in [1]. Table 2.3, reproduced from [1], shows a comprehensive summary of techniques used in each category and example research efforts. A brief introduction to heuristic-based and model-based techniques is described in Section 2.3. 2.2.2 Categories based on information collecting methods Information collecting methods refer to the ways that user feedback is collected. Depending on the level of user involvement, recommender systems can be classified as follows: 1. Intrusive recommender system: A significant level of user involvement is required to get the feedback. 2. Non-intrusive recommender system: Little or no explicit user involvement is required to get the feedback. Most of the practical recommender systems now fall into the intrusive recommender systems. The most widely used way to collect user feedback is to explicitly ask users to rate the items they have reviewed or purchased. For example, in almost all the online shopping systems such as BestBuy and ebay, customers are encouraged to rate the items they bought. While being predominant, intrusive recommender systems suffer from the fact that users are lazy in that they would not bother to even come up with the appropriate rating [17], resulting in an extremely small number of ratings per user. An alternative way to get feedback is to leverage certain proxies to estimate the real rating a user will give to an item. Minimizing intrusiveness while keeping the accuracy

Chapter 2. Recommender System Overview 10 Table 2.3: Classification of recommender systems research [1] Recommendation Recommendation Technique Approach Heuristic-based Model-based Content-based Commonly used techniques: Commonly used techniques: TF-IDF (information retrieval) Bayesian classifiers Clustering Clustering Representative research examples: Decision trees Lang 1995 Artificial neural networks Balabanovic & Shoham 1997 Representative research examples: Pazzani & Billsus 1997 Pazzani & Billsus 1997 Mooney et al. 1998 Mooney & Roy 1999 Billsus & Pazzani 1999, 2000 Collaborative Hybrid Commonly used techniques: Nearest neighbor (cosine, correlation) Clustering Graph theory Representative research examples: Resnick et al. 1994 Hill et al. 1995 Shardanand & Maes 1995 Breese et al. 1998 Nakamura & Abe 1998 Aggarwal et al. 1999 Delgado & Ishii 1999 Pennock & Horwitz 1999 Sarwar et al. 2001 Combining content-based and collaborative components using: Linear combination of predicted ratings Various voting schemes Incorporating one component as a part of the heuristic for the other Representative research examples: Balabanovic & Shoham 1997 Claypool et al. 1999 Good et al. 1999 Pazzani 1999 Billsus & Pazzani 2000 Tran & Cohen 2000 Melville et al. 2002 Zhang et al. 2002 Commonly used techniques: Bayesian networks Clustering Artificial neural networks Linear regression Probablistic models Representative research examples: Billsus & Pazzani 1998 Breese et al. 1998 Ungar & Foster 1998 Chien & George 1999 Getoor & Sahami 1999 Pennock & Horwitz 1999 Goldberg et al. 2001 Kumar et al. 2001 Pavlov & Pennock 2002 Shani et al. 2002 Yu et al. 2002, 2004 Hofmann 2003, 2004 Marlin 2003 Si & Jin 2003 Combining content-based and collaborative components by: Incorporating one component as a part of the model for the other Building one unifying model Representative research examples: Basu et al. 1998 Condliff et al. 1999 Soboroff & Nicholas 1999 Ansari et al. 2000 Popescul et al. 2001 Schein et al. 2002 Table 2: Classification of recommender systems research. 3.1. Comprehensive understanding of users and items As was pointed out in [2, 8, 54, 105], most of the recommendation methods produce ratings that are based on a limited understanding of users and items as captured by user and item profiles and do not take full advantage of the information in the user's transactional histories and other

Chapter 2. Recommender System Overview 11 of recommendations is still an important research topic because of its difficulty and promising potential. The non-intrusive approach is mostly used in online systems such as newsgroup article recommender systems and the most common proxies are the click behaviour and the time spent on a article. Morita and Shinoda [23] have found that user preference to NetNews articles are well reflected by the time spent reading these articles regardless of the length of the article. The recommender system described in this thesis is one of the non-intrusive recommender systems, since the time users spend in a certain shop is used as a proxy to estimate their ratings. After determining the proxy, another important factor that will influence the efficiency of the recommender system is how to actually derive the appropriate estimation of preference from the implicit information. Different mapping functions are discussed in Chapter 5. 2.2.3 Categories based on evaluation methods A recommender system is a tool to generate the best recommendations. Therefore, before making a recommendation to the user, the recommender system should determine which recommendation is the best one. The ideal recommender would be a psychic that could know exactly user preferences towards different items. However, unlike cases such as solving a mathematical problem, where a golden key exists and can be used to examine an algorithm, no one can know exactly how much a user will like a new item, including the user itself, in a recommender system. Different recommender systems are trying to achieve a certain goal that they believe will result in the best recommendation. This goal, though it will never be perfect, is referred to as the evaluation method. The two most prevailing evaluation methods are: 1. Scoring: Set aside a small part of the real data set as test data set, then estimate the test set with the remaining data set, and finally compare the difference between the estimated values and the real ones.

Chapter 2. Recommender System Overview 12 Table 2.4: A simple example of average difference and root-mean-square difference. [2] Item 1 Item 2 Item 3 actual value 3.0 5.0 4.0 estimated value 3.5 2.0 5.0 difference 0.5 3.0 1.0 average difference =(0.5+3.0+1.0)/3=1.5 root-mean-square = (0.5 2 + 3.0 2 + 1.0 2 )/3 = 1.8 2. Precision and recall: Similar to the scoring method, but instead of comparing values, the top n items are returned for each user. The items are then used to calculate the precision and recall. Precision is the proportion of top recommendations that are good recommendations (existing highly rated items) and recall is the proportion of good recommendations that appear in top recommendations. The precision and recall method is not used in this thesis because it generally requires a relatively large number of ratings from the users, which is not applicable to this work. A more detailed description can be found in [2]. For the scoring evaluation method, after getting the estimated preference values from a collaborative filtering algorithm, the recommender systems need to determine the overall error metric they want to minimize. Average difference and root-mean-square of the differences are the two most common metrics to use. Table 2.4 gives a simple example to illustrate how these two error metrics work. The two metrics are similar in terms of the quality of recommendations. The average difference is adopted in this thesis. A smaller value indicates better performance of the recommender system.

Chapter 2. Recommender System Overview 13 2.3 Collaborative Filtering Algorithms A collaborative recommender system tries to estimate the unknown ratings based on the items previously rated by other users. Therefore, unlike a content-based approach, a collaborative recommender system can only depend on the ratings rather than any domain specific information of items. According to Breese [24], collaborative filtering algorithms can be classified into two groups: 1. Heuristic-based: Make rating estimations based on the entire collection of previously rated items by the users. 2. Model-based: Use the collection of ratings to learn a model that will be used to make rating predictions. The model-based collaborative filtering algorithms are getting more and more attention recently as the interest in machine learning grows among academia. Many techniques such as artificial neural networks, singular value decomposition and clustering are leveraged to learn different models for recommender systems and are proven to be efficient in many applications [1]. Despite its great potential, it is not implemented in this thesis. Instead, the simplest-first approach is taken because the goal is to first implement a complete system that can then be studied and measured to determine where improvements are required. The heuristic-based collaborative filtering (CF) algorithms are the focus of this thesis. They are widely used because of their simplicity and efficiency. They can be further divided into two categories: 1. User-based CF algorithm: Estimate the unknown ratings based on other similar users. 2. Item-based CF algorithm: Estimate the unknown ratings based on the similarities between the target item and other items that are co-rated by the the same user.

Chapter 2. Recommender System Overview 14 2.3.1 User-based CF algorithm The shopping mall case in this thesis is taken as an example to illustrate how the userbased CF algorithm works. Around Christmas, there will be many people going to the shopping mall to prepare for a Christmas party. Most of them will go to the candy shop, the gift shop and the decoration shop during their visits. So if a customer has already been to the candy shop and the gift shop, then it can be inferred that he or she is similar to those who are preparing for a Christmas party and the decoration shop can be a good recommendation to the customer. The reasoning is intuitive because people tend to like things that similar customers like. The first problem is how to define the similarity metrics, which will be discussed in Section 2.3.4. Several terms, which will be used in this section, need to be defined before the algorithm is introduced. 1. Similarity: How similar a user is to another one in terms of their ratings to the items that both rated. 2. Nearest neighbours: The users that are most similar to a certain user according to the similarity metric. 3. Neighbourhood size: The number of the neighbours used in a user-based recommender system. Algorithm 1 lists a generic user-based algorithm. The first for loop aims to get the n nearest neighbours of each user. Note that we can simply ignore this step to include all the other users in the second for loop, which is equivalent to setting n to infinity. However, setting an appropriate value to n will accelerate the computation and always result in better recommendations, because more similar users tend to provide more reliable predictions. The second for loop estimates the unknown ratings by calculating a weighted average of ratings that the n nearest neighbours give to the items.

Chapter 2. Recommender System Overview 15 for every other user w do compute a similarity s between u and w; retain the top users, ranked by similarity, as a neighbourhood n; end for every item i that some user in n has a preference for, but u has no preference for yet do for every other user v in n that has a preference for i do compute a similarity s between u and v; incorporate v s preference for i, weighted by s, into a running average; end end Algorithm 1: User-based collaborative filtering algorithm [2] 2.3.2 Item-based CF algorithm The same example in Section 2.3.1 is taken to illustrate how the item-based CF algorithm works. Around Christmas, there will be many people going to the candy shop, the gift shop and the decoration shop. So if a customer has already been to the candy shop and the gift shop, then it can be concluded that since the candy shop, the gift shop and the decoration shop tend to be visited altogether, the decoration shop can be a good candidate to recommend to the customer. Though the recommendation result of this item-based approach is the same as the user-based one, the basis to draw the conclusion is different. The recommendation is based on the similarity of shops regardless of the users. Therefore, the term nearest neighbours does not exist in the item-based CF algorithms. The similarity, however, still needs to be calculated though from a different perspective. Algorithm 2 lists a generic item-based algorithm. The outer for loop iterates all the items that a user has not rated yet. The inner for loop calculates the weighted average

Chapter 2. Recommender System Overview 16 for every item i that u has no preference for yet do end for every item j that u has a preference for do end compute a similarity s between i and j; add u s preference for j, weighted by s, to a running average; return the top items, ranked by weighted average; Algorithm 2: Item-based collaborative filtering algorithm [2] of the ratings of items that the user has already rated. Note that the similarity s here refers to the similarity between items instead of users. There are several advantages of the item-based approach compared to the user-based one: 1. Scalability: The run time of an item-based recommender system scales up as the number of items increases, thus if the number of items is relatively low compared to the number of users, an item-based recommender system is more preferable. 2. Less subject to change: Over time the similarities between items tend to converge, while user tastes can vary vastly. Therefore, an item-based recommender system typically can start making reasonable recommendations after a user s first rating, while user-based ones need enough ratings to find nearest neighbours. 3. Able to be preprocessed: Since the item-item similarities are more fixed, it is reasonable to precompute them to speed up the execution time. 2.3.3 Slope-One algorithm The Slope-One item-based collaborative algorithm was proposed by Lemire in 2005 to reduce over-fitting, improve performance and ease implementation [25]. It has quickly

Chapter 2. Recommender System Overview 17 become popular. The assumption of the Slope-One filtering algorithm is that a certain linear correlation between the preference values for one item and another exists and can be used to estimate the preferences for some item Y based on the preferences for item X, via some linear functions like Y = mx + b. for every item i do end for every other item j do end for every user u expressing preference for both i and j do end add the difference in u s preference for i and j to an average; for every item i the user u expresses no preference for do end for every item j that user u expresses a preference for do end find the average preference difference between j and i; add this difference to u s preference value for j; add this to a running average; return the top items, ranked by these averages; Algorithm 3: Slope-One algorithm [2] The Slope-One algorithm is shown in Algorithm 3. The first for loop of the algorithm calculates item-item differences in preference values based on all the user ratings. The second part of the algorithm estimates each unknown rating that a user gives to a certain item, by averaging each estimated rating based on another item that the user rated and the differences between these two items. Finally the top items are returned for recommendations.

Chapter 2. Recommender System Overview 18 The Slope-One algorithm is attractive since the online portion is fast and scalable. Moreover, it is easily updated when a preference changes, because only the relevant difference values need to be updated. However, it suffers from large memory requirements since O(n 2 ) space is required to store the item-item differences, where n refers to the number of items. 2.3.4 Similarity metrics Both user-based and item-based collaborative filtering algorithms require the calculation of similarity. Note that the utility matrix in Table 2.2 can be described as a set of user vectors, each one of which refers to a row in the matrix, as well as a set of item vectors, each of which refers to a column in the matrix. Therefore, both the item-based and user-based approaches can share the same general vector similarity metrics. Various similarity metrics can be found in different implementations to calculate the similarity between two vectors sim(x, y). The common metrics include the Pearson correlation-based similarity, Euclidean distance-based similarity, cosine measure similarity and log-likelihood test [2]. Or alternatively, a custom one can be implemented to leverage domain-specific information. For example, if the similarity metric is determined by the attributes of items instead of ratings, then a content-based recommender system can be built using the collaborative algorithm. The similarity used in this thesis is the Pearson correlation-based similarity due to its popularity and efficiency. Equation (2.1) shows the naive equation of the Pearson correlation-based similarity for the user-based algorithms. The equation works for the item-based ones with a slight modification of the notations. Let S xy be the set of all items rated by both users x and y, r x,s be the rating user x gives to item s and r x be the

Chapter 2. Recommender System Overview 19 average rating of all the ratings from user x. Then sim(x, y) can be calculated as: (r x,s r x )(r y,s r y ) s S xy sim(x, y) = (r x,s r x ) (2.1) 2 (r y,s r y ) 2 s S xy s S xy The Pearson correlation is a number between -1 and 1 that measures the tendency of two vectors. A larger value implies a higher positive linear correlation between the two vectors. Intuitively, the more items that are co-rated by both users, the more reliable one user s rating can be used to predict the other s. However, the naive Pearson correlation fails to consider the number of items over which it is computed. Therefore, a weight that can reflect the number of common rated items is sometimes added to the equation of the naive Pearson correlation.

Chapter 3 Related Work To the best of our knowledge, there has never been any work on an indoor locationbased collaborative recommender system before. A significant reason is the lack of an indoor positioning technology, which will be discussed in Chapter 4. However, there are a handful of efforts to enhance user experience in a physical indoor shopping mall with recommendations. The consumer-friendly shopping assistance system built by Sae-Ueng [26] is the most similar attempt to this work. The system collects personal behaviour to a log file automatically with RFID and camera sensors in the ubiquitous environment. By implicitly inferring customer preferences from their behaviours, such as touching and purchasing a product, the system can then make customized recommendations to the customers. However, the fact that they use RFID to identify customers requires all the customers to wear a customized RFID tag, which generally is not applicable to a shopping mall. And using cameras to detect customer behaviour not only increases the system cost, but is not likely to guarantee a correct detection rate in reality, where people may be close to each other. The data collection method in this thesis is based on standard WiFi-enabled devices carried by the customers, which generally will not require any specific hardware from the shopping mall. 20

Chapter 3. Related Work 21 Though not a location-based recommender system, the personalized shopping recommender built by Hsu [27] tries to predict user preferences based on their transactional histories. Instead of building a collaborative filtering method based on ratings (e.g., GroupLens [17]) to perform personalized shopping recommendations, they derive a model-based recommender system based on a customized probabilistic graphic model called Hybrid Poisson Aspect Modelling (HyPAM) to address data skewness and sparsity. HyPAM applies a cluster model to cluster customers and an aspect model to model the relationships between customer clusters and products. Experimental results show that HyRAM outperforms representatives of the collaborative filtering methods and data mining methods by a large margin. In this thesis, only collaborative filtering algorithms are implemented to test the performance of different recommender systems. Asthana [28] designs a shopping assistant service that personalizes the attention provided to a customer based on individual needs. The system consists of a wireless communication device called a Personal Shopping Assistant, and a centralized server storing customer profiles. The server provides personalized service by pushing retail information to the user devices. However, the user cannot update the database, preventing the server from making recommendations based on user s purchase history. Also, the devices are provided by the retailer rather than integrated in a standard hand-held device. Reischach and Michahelles [29] present a concept called Apriori that enables consumers to access and share product recommendations using their mobile phone. This work encourages users to submit ratings by providing a better user interface with their mobile phones. However, the system only enables users to submit their ratings and receive others ratings, but fails to make customized recommendations to users. In this thesis, the customers are not required to explicitly rate any items and the server can make recommendations based on their behaviour. Anacleto [30] describes a customized one-to-one recommendation system inside a virtual shopping center, considering server, product, and facility at the same time. The

Chapter 3. Related Work 22 system extracts a purchase pattern for each customer from the shopping path history. Then it provides customized recommendations of certain brands based on the customer s current location in the virtual shopping mall to achieve improvement in sales and profit of a retail company. In this thesis, instead of specific items, shops are being recommended to customers. An extensive study on indoor wireless network traces is presented in Hsu [31]. The data is collected from the networks at the University of South California and the University of Florida over a period of several months. Instead of getting the exact location, the location is roughly represented by the APs that the mobile devices are associated with. Though no recommender system is developed in this design, an efficient way for mobile users to summarize their mobility preferences is constructed based on singular value decomposition (SVD). Then the mobility summaries are used to calculate the distance between users and identify user groups in the population based on their mutual similarities. The location data used in this thesis is in a much finer granularity, with a precision of 1-2 meter instead of generally tens of meters using APs for localization. Instead of focusing on grouping users, we move a step forward by building a recommender system with the traces. The B-MAD system (Bluetooth Mobile Advertising) [32] delivers location-aware mobile advertisements to mobile phones using Bluetooth positioning and Wireless Application Protocol (WAP) push, with an accuracy of 50 to 100 meters. Each device is identified by its phone number (MSISDNs). The Ad Server recommends any undelivered advertisements associated with the location that have not been delivered to the end user. The basic system architecture of the B-MAD system is similar to the one in this work. However, the recommendation algorithm running on the Ad server is much simpler, since advertisements associated with each location are pre-defined. Therefore, little computation, if any, is required to make recommendations to users. Another related research area is outdoor location-based recommender systems, which

Chapter 3. Related Work 23 has been receiving great attention recently. While the characteristics of outdoor location data are quite different from that of the indoor one (discussed in Chapter 4), the techniques used in both cases are somewhat similar. However, even for outdoor LBS, few pure location-based collaborative recommender systems, if any, are found. The main reason is that the pattern of outdoor activities tends to be more fixed. For example, people will be in the work place during daytime and go back home at night. With WiFi indoor localization, the physical mall will be more like an online shopping system, as people move around fast and arbitrarily, which will be discussed in Section 4.1. Cyberguide, a mobile context-aware tour guide [33], is designed to be a location-based tour guide for indoor and outdoor users on a number of different hand-held platforms. The indoor positioning system is based on using TV remote control units as active beacons and a special infrared receiver, while standard GPS is used in the outdoor environment. The goal of this system is to predict tourist destinations based on the current location and a history of past locations. Cyberguide is designed mainly for navigation and easier communication between mobile devices. No recommender system is implemented in [33]. Li and Zheng [34] propose a framework called hierarchical graph-based similarity measurement (HGSM) to consistently model each individual s location history and effectively measure the similarity among users. Both the sequence property of people s movement behaviours and the hierarchy property of geographic spaces are taken into account. The framework is evaluated by using the GPS data collected by 65 volunteers over a period of six months in the real world. Though no recommender system is built in [34], the similarity metric developed can be a good candidate for a location-based recommender system and can be integrated into the work in this thesis. Ahn [35] presents a novel advertisement recommendation model for mobile users called Mobile Advertisement Recommender using Collaborative Filtering (MAR-CF). The model is a multi-dimensional personalization model based on the traditional CF algorithm, taking into account user location, interest and time. The utility matrix is

Chapter 3. Related Work 24 modified to incorporate additional contexts to items and a multi-dimensional similarity metric is developed to replace the conventional Pearson correlation metric. Though a collaborative filtering recommender system as well, the recommender system developed in this thesis is a pure location-based recommender system, which means the location itself is the item. Therefore, only the time and location information is used in this thesis, as in a physical shopping mall the transaction history is almost impossible to acquire.

Chapter 4 Collecting Data As a garbage in, garbage out filtering algorithm, the performance of a recommender system depends significantly on the quality of the input data. This chapter presents the advantages of WiFi-based positioning over the other indoor positioning methods and the positioning architecture using the CISCO Mobility Service Engine (MSE) [36] in our experiment. An analysis of the Bow Valley shopping mall data set used in this thesis is presented. 4.1 Indoor Localization with WiFi Since its first appearance at the beginning of the 1990s, LBSs have been explored in conjunction with research on ubiquitous computing. While traditional LBSs were designed for supporting outdoor applications, there has been a growing demand for indoor LBSs for asset management and better shopping experiences. While GPS has become a standard and efficient way for outdoor positioning, there are not any known large-scale indoor positioning systems due to the absence of a standard way for indoor positioning. Conventional GPS receivers do not work inside buildings due to the signal absorbing effects of the buildings, while cellular positioning methods generally fail to deliver a satisfactory degree of accuracy [6]. An increasing number of efforts are spent in extracting 25

Chapter 4. Collecting Data 26 values from GPS tracks, for example, mining interesting places. However, compared to outdoor tracks, analysing indoor location tracks can provide more direct benefits to the service providers, such as a shopping mall in this thesis, for the following reasons: 1. More total time spent: People spend more than 90% of their time indoors. Therefore, a collecting period of a meaningful data set for GPS tracks is generally above half a year for each person, while the lifetime of a user track in this application is generally less than one hour. 2. Higher mobile frequency: People tend to change their locations more frequently when they are indoors compared to outdoors. After a move outdoors, people are likely to stand statically in one area, which is not distinguishable by GPS, for a long time. But people tend to move frequently when they are in a museum or a shopping mall. 3. Finer granularity: The physical distance between locations outdoors is generally greater than indoors, forcing more constraints such as time constraints to a locationbased recommender system. For example, people are not likely to travel two hours for a lunch even though the recommended restaurant may be a perfect match. On the other hand, the locations recommended by an indoor location-based recommender system are generally more reachable. Despite the lack of a standard positioning method, a variety of techniques are explored by researchers in a limited scale. Among them, the most popular methods include RFID, Bluetooth, Ultrasound, WiFi and Infrared. WiFi-based positioning systems are now getting the most interest due to their ability to leverage the existing network architecture and the popularity of WiFi-enabled devices. Moreover, CISCO has embedded its positioning hardware engine, MSE, into the CISCO network infrastructure, increasing the potential of WiFi as a standard way for indoor positioning.

Chapter 2 Architectural Overview M obility Services Engine to Application Chapter 4. Collecting Data 27 Figure 2-2 Typical wireless controller to AP deployment for location To calculate location for the laptop shown in Figure 2-2, a MSE running the Context-Aware Mobility Service must collect information from all controllers (and their access points) in the surrounding physical Figure environment, 4.1: Typical rather than wireless a single wireless controller controller to AP(and deployment its access points). for location Because [3] of this, it is necessary to run the Context-Aware Mobility Service on an appliance or server that aggregates all access point measurements from multiple wireless controllers. 4.2 Data Collection Architecture In addition, location calculations must be performed at a high rate (and within a matter of a few seconds) to enable context consumers to take advantage of context-aware information. The results of a dedicated platform include a more scalable service that can meet the needs of high performance applications that use contextual information. The CISCO MSE is a platform that runs one or more mobility services including the M obility Services Engine to Application Context-Aware mobility service, which can capture information on network equipment, network sensors, environmental sensors, mobile network devices, and mobile assets. It The Mobility Services API is an interface that provides management and data access to the services can running easily on integrate the MSE as device shown information, Figure 2-3. such as location, with other systems to improve their business functionality. With appropriate configuration, the MSE can provide an accuracy within one meter. Figure 4.1 shows the infrastructure when the MSE is used in a positioning system. OL-17775-01 The laptops or other WiFi-enabled devices being located periodically emit beacons to Context Aw are M obility API W hite Paper 2-3

Introduction to Architectural Overview Chapter The high level 4. architecture Collecting of the Data MSE is shown in Figure 2-1. 28 Figure 2-1 MSE High Level Architecture The Context-Aware Mobility Service, like other mobility services, is a software instance running on the MSE. Figure 4.2: MSE High Level Architecture [3] The Context-Aware Mobility Service has the following characteristics: It functions across multiple edge technologies such as 802.11 wireless and 802.3 wired networks. several directions at each reference position. The access points (APs) in the surrounding OL-17775-01 area receive these beacons and record the associated Receive Signal Strength (RSS). Context Aw are M obility API W hite Paper These data are then transferred to the MSE through the network controllers. 2-1 Then the Fingerprinting positioning algorithm [6] is run on the MSE to calculate the exact locations of the devices in the map, which are then stored to the database inside the MSE. The advantage of such a network-based mode is that unlike a GPS analysis tool, no software is required to be installed in the users devices, increasing the accessibility of the positioning system. In addition, since the MSE is integrated into the basic network infrastructure, the total cost of such an additional function is much cheaper compared to other indoor positioning systems such as RFID in a network-enabled place. Figure 4.2 describes the high-level architecture of the MSE. After storing the locations