Large Scale Wireless Indoor Localization by Clustering and Extreme Learning Machine

Transcription

1 arge Scale Wireless Indoor ocalization by Clustering and Extreme earning Machine Wendong Xiao, Peidong iu 2, Wee-Seng Soh 2, Guang-Bin Huang 3 School of Automation and Electrical Engineering, University of Science and Technology Beijing, China 2 Department of Electrical & Computer Engineering, National University of Singapore, Singapore 3 School of Electrical & Electronic Engineering, Nanyang Technological University, Singapore wendongxiao68@gmail.com, {u080428, elesohws}@nus.edu.sg, egbhuang@ntu.edu.sg Abstract Due to the widespread deployment and low cost, WAN has gained more attention for indoor localization recently. However, when we apply these WAN based localization algorithms to large-scale environments, such as a wireless city, they may encounter the scalability problem due to the huge RSS database. The huge database may cause long response time for the terminal clients if the localization algorithm needs to search the database for the real time localization phase. In this paper, we propose a novel clustering based localization algorithm for large scale area by utilizing Nearest Neighbor (NN) rule and Extreme earning Machine (EM). The proposed algorithm has shown competitive advantage in terms of the real time localization efficiency as well as the localization accuracy. Keywords- Scalability; WAN; Clustering; EM I. INTRODUCTION ocation Based Service (BS) is a key-enabling technology and widely exists in today s wireless communication networks from the short-range Bluetooth networks to the long-range telecommunication networks []. The accurate localization or positioning is a key research task for any location aware system. The Global Positioning System (GPS) is most widely used outdoors. However, due to the signal path loss and other technical limitations, GPS is unsuitable for the complex indoor environment. Many solutions utilizing the Angle of Arrival (AoA), the Time of Arrival (ToA) and the Time Difference of Arrival (TDoA) information have been proposed. Although these solutions provide high accuracy, they usually require advanced hardware, which will increase the cost and complexity of the systems. Due to the widespread deployment and low cost of Wireless ocal Area Network (WAN), WAN based indoor localization solutions have gained more attention, and the Received Signal Strength (RSS) based fingerprinting approach is recognized as the most popular one among them [2-4]. The fingerprinting localization procedure usually consists of two steps: offline phase and online phase. During the offline phase, an empirical database (or radio map) is built. Then they are utilized to do the real-time localizations. The methods can be grouped into three main categories: proximity based method, probability based method and Artificial Neural Network (ANN) based method. The proximity and probability based methods usually have large computation complexity during the online localization phase, which will cause a long response time if many clients are querying for the service at the same time. Meanwhile, the slow learning speed has been a major bottleneck for the traditional ANN in the past decades. The reason is that the hidden node parameters of the ANN need to be tuned recursively. In this paper, we utilize a new proposed machine learning algorithm: Extreme earning Machine (EM) proposed in [5]. As demonstrated in our previous work [6], EM based localization method can provide competitive fast real-time localization speed due to their ability to approximate the locations without searching the huge empirical database. This can also reduce the storage burden for the terminal clients. In the literature, several works have discussed how to apply these localization algorithms to large-scale environments, such as a wireless city (typically with a size about a few square kilometres) [7-0]. However, they may encounter the scalability problem due to the huge RSS database. We might need a very huge ANN architecture for the whole city if we do not scale it into small areas during the training and online localization phase. Even if we do not utilize ANN based method, the other two types of methods will bring long response time for the online phase due to the searching of the huge empirical database. The centralized approach also has many drawbacks for the database maintenance, e.g., Single point of failure, infrastructure cost etc. Several works have been done in the literature to address the scalability issue. Two clustering approaches were proposed in []: Explicit clustering approach and Implicit clustering approach. The Explicit clustering approach determines the clusters during the offline training phase as a separate step. The approach groups locations according to the Access Points (AP) that cover them. However, there are many parameters need to be calibrated for this approach. The Implicit clustering approach utilizes the property that each AP defines a subset of area that is covered by this AP. The method it used to locate these areas is to check the AP associated with the strongest mean RSS for each sample during the online phase. Then apply the probability based localization method to estimate the location by using the observation sequence from this access point alone. The localization method will give a location list; each location in the list is associated with a probability. If the largest probability is not significantly higher than the second largest one, the procedure will be repeated for the second AP which associated with the second largest mean RSS. This clustering method is time-consuming and the location might never be found since there might have no locations associated with a significantly probability. Another issue brought by this method is that there are many parameters need to be tuned for the real-time localization phase. A space partition scheme was introduced in [2] according to the locations of the reference points. This partition strategy gave little or no respect to the underlying 609

2 environment radio propagation properties. Another partition method called Intersection of APs was proposed in [3]. This method can only be applied to proximity based method and probability based method. Real-time localization still needs to search the empirical database. This still increases the response time of the query, although it can reduce the storage burden of the terminal devices. In this paper, we adopt a hybrid approach for large scale localization, similar to []. We determine the clusters in the offline phase and utilize the property that each AP defines a subset of locations covered by this AP. However, the procedure to do clustering is quite different from the methods in []. Nearest Neighbour (NN) rule are utilized in this paper. A location might not fall into the cluster defined by an AP, even the location is covered by this AP. The main contribution of this paper is that the proposed algorithm can give competitive clustering performance, the localization accuracy and efficiency given by the clustering method can also be improved compared to the method of no clustering. The rest of the paper is organized as follows. In section II, the background knowledge for this paper is provided. In section III, the detailed algorithm procedures are described. In section IV, the details of the experiment setup are given. Experimental results and evaluation of the proposed algorithm are given reported in Section V. Finally, the conclusion and future work are given in section VI. II. BACKGROUND KNOWEDGE A. NN rule Given a set of n pairs,,,,,,, where representation takes values in a metric space X, and category takes value in the set,2,,. Each is considered to be the index of the category to which the individual belongs, and each is the outcome of the set of measurements made upon that individual. For convenience, we shall say belongs to when we mean precisely that the individual, upon which measurements have been observed, belongs to category. A distance metric d is defined on space X. When a new pair of observation, is given, we need to determine the value of by using the observation. can take value from set,,,, and we say,,,, where associates with, is the value for if d( x, x) = min d( x, x), i =,2,, n. () NN i The distance function, can be Euclidean function or any other user defined distance functions. The NN rule determines an observation belongs to which category. A mistake is made if, and we call this a misclassification. NN rule utilizes only the classification of its nearest neighbor. The remaining neighbors are ignored [4]. B. EM EM is a kind of machine learning algorithm based on Single-hidden ayer Feedforward neural Networks (SFNs) architecture. It has been proved to provide good generalization performance at extremely fast learning speed [5]. Following is the brief description about EM given by Huang [5]. For SFNs, the outputs with hidden nodes can be represented as y ( x ) = β g ( x ) = β G( a, b, x ), (2) N i i i i i i= i= Where, are the weights and bias connecting the input nodes and the hidden node, are the output weights connecting the hidden node and the output nodes, and,, is the activation function which gives the output of the hidden node with respect to the input vector. Hornik [5] proved that if the activation function is continuous, bounded and non-constant, then neural networks can approximate the continuous functions over compact input sets. eshno [6] further proved that the feedforward networks with a non-polynomial activation function can approximate continuous mappings. However, in real applications, the training input sets are always finite. In order to be more applicable, Huang and Babri [7] shows that a SFN with at most N hidden nodes and with almost any nonlinear activation function can exactly learn N distinct observations. Suppose we have N arbitrary distinct training samples,,,2,,, we can represent the SFN for each sample as equation: N j i i i j i= y ( x ) = β G( a, b, x ), j =,2, N. (3) Now the above N equations can be written compactly as: Hβ = T, (4) where G( a, b, x) G( a, b, x) H =, (5) G( a, b, xn) G( a, b, xn) N T T β t β = and T =. (6) T T β m t N N m is called the hidden layer output matrix of EM; the column of is the hidden node s output vector with respect to inputs,,,, and the row of is the output vector of the hidden layer respect to the input vector of. Unlike the traditional training algorithms for neural network, which needs to adjust the input weights and hidden layer biases, Huang in [5] has proved that these parameters of SFN can be randomly assigned if only the activation function is infinitely differentiable. Thus, the hidden layer output matrix remains to be fixed once these parameters are randomly initialized. To train a SFN is simply equivalent to find a least solution of equation 4, which gives: norm( Hβ T) = min norm( Hβ T ). (7) S If the number of hidden nodes is equivalent to the number of samples N, is thus square and invertible according to [8]. The SFN can thus approximate these training samples with β 60

3 zero error. However, in most cases, the number of hidden nodes is much smaller than the number of training samples: <<N. is thus a non-square matrix and there may not exist,,, where i=,2,,, such that 0. In [9], it has been proved that the smallest norm least-square solution of Equation 4 is, where is the Moor- Penrose generalized inverse of H. III. AGORITHM FORMUATION The proposed algorithm can be divided into three main parts as shown in Figure : Selection of andmarks, Clustering based on NN rules, and ocalization by EM. A set of landmarks,,,, where is an observation/fingerprinting sample, will be selected based on the training data. For each landmark,,2,,, there will have only one cluster associated with it. Then for a new pair,, we need to find,,,, according to the NN rule. Since one fingerprinting sample is associated to one particular location l, thus we can further determine location l belongs to which cluster. Since there are more than one observations/fingerprinting samples for each location l, thus one location l might belong to several clusters due to the signal noise. Once all the locations have been clustered, the training data can be further divided into several subsets of training data according to their belonging clusters. Thus, the EMs associated with each cluster can be trained by utilizing the training data for that particular cluster. In order to simulate the real situation, the testing samples from testing data set will be fed into the trained EM one by one. For each testing sample, respective belonging cluster will be determined by the NN rule. Then respective trained EM for that cluster will be responsible for giving the estimated location associated with the observation. The testing procedure will be repeated for each observation until all the observations have been tested. Figure : Flow chart of the algorithm A. Selection of andmarks An Access Point (AP) can only be responsible for the data communication of a small area. For a large area, say a library, the data communication infrastructure consists of many APs deployed evenly across the whole area. For a small area dominated by one particular AP, the position of the strongest RSS for that AP should be the position or near the position of the AP. According to this property, we develop an algorithm to select these locations from the whole chunk of training data. The selection of the landmarks is based on the fingerprinting samples. We do not need to know where the APs are, since they are hidden sometimes. For each training location, there should have a dominated AP, which gives the largest mean RSS. Thus, there is a largest mean RSS of the dominated AP associating with each location, it can be represented as _,. Since the training locations are evenly distributed across the whole area. We can take a window operation to determine whether the centred location of the window has the largest among all the locations covered by the window. If yes, then the location will be determined as a landmark for further clustering. The RSS samples of the dominated AP for these landmarks will be recorded to the landmark database for further clustering. The outcomes for the selection of landmarks depend on the size of the window. It will be further discussed later. B. Clustering Based on NN rules Since the landmarks normally locate near the APs, thus they usually have large RSS for that particular AP. Therefore, the AP which dominates a landmark will be easily figured out by checking the RSSs of each AP. One AP will in charge of several channels in the data communication infrastructure. For each channel, there will have an allocated MAC address. Since we are using MAC address to identify different APs, we might measure similar RSS for several MAC addresses. The associated information with each landmark is thus these RSSs and MAC addresses of the dominated AP. They can be represented in mathematical form as,,,,,,, where,,, normally have similar values, since they are measured for different channels of the same AP. The Euclidean signal distance between each training sample and each landmark only in terms of the dominated AP will be calculated. Then we can use NN rule to determine its nearest landmark (or NN) for that sample. Thus, the samples are being clustered. The process for testing samples is the same. Thus, the locations associated with each sample are also being classified into different clusters. It should be noticed that one location may belong to several clusters. C. ocalization by EM The proposed localization algorithm by using EM is actually a regression method. The algorithm consists of offline and online phases. During the offline phase, an empirical database is built for each cluster. For each location, as long as it has one sample being classified into a cluster, all the samples associated with that location will be incorporated into that database for future EM training. Each entry of the database can be represented as,,,2,,, (n is the number of samples) in mathematical form. The vector are the inputs of the EM and the corresponding location vector are the outputs of EM. Thus, the EM can be trained as mentioned in Section II. The detailed steps are illustrated below: Step: Randomly assign values to hidden node parameters. 6

4 Step2: Calculate the hidden layer output matrix H. Step3: Calculate the output weight by: β = H, (8) where is the Moor-Penrose generalized inverse of H. For online localization, we only need to feed new measured RSS samples into the trained EM. The output given by EM is the estimated location. Figure 3: Selected andmarks with Window size=6m Figure 2: ayout and Distribution of Reference Points IV. EXPERIMENT The data used to evaluate the performance were collected at evel 6 of the Central ibrary of National University of Singapore. The size of the test-bed is approximately 59m 28m. The distance between adjacent points is normally.2m. Some may be larger than.2m, due to the skip of the table areas, which is not convenient for measurements. The layout of the area and the distribution of the reference points are shown in Figure 2. For each reference points, 60 samples were collected in total and 5 samples per direction. The data were further being divided into training data and testing data. For training data, the minimum distance of adjacent points is 2.4m. All the simulations are carried out in the MATAB 7. environment, which was running in an Intel Core i5, 2.4GHz CPU. V. RESUTS AND EVAUATIONS A. Result of andmark Selection Three different sizes of the window have been used to explore the relationship between the number of selected landmark and the window s size. It can be seen from Figure 3 to Figure 5 that the smaller the window size, the more landmarks will be chosen. There are two types of changes for RSS as shown in Figure 6, one is due to the multipath effect and noise, the RSS will fluctuate, although the samples are taken at the same location. Another kind of change is due to the path loss. Figure 4: Selected andmarks with Window size=0m Figure 5: Selected andmarks with Window size=0m For two samples of the same AP measured at two different locations, the sample which is measured nearer the AP will have larger RSS normally. If the distance between two reference locations is small, then the first kind of change will dominate the change of RSS, since the path loss is small 62

5 compared to the amplitude of the fluctuation of RSS. However, if the distance between two reference points is relatively large, the change caused by path loss will dominate the change of RSS as shown in Figure 6. Now it is easy to explain the above performance for different window size. When the window s size is 3m, the first kind of change will dominate the change of RSS instead of the path loss. Thus, unexpected landmarks will be selected. If the window s size is increased to 5m, more neighbor reference points for a location will be covered by the window, thus the path loss will dominate the change of RSS. It can give us suitable landmarks which are the real locations of each AP as shown in Figure 4. However, if the window s size keeps increasing, 2 potential correct landmarks (one landmark is centered in the window) might be covered by the same window at the same time. Unexpected loss of landmarks might occur in this case. testing locations. It is reasonable since they are treated by the same clustering procedure. Although there are some exceptions (as shown in Figure 7, some testing locations are a bit further away from the cluster), there are very few samples for these locations. This is due to the multipath effect and noise. If the amplitude of the fluctuation of RSS increases, the clustering performance will become worse. However, if the fluctuation is too large, there will also no other better solutions to provide acceptable localization service. C. Selection of the inputs for each EM The input variables for EM should be fixed so that the EM can be trained and used for localization. The input variables of the EM utilized in this paper are the original RSS samples. The performance of EM is explored by choosing different number of input variables experimentally. It can be seen from the Figure 8 that as we have more inputs, the better mean testing localization accuracy we will have. EM can also give more stable performance. However, if we analyse the performance of EM theoretically, we will find that there should be an upper bound for the number of input variables. The reason is that unnecessary input variables may bring unnecessary hidden nodes parameters. This will cause EM to be unstable and be easily over-fitted. This effect did not occur in our experiment. This might due to the number of our input variables is still smaller than this upper bound. For simulations, we choose the number of input variables experimentally. Figure 6: Two Types of RSS Change Figure 8: Testing Accuracy regards Number of input and hidden nodes Figure 7: One of the clusters, cluster 2. B. Result of Clustering Figure 7 demonstrates the performance of the clustering method by choosing an example cluster, e.g., cluster 2. It can be seen from Figure 7 that the area covered by the clustered training location matches the area covered by the clustered D. Final ocalization Accuracy and Comparison In order to evaluate the performance of our clustering method, the performance of no clustering method is also being explored. In this experiment, the parameters (e.g., the number of hidden nodes of EMs) of the clustering method are being adjusted to give the best localization accuracy regardless the time consumption. The parameters of the no clustering method are chosen to give two different kinds of performance. One kind of parameters selection is such that the method can consume the same time as the clustering method. Then the corresponding localization accuracies of the two different methods are being compared as shown in Figure 9. Another kind of parameters selection is such that the no clustering 63

6 method can give its best localization accuracy regardless the time consumption. Then the time consumption (Efficiency) is being compared with that of the clustering method as shown in Figure 0. Figure 9: Comparison of ocalization Accuracy Regards same time consumption Figure 0: Comparison of Time Consumption for the Best Accuracy It can be seen from Figure 9 that the clustering method can give an approximately m s better mean localization accuracy than that without clustering for the same time consumption. In terms of the best localization accuracy, it can be seen from Figure 0 that the mean localization accuracy can only reach approximately 4m at its best performance (The clustering method can give mean localization accuracy at m). However, the time consumption (or efficiency) is already approximately four times longer than the time consumed by the clustering method. It can be seen that the clustering method can give a better performance in terms of both of the efficiency and the localization accuracy. VI. CONCUSION AND FUTURE WORK More and more researchers are paying attention to the indoor localization solutions due to their wide applications in many areas recently. However, most existing works on indoor localization are proposed for small scale networks without the scalability consideration, which is an essential requirement in the real applications. In this paper, we have proposed a novel joint clustering and EM machine learning method to solve the scalability problem for indoor localization. It can be seen that the proposed method gives a better performance in terms of both the efficiency and the localization accuracy. In the future, the feature extraction of input variables may be explored, since it is an important part for the modelling of EM. REFERENCES [] Y. H. iu, Y. zheng, ocation, localization, and localizability: locationawareness technology for wireless networks, Springer, New York, 20. [2] P. Bahl and V. N. Padmanabhan, Radar: An in-building RF based user location and tracking system, In proceedings of IEEE Infocom, [3] W. Xiao, W. Ni, Y. K. Toh, Integrated Wi-Fi Fingerprinting and Inertial Sensing for Indoor Positioning, International Conference on Indoor Positioning and Indoor Navigation (IPIN), 20. [4] W. Meng, W. Xiao, W. Ni, and. Xie, Secure and Robust Wi-Fi Fingerprinting Indoor ocalization, 20 International Conference on Indoor Positioning and Indoor Navigation (IPIN), 20. [5] G.-B. Huang, Q.-Y. Zhu, C.-K. Siew, Extreme learning machine: theory and applications, Neurocomputing 70 ( 3) (2006) [6] W. Xiao, P. iu, W. Soh, Y. Jin, Extreme learning machine for wireless indoor localization, th Internation Conference on Information Processing and Sensor Networs, IPSN 202. [7] J. etchner, D. Fox, and A. amarca. arge-scale lcalization from Wireless Signal Strength. In Proc. of the Nat'l Conf on Artificial Intelligence (AAAI), pages 5-20, [8] Y.-C. Cheng, Y. Chawathe, A. amarca, and J. Krumm. Accuracy Characterization for Metropolitan-scale WiFi ocalization. In ACM MOBISYS, volume 5, pages , [9] A. amarca, J. Hightower, I. Smith, and S. Consolvo. Self-Mapping in 802. ocation Systems. In Proc. 7th Int'l Conff on Ubiquitous Computing (UBICOMP), pages Springer, [0] A. Haeberlen, E. Flannery, A. M. add, A. Rudys, D. S. Wallach, and. E. Kavraki. Practical Robust ocal ization over arge-scale 802. Wireless Networks. In IEEE ACM MOBI COM, pages 70-84, [] M. Youssef, A. Agrawala, ocation-clustering techniques for WAN location determination systems (2006), International Journal of Computers and Applications, 28 (3), pp [2] M. Borenović, A. Nešković, D. Budimir, Space partitioning strategies for indoor WAN positioning with cascade-connected ANN structures, International Journal of Neural Systems, 2 (), pp. -5, 20. doi: 0.42/S [3] T. King, T. Haenselmann, W. Effelsberg, On-demand fingerprint selection for 802.-based positioning systems, IEEE International symposium on a world of wireless, mobile and multimedia networks (WoWMoM 2008), 2008 [4] T. M. Cover, P. E. Hart, Nearest neighbour pattern classification, IEEE Transction on Information Theory, Vol. IT-3, No., pages 2-27, 967. [5] K. Hornik, Approximation capabilities of multilayer feedforward networks, Neural Networks 4 (99) [6] M. eshno, V.Y. in, A. Pinkus, S. Schocken, Multilayer feedforward networks with a nonpolynomial activation function can approximate any function, Neural Networks 6 (993) [7] G.-B. Huang, H.A. Babri, Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions, IEEE Trans. Neural Networks 9 () (998) [8] G.-B. Huang, D. Wang,. Yuan, Extreme learning machine: a survey, Int. J. Mach. earn. & Cyber,Vol 2, page 07-22, 20 [9] D. Serre, Matrices: Theory and Applications, Springer, New York,