A Study of the Correlation between the Spatial Attributes on Twitter

Transcription

1 A Study of the Correlation between the Spatial Attributes on Twitter Bumsuk Lee, Byung-Yeon Hwang Dept. of Computer Science and Engineering, The Catholic University of Korea 3 Jibong-ro, Wonmi-gu, Bucheon-si, Gyeonggi-do, 2-73, Korea {bslee, byhwang}@catholic.ac.kr Abstract In recent years, much attention has been given to the topics on social network analysis. Previous research has shown that each Twitter user can work as a sensor to detect the target events such as the earthquakes. They used the spatial attributes on Twitter to estimate the location of the target event. Although the precision of the location information may affect the performance of the estimation, they did not consider its reliability. In this paper, we investigate the correlation between the profile locations on Twitter and the GPS coordinates in tweets. A text-based grouping method is applied to the spatial attributes to cluster them with considering of the administrative districts. The analysis result shows that nearly 5% of users post the most of their tweets in the profile locations while 3% of users, who may have high mobility in a wide range, do not have any tweets in their locations. I. INTRODUCTION Social network services make it possible to connect people who share interests and activities across political, economic, and geographic borders. They contributed to attract support from the people around the world during the democratic uprising in the Middle East [1]. This shows the social network services reflect and react to the real world issues actively. Twitter has explicit social structures among users and tweets contain personal life stories, social issues, and events. These characteristics let the service can be used in various research fields. For our research, we can detect an event or an incident with exact location from tweets because, now, many people access Twitter with their smart mobile devices. Twitris [2] system by Meenakshi et al. and Toretter system [3] by Takeshi et al. show us that it is possible to detect an event from Twitter data. They regards each user on Twitter as a sensor that observes events in their location and reports it on Twitter in real-time. The main difference between the two systems is their approach to detect an event. While Twitris extracts frequently used terms based on the TFIDF algorithm from the tweets in a given time gap at a specific location, Toretter observes the trends of the given terms. However, both systems did not discussed about the credibility of the location information although it may affect the performance of their systems. A study has investigated the information credibility of news propagated through Twitter [], but they do not consider the credibility of the location information that was provided in free text by users. Each user can define the location in their profile and their tweets may have GPS coordinates if the user posts tweets on the smart mobile device. This paper presents an investigation result of the correlation between the location in Twitter profile and the GPS coordinates of their tweets. The result shows that nearly 5% of users post tweets in their profile locations, and they have 3 major spots for posting tweets. We can evaluate the reliability of the location information in the Twitter profiles, and we expect that it can be used to determine the weight factor for the event detection systems [5, ] on Twitter. The contributions of this paper are summarized as follows: We present a text-based grouping method, which is applied to the spatial attributes to cluster them with considering of the administrative districts. This paper explains the result as a preliminary study about the correlation between the user location in Twitter profile and the GPS coordinates of Tweets. This analysis result will provide the degree of the information reliability on Twitter for the follow up research. This paper is organized as follows: In the next section, we explain the previous researches using Twitter data, followed by the introduction of the data collection and the experiment method in Section 3. In Section, we present the analysis results. Finally, we discuss the result and conclude the paper in Section 5. II. EVENT DETECTION SYSTEMS In this paper, we investigate the correlation between the location information in user profile on Twitter and the GPS coordinates of their tweets. Nevertheless, we devote this section to related works about the event detection system such as Twitris and Toretter because this research is a preliminary study for the event detection system. These systems detect events that are visible through tweets such as earthquakes, car accidents, or fires. Meenakshi et al. developed Twitris [2], a platform for analyzing and extracting the summaries of citizen perceptions with time, space, and theme. This system used the TFIDF algorithm to extract popular terms in a day. In terms of location, they regarded the registered location in the user profile as an approximation for the current location of a tweet. The contribution of their research was that they presented a new paradigm in browsing citizen sensor observation in three dimensions: time, space, and theme. Figure 1 illustrates the Twitris user interface that has effective browsing of the when, where, and what slices of social perceptions behind an event.

2 Fig. 1 Visualization components of Twitris Toretter [3] by Takeshi et al. is a system that detects earthquakes by observing two predefined terms: earthquake and shaking. The Kalman filter and the Particle filter were applied to the spatial attributes on Twitter for location estimation of the event. Figure 2 shows the actual center, the estimated median, and the estimated center. The distance between the actual center and the estimated center was fairly close to show the efficiency of the location estimation algorithm. Moreover, the alert of the system was far faster than the rapid broadcast of announcement of Japan Meteorological Agency. limitation is the maximum length and it is 3 characters. When we use the profile locations, it is needed to refine the information because they are not normalized or geocoded in any way. Figure 3 shows the profile locations provided freely by users in different languages. Many users in Figure 3 provided fairly good information to use, and some provided the exact addresses or the GPS coordinates. However, we can find some meaningless information like darangland :) or Earth. The user id provided two locations; one is Gold Coast Australia and the other one written in Korean is the name of the administrative district in Seoul, Korea. In this case, we do not know which the current location of the user is. Fig. 2 Earthquake location estimation based on tweets Both papers proposed the event detection system that considers time and space. However, they did not concern about the reliability of the spatial attributes on Twitter. We present the analysis result on this topic in this paper as a preliminary study for developing an event detection system. Fig. 3 Locations in user profiles on Twitter Some tweets contain the GPS coordinates when users post tweets using their smart mobile devices. The GPS coordinates can be used as the credible source to identify the actual place for posting the tweet. If a tweet describes an event or an incident around the user, we can regard the GPS coordinates in the tweet as the place where the event is occurring. Figure is a sample collection of tweets with the GPS coordinates. In Figure, we found out some tweets mentioned about their current locations and those are the same places of the GPS coordinates. III. DATA COLLECTION We focus on the spatial attributes on Twitter in this paper. This section describes what the spatial attributes are and how we refine them for our experiment. A. Spatial Attributes on Twitter There are three sources of the spatial attributes: the locations in user profiles, the GPS coordinates of the tweets, and the places mentioned in tweet contents. We consider the first two attributes in this paper because our goal is to study the spatial attributes related to the response to an event. The locations in the user profiles (profile locations) are free text information written by users in their profiles. The only Fig. A sample collection of tweets with GPS coordinates B. Text-based Grouping Method In order to investigate the correlation between the profile locations and the GPS coordinates in tweets, we have to choose well-defined locations from the user profiles. Since we

3 want to cluster the locations on Twitter with consideration of the administrative districts, each location has to be transformed into the name of the district. The text-based grouping method is applied to the locations. We simply group by the name of cities, but we divide the locations in the metropolitan cities into the relatively small districts because these cities are too large and the populations are extremely high. The first step of our refinement is to collect and to select the appropriate users. Due to the changed policy of Twitter, we collect the users with crawler that explores the every followers of the given seed user. We collected more than 5, Korean Twitter users and we select only about 3, users who had well-defined profile locations. We had to remove many users from our data collection because of the vague (e.g. my home) and insufficient (e.g. Earth, Seoul, or Korea) information. In addition to that, we needed the users who have the GPS coordinates in their tweets. Although we gathered,, tweets from the Korean users but only 2, tweets had the GPS coordinates. As Meenakshi et al. mentioned in their paper [2], we faced the lack of GPS coordinates in the tweets, too. Due to this problem, most of our users were eliminated and, finally, 1, users were left with the well-defined profile locations and the GPS coordinates in their tweets. Fig. 5 A sample XML file of Yahoo API Next step is the transforming the location information both the profile locations and the GPS coordinates in tweets into the names of the cities or districts. We used Yahoo API [7] for this work. The result set in XML format has four elements under the <location> element; the four elements are <country>, <state>, <county>, and <town>. Figure 5 is a sample XML response for a query with the latitude and longitude Since we collected the data only from the Korean Twitter users, what we needed from the XML file were <state> and <county> elements. For the third step, we made a text string for each tweet with user id, profile location, and tweet location. Some example strings are shown in Table I. The first row in the Table I shows the properties of the strings, and the sharp (#) is a delimiter for each property. These strings were originally written in Korean but we changed them in English for this paper. In the strings, the term -gu means district and -si means city in Korean. TABLE I. EXAMPLE STRINGS FOR LOCATION INFORMATION User id#state in profile#county in profile#state in tweet#county in tweet #Seoul#Yangchun-gu#Seoul#Seodaemun-gu #Seoul#Yangchun-gu#Seoul#Jung-gu #Seoul#Yangchun-gu#Seoul#Jung-gu 71#Gyeonggi-do#Uiwang-si#Gyeonggi-do#Uiwang-si 71#Gyeonggi-do#Uiwang-si#Gyeonggi-do#Uiwang-si 71#Gyeonggi-do#Uiwang-si#Gyeonggi-do#Seongnam-si Finally, we merged the same strings in the list and ordered them by the number of the merged strings. For example, User in Table I has strings and of them are same ( ). We merged them and indicated the number of the merged strings. Table II shows the list after the last step. This approach lets us to group the locations by the administrative districts. If there is a string that the profile location and the tweet location are matched, we call it the matched string. To investigate the correlation between the spatial properties on Twitter, the place of the matched string in the ordered list is important for us. Thus, we categorized a user into the Top-k group when the matched string is placed k th in the list. For example, the matched string of the user is #Seoul#Yangchun-gu#Seoul#Yang chun-gu () and it is placed first in the list. Thus, this user is categorized into the Top-1 group. The user 71 s matched string is 71#Gyeonggi-do#Uiwang-si#Gyeo nggi-do#uiwang-si (2) and this user is in the Top-2 group because the string is placed second in the list. TABLE II. EXAMPLE OF MERGED AND ORDERED STRINGS User id#state in profile#county in profile#state in tweet#county in tweet (n) () (3) #Seoul#Yangchun-gu#Seoul#Jung-gu (2) #Seoul#Yangchun-gu#Seoul#Seodaemun-gu (1) (3) 71#Gyeonggi-do#Uiwang-si#Gyeonggi-do#Uiwang-si (2) 71#Gyeonggi-do#Uiwang-si#Gyeonggi-do#Seongnam-si (1) What we could know from Table II is that the user posts a half of his/her tweets at the profile location while the user 71 has another place for posting tweets instead of the profile location.

4 IV. ANALYSIS RESULT In this section, we describe the analysis results to investigate the correlation between the spatial attributes on Twitter. As we mentioned above, there are 1, users left after the refinement process, and we classified the users into the Top-k groups based on the place of the matched string in the ordered list Fig. The average number of tweet locations in each group For the first result, the average number of the administrative districts for each Top-k group has is shown in Figure. Top-1 group has 3. places for posting tweets in average and the numbers of the places are increased. In other words, the correlation between the profile location and the posting location for tweets is decreased as the user has more places for posting tweets. The interesting fact can be found in the category named None. There are 33 users in this category who do not have any matched strings at all, and they have 2.5 districts in average. There would be many possible reasons. One possible scenario is that the users may provide their hometown location for the profile, but they usually stay outside for work and return home late only for sleep. Also they may stick in a specific place for a long time, and their mobility range may not be wide Fig. 7 The number of users in each group Figure 7 depicts the number of users in each group. More than % of all users are in the Top-1 group and Top-2 group. If we assume that many users provide their residential place.2.1 Top-1 Top-2 Top-3 Top- Top-5 Top-+ None Top-1 Top-2 Top-3 Top- Top-5 Top-+ None for the profile location, we could say that nearly half of all users post tweets in their hometown. However, there are still about 3% of all users who do not have any tweets in their profile locations although their profile locations are explicit and well defined. Finally, we calculated the average number of districts for posting tweets for all users with regarding of the number of users in each group, and they have tweet locations in average. V. CONCLUSION As we described in this paper, we investigated the correlation between the locations in the profiles on Twitter and the GPS coordinates of tweets. A text-based grouping method was applied to the tweets to cluster them with consideration of the administrative districts. Our analysis result shows that nearly 5%of all users post tweets in the profile location while about 3% of all users do not have any tweets posted in the profile locations. According to the result, we have to consider the reliability of the spatial attributes to avoid the performance deterioration when we use them for estimating location of an event. In other words, we can use the analysis result of this paper to determine the weight factor for the location information, and it might be helpful to improve the performance for the event location estimation. For the future work, we will apply this result to the existing event detection systems to confirm that our result will improve their performance in terms of the event location estimation. It is hoped that this paper provides some insight into the reliability of the location information offered voluntary in free text on Twitter. ACKNOWLEDGMENT This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (No ). REFERENCES [1] N. K. Chebib and R. M. Sohail, The Reasons Social Media Contributed to the 211 Egyptian Revolution, International Journal of Business Research and Management, vol. 2, issue 3, pp , 211. [2] M. Nagarajan, K. Gomadam, A. P. Sheth, A. Ranabahu, R. Mutharaju, and A. Jadhav, Spatio-Temporal-Thematic Analysis of Citizen Sensor Data: Challenges and Experiences, In Proceedings of the th International Conference on Web Information Systems Engineering, ser. Lecture Notes in Computer Science, vol. 52, pp , 29. [3] T. Sakaki, M. Okazaki, and Y. Matsuo, Earthquake Shakes Twitter Users: Real-time Event Detection by Social Sensors, In Proceedings of the 19th International Conference on World Wide Web 2, pp. 51-, 2. [] C. Castillo, M. Mendoza, and B. Poblete, Information Credibility on Twitter, In Proceedings of the 2th International Conference on World Wide Web 211, pp. 75-, 211. [5] M. Mathioudakis and N. Koudas, TwitterMonitor: Trend Detection over the Twitter Stream, In Proceedings of the 2 International Conference on Management of Data, pp , 2. [] H. Becker, M. Naaman, and L. Gravano, Beyond Trending Topics: Real-World Event Identification on Twitter, In Proceedings of the 5th International AAAI Conference on Weblogs and Social Media, pp. 3-1, 211. [7] Yahoo! Korea - Open API,

5 Dataset Korean Dataset 52,2 Korean Twi2er users 11,139,92 Tweets Search API Lady Gaga Dataset 2,,9 Users 7,7 Tweets Streaming API Lee & Hwang, STIR, ICDE Average number of tweet locaons in each group Top-1 Top-2 Top-3 Top- Top-5 Top-+ Hwang, STIR, 2 2 Lee & ICDE Number of users in each group (percentage) Number of tweets in each group % % 9.73 % % 7. % % 29.1 % % 2% 11.3 % 2. % 1.51 % 1.15 % 1. % % Top-1 Top-2 Top-3 Top- Top-5 Top-+ None Hwang, STIR, 2 3 Lee & ICDE % 2.9 % 1.3 % 2%. %.77 % 5.57 % % Top-1 Top-2 Top-3 Top- Top-5 Top-+ Lee & Hwang, STIR, ICDE 2 Number of users in each group (percentage) Top- 1 Top- 2 Top- 3 Top- Top- 5 Top- + None Korean Dataset Lady Gaga Dataset Lee & Hwang, STIR, ICDE 2 5 Average number of tweet locaons in each group Top- 1 Top- 2 Top- 3 Top- Top- 5 Top- + Korean Dataset Lady Gaga Dataset Lee & Hwang, STIR, ICDE 2