Power Tags as Tools for Social Knowledge Organization Systems

Transcription

1 Power Tags as Tools for Social Knowledge Organization Systems Isabella Peters Abstract Web services are popular which allow users to collaboratively index and describe web resources with folksonomies. In broad folksonomies tag distributions for every single resource can be observed. Popular tags can be understood as implicit consensus where users have a shared understanding of tags as best matching descriptors for the resource. We call these high-frequent tags power tags. If the collective intelligence of the users becomes visible in tags, we can conclude that power tags obtain the characteristics of community controlled vocabulary which allows the building of a social knowledge organization system (KOS). The paper presents an approach for building folksonomy-based social KOS and results of a research project in which the relevance of assigned tags for particular URLs in the social bookmarking system delicious has been evaluated. Results show which tags were considered relevant and whether relevant tags can be found among power tags. 1 Introduction In the last years such web services became popular which allow users to collaboratively and intellectually index and describe web resources (e.g. bookmarks) with user-generated keywords, so-called tags. These performed tagging actions result in a folksonomy of this particular collaborative information service (CIS) [9]. The folksonomy of a CIS [4] F CIS can be defined as a tuple F CIS := (U,T,R,Y ) where U,T,R are finite sets of the elements user names U, tags T and resource identifiers R, and Y is a ternary relation between them, i.e., Y U T R whose elements are called tagging actions. The F CIS is composed of all personomies P 1...P n of a CIS called P CIS and all docsonomies D 1...D n of the CIS called D CIS. P CIS is defined as a multiset P CIS := (U,T,X) b where X U T and {(u,t) U T (u,t) X}, b N +. Isabella Peters Heinrich-Heine-University Duesseldorf, Universitaetsstr. 1, Duesseldorf, Germany isabella.peters@uni-duesseldorf.de 1

2 2 Isabella Peters P CIS becomes P u by substituting X with X u where u U. D CIS is defined as a multiset D CIS := (T,R,Z) b where Z T R and {(t,r) T R (t,r) Z}, b N +. D CIS becomes D r by substituting Z with Z r where r R. It follows that F CIS P CIS D CIS and P u P CIS and D r D CIS. The differentiation between F CIS and D CIS (and D r respectively) and the notion of D CIS as multiset is important for our later discussion of tag distributions. What is more, folksonomies may allow the multiple assignment of a single tag to a particular resource, so that we can speak of a broad folksonomy in this case. In contrast to this, narrow folksonomies only allow the addition of new tags to the resource [16]. Typical examples for broad and narrow folksonomies are delicious.com and youtube.com. The major difference between broad and narrow folksonomies is that in broad folksonomies tag frequency distributions on the resource level can be observed. Having this information about folksonomies at hand the paper aims at discussing two research questions (RQ) in detail: RQ1: How to build social knowledge organization system (KOS) (automatically) by using folksonomies? RQ2: Are Power Tags reflecting collective user intelligence and as such are most relevant for a resource? 2 Related Work Several studies have been concerned with automatic construction of KOS by using folksonomies refering to this idea as emergent semantics [13]. Early work of Schmitz et al. [12] make use of association rules to detect related tags. Schmitz [11] discusses how to build an ontology of Flickr tags with a statistical model for subsumption based on tag co-occurrences but does not use broad folksonomies. Marinho et al. [7] are using frequent itemset mining to enhance ontologies with folksonomic tags. Mika [8] compares tag clusters either built from the links of tagger and tags or the links of tags and resources. His evaluation shows that tag-resource connections are appropriate for concept mining, whereas the tagger-tag connections can be used for automatic detection of hierarchical relations (e.g. broader-narrower concept) between tags. In our approach we combine the ideas of [8] for extracting emergent semantics of folksonomies and consider tags which have been tagged for a single resource (D r ) and then tags which have been indexed for many resources (tag-resource link; D CIS ). Differing from [7] and [8] we conduct the first step on the docsonomy level D r and only respect high-frequent tags of D CIS for processing. 3 Using Relevant Power Tags for Constructing Social KOS In this section we discuss the aforementioned research questions. RQ1 will be answered theoretically whereas RQ2 will be answered by means of empirical data.

3 Power Tags as Tools for Social Knowledge Organization Systems RQ1: Folksonomy-based social KOS In this paper we define social KOS as a collaboratively built knowledge representation tool with natural-language terms. Tasks of social KOS are both, finding of appropriate concepts and finding of paradigmatic structures in folksonomies, e.g. hierarchies. The docsonomy reflects via tags the users collective intelligence [15] in giving meaning to this resource. In broad folksonomies most popular tags for a resource can be determined and it is widely assumed that popular tags are the most important tags for a resource [18] as they reflect an implicit concensus of users for describing it. To give credit to the theories of collective intelligence we call these most popular tags power tags [10]. In order to establish a social KOS we need candidate tags for the finding of concepts and relations. For this purpose we propose power tag pairs. Let us explain the idea of power tag-based tag pairs with an example. A resource App Inventor for Android was saved in the social bookmarking system delicious and was indexed with different tags, e.g. android, google, development (see figure 1). In order to find tag pairs for social KOS we must first define the power tags of this resource reflected in D r. The determination of power tags depends on the tag frequency distribution of a digital resource. The basic assumption is that different distributions of tags may appear in folksonomies: a) an inverse Power Law distribution, a Lotka-like curve, b) an inverse logistic distribution [14], and c) other distributions. A power law has the form f (x) = x C a where C is a constant, x is the rank of the tag relative to the resource, and a is a value ranging normally from about 1 to about 2 [2]. If this assumption is true, we see a curve with only few tags at the top of the distribution, and a long tail of numerous tags on the lower ranks on the right-hand side of the curve. The discussions about collective intelligence are mainly based on this observation: the first n tags of the left hand side of the power law reflect the collective user intelligence. The inverse logistic distribution shows a lot of relevant tags at the beginning of the curve ( long trunk ) and the known long tail. This distribution follows the formula f (x) = e [ C (x 1)] b where e is the Euler number, x is the rank of the tag, C is a constant and the exponent b is approximately 3 [14]. In comparison with the power law the inverse logistic distribution reflects the collective intelligence differently. The curve shows a long trunk on the left and a long tail of tags on the right. Since all tags in the long trunk have been applied with similar (high) frequency, all left-hand tags up to the turning point of the curve should be considered as a reflection of collective intelligence. For the determination of power tags we have to keep in mind both known tag distributions. If the resource-specific distribution of tags follows the inverse power law, the first n tags are considered as power tags. The value of n must be determined empirically. If the tag distribution forms an inverse logistic distribution, all tags on the left-hand side of the curve (up to the turning point) are marked as power tags.

4 4 Isabella Peters The concrete processing of power tags works as follows: the first step is to determine power tags for each docsonomy D r of the CIS. According to the above explanations, two different tag distributions may appear which each identify different numbers of power tags (we call them power tags I). Since these power tags I are important tags in giving meaning to the resource, they have to be processed in the next step. Now, the n numbers of power tags I should each be investigated regarding their relationships to other tags of the whole database D CIS - in other words, a calculation of co-occurrence is carried out for the power tags I. This calculation produces again specific tag distributions, where we can determine power tags as well (we call them power tags II). These new power tag I- and II-pairs are now the candidate tags for the detection of paradigmatic relations since their connection to the power tags I seems to be very fruitful. The tags for the resource displayed in figure 1 form a power law distribution with a sharp decline between rank 1 and rank 2, so that only one tag is considered as power tag I, here: android. For this tag we examine co-occurrences with all other tags of the database. An exemplary search in delicious for the tag android results in the co-occurring tags displayed in figure 2 and says that the tag android appears together with the tag mobile in 70,614 resources. Those co-occurring tags follow an inverse-logistic distribution (see figure 2). The first n tags (say n = 7) are considered as power tags II. The pairs of power tags I and power tags II are now the source for building a social KOS. Fig. 1 Tag distribution for the delicious-resource appinventor.googlelabs.com/about Up to this point, the construction of a social KOS can be carried out automatically. But, we state that it is not sufficient to calculate co-occurrences to obtain semantic relations which form the basis for social KOS. Co-occurrences can only work as indicators that some kind of relations may exist between two concepts. Yet, to determine which kind of relation is at hand, we have to add intellectual analyses from people like users or administrators of the CIS.

5 Power Tags as Tools for Social Knowledge Organization Systems 5 Fig. 2 Co-occurrence distribution for the tag android Figure 2 shows that the tag android is frequently combined with the tags mobile, google, iphone, development, apps, phone, and software. The tag pair android-mobile shows an associative relation, when we rely on semantic relations used in thesauri [1], or a more specific relation like used in, when we consider ontology-like relations. All found relations are shown in table 1. Table 1 Descriptor set for descriptor android extracted from tags abbreviation descriptor name of relation type of relation Broader Term phone hierarchy, part of meronymy BT software hierarchy, is a hyponymy Related Term mobile related term, used in association RT google related term, produced by association RT iphone related term, is competitor association RT development related term association RT apps related term association The given assumptions result in an entry of controlled terms, e.g. descriptors, with which we now can build a social knowledge organization system. Here we are able to answer research question one. Yes, it is possible to construct a social KOS by using the users tags. But: no, it is not possible to automate this process to the full extend. We are always in need of some intellectual endeavors to identify the type of a relation between tags. 3.2 RQ2: Relevance of Power Tags If we presuppose that power tags are the reflection of the collective user intelligence and along with it the most relevant tags for the resource we have to prove that this

6 6 Isabella Peters assumption is true by making use of empirical data. As tagging behavior and tags are highly subjective and error-prone (regarding spelling etc.) as well as such facts cannot be accepted in the construction of social KOS, the question arises whether collective intelligence of users is capable of ironing out too personal and erroneous tags so that all users are satisfied with high-frequent power tags as best matching keywords for the resource. That is why we have conducted a survey with 20 information science students at the HHU Duesseldorf as assessors to determine the relevance of power tags for resources. We downloaded 30 resources from delicious in February 2010 which fulfilled two preconditions: 1. resources must be tagged with the tag folksonomy to guarantee that students have the technical knowledge to judge the relevance of the tags and can as such serve as independent domain experts, 2. resources must be tagged from at least 100 users to guarantee that broad tag distributions with recognizable characteristics are downloaded. Each assessor received each resource and the tags assigned to it, but the assessors did not know which rank that tag gained from the delicious-users. To be able to judge the relevance of the tags the assessors had access to the resources. Following the common procedure in information retrieval system evaluation the students indicated each and every tag of the resource with 1 for a relevant tag and with 0 for a non-relevant tag. In this context relevant means that a particular tag describes the content of the resource in an appropriate manner. Please note that we worked with binary relevance judgements, similar to the now classic methods of Cranfield and TREC [17]. Moreover, the relevance judgements of the assessors were considered objective and neutral as assessors are not regarded as system users (and along with it not as tagging users) in Cranfield-like retrieval system test settings [5]. Receiving 20 assessors relevance judgements we constructed tag relevance distributions for every resource. After this we defined which ratio of assessors judgements determines the relevance of a resource s tag, which was 50%, and we extracted the Top 10-tags for every resource inclusive tag frequency from delicious. The process for an example is displayed in figure 3 where column relevance frequency reflects how many assessors considered this tag relevant and where column tag frequency shows how many delicious-users used this tag for indexing. Pearson correlation is also given for the two columns. The average value of Pearson correlation for 30 resources was The summarization of all assessor judgements can be found in figure 4. It presents in how many cases a tag on rank was judged relevant by the majority of the assessors. The evaluation shows a clear result: in most cases only the first two tags, which are the power tags, have been considered relevant from the assessors. During the analysis of the assessors relevance judgements it was noticeable that they show a slight bias to German tags as the mother tongue of most students was German and therefore German tags were more relevant for them. Students also preferred the original spelling of proper nouns, e.g. the tag LibraryThing was considered more relevant than librarything.

7 Power Tags as Tools for Social Knowledge Organization Systems 7 Fig. 3 Are power tags the most relevant tags for a resource? Example Fig. 4 Assessors relevance judgements for Top 10-tags of 30 delicious-resources 4 Discussion & Outlook We presented an approach to construct social KOS on the basis of broad folksonomies. It could be shown that tag co-occurrence analyses of power tags I and power tags II produce fruitful tag pairs which may work as candidate tags for further semantic enrichment of folksonomies via defined relations between tags. Although the extraction of power tag pairs can be carried out automatically the need for their intellectual processing to assess the the type of tag relation cannot be denied. So, folksonomies deliver concept candidates for social KOS which make use of intellectually built tag structures. We have also shown that power tags do reflect the collective user intelligence in giving meaning to the resources as assessors judged Top 2-delicious-tags relevant in most cases. As such, power tags work as community-proved controlled vocabulary which can be utilized for social KOS. The method could be optimized with a combination of morphologically - not semantically - similar tags (e.g. folksonomy and folksonomies ), the combination of phrase tags (e.g. information and architecture to information architecture ), the unifaction of multi-term tags (e.g. social classification, socialclassification, and SocialClassification ) and the joining of cross-language synonyms (e.g. classification and Klassifikation ). A relevant aspect for building social KOS based on folksonomies is still open for research although it has yet been widely discussed: the stability of tag distributions

8 8 Isabella Peters (amongst others: [3; 6]). Research shows that tag distributions remain stable after a certain point in time, meaning that the shape of the distribution does not change and that no rank permutations in tags appear anymore. The construction of social KOS is only then able to start when the resource-specific tag distributions achieve stability. The determination of the point of stability is due to our future work. Acknowledgements The author would like to thank her colleagues and students from the HHU Duesseldorf, the participants of the 34th Annual Conference of the German Classification Society and the reviewers for their helpful comments. The presented work is funded by the German research fund DFG (STO 764/4-1). References 1. Aitchison J, Gilchrist A, Bawden D (2000) Thesaurus construction and use. Europa Publishing, London, New York 2. Egghe L, Rousseau R (1990) Introduction to informetrics. Elsevier, Amsterdam 3. Halpin H, Robu V, Shepherd H (2007) The complex dynamics of collaborative tagging. In: Williamson C L, Zurko M E, Patel-Schneider P F et al. (eds.) Proc. of the 16th Int. WWW Conf., Banff, Alberta, Canada. ACM, New York, pp Hotho A, Jaeschke R, Schmitz C et al (2006) Information retrieval in folksonomies: Search and ranking. Lecture Notes in Computer Science 4011: Kamps J, Lalmas M, Larsen B (2009) Evaluation in context. Lecture Notes in Computer Science 5714: Kipp M, Campbell D (2006) Patterns and inconsistencies in collaborative tagging systems: An examination of tagging practices. In: Grove A (ed.) Proc. of the 17th Annual Meet. of the Am. Soc. for Inf. Sci. and Tech., Austin, Texas, USA (CD-Rom) 7. Marinho L B, Buza K, Schmidt-Thieme L (2008) Folksonomy-based collabulary learning. Lecture Notes in Computer Science 5318: Mika P (2007) Ontologies are us: A unified model of social networks and semantics. J. of Web Semant. 5 (1): Peters I (2009) Folksonomies: Indexing and retrieval in Web 2.0. De Gruyter Saur, Berlin 10. Peters I, Stock W G (2010) Power tags in information retrieval. Libr. Hi Tech 28(1): Schmitz P (2006) Inducing ontology from Flickr tags. In: Carr L, De Roure D, Iyengar A et al. (eds.) Proc. of the 15th Int. WWW Conf., Edinburgh, Scotland. ACM, New York 12. Schmitz C, Hotho A, Jaeschke R et al (2006) Mining association rules in folksonomies. In: Batagelj V, Bock H, Ferligoj A et al. (eds.) Data Science and Classification, Springer, Berlin, Heidelberg, pp Staab S, Santini S, Nack F et al (2002) Emergent semantics. Intell. Syst. 17(1): Stock W G (2006) On relevance distributions. J. of the Am. Soc. for Inf. Sci. and Tech. 57(8): Surowiecki J (2005) The Wisdom of crowds. Anchor Books, New York 16. Vander Wal T (2005) Explaining and showing broad and narrow folksonomies. Available via Cited 13 Aug Voorhees E M (2001) The philosophy of information retrieval evaluation. Lecture Notes in Computer Science 2406: Weiss A (2005) The power of collective intelligence, networker 9(3):16-23