LECTURE 12. Web-Technology

Transcription

1 LECTURE 12 Web-Technology

2 Household issues Course evaluation on Caracal o o Between and Assignment 3 deadline extension No lecture/practice on Friday 30/03/18 2

3 Adaptive Web 3

4 Size of the Web - Content # of Web-hosts: > 1 billion # of Web-pages: > 55 billion 80% of Web-content generated by users: Daily growth: o 500 Million Tweets o Half a Million Hours of YouTube videos o 3.6 Billion Instagram Likes o 4.3 Billion Facebook messages

5 Size of the Web - Users

6 Size of Web - Activities Searching and Browsing Reading, listening, watching Shopping and banking Communicating and exchanging information Working and having fun 6

7 The big question is Does one size fit all? 7

8 An alternative: Adaptive systems Adaptive information systems attempt to treat different users differently Adaptive Web: o combines principles of Artificial Intelligence (formal models of user, task, domain, ) and Human Computer Interaction (interactive interfaces with dynamically personalized components) o builds systems that provide personalized experience by supporting users in their online information tasks Collects information about individual user Adaptive Web System Provides adaptation effect User Modeling side User Model Adaptation side Classic loop user modeling - adaptation in adaptive systems

9 Evolution of personalization issue Starting late 1970s s: research topic o What are the principles, approaches, methodologies Starting late 1990s: practical topic o What are the applications Starting late 2000s: commercial topic o What are the revenues, KPIs Starting recently: ethical topic o Privacy and personal data policies o Filter bubble o Targeted (political) advertisement 9

10 What can be modelled by a user model? Knowledge about the content and the system Short-term and long-term goals Interests, preferences and needs Navigation / action history User category, background, profession, language, capabilities Platform, bandwidth, context the exact content of a user model varies across systems and tasks they support

11 What Can be Adapted? Adaptive Search Systems o tailor search results Adaptive recommender systems o Suggest new information items Adaptive Hypermedia Systems o adaptive presentation o adaptive navigation support Adaptive news systems o Filter out irrelevant news or reorder news feeds Adaptive GUIs o menu adaptation o dialog form adaptation Adaptive Educational Systems o adaptive course sequencing o adaptive problem solving support...

12 What personalization can help solving Information overload o when the amount of input to a system exceeds its processing capacity o Decision makers have limited cognitive processing capacity o Consequently, information overload causes reduction in decision quality. Lost in a hyperspace o A phenomenon of disorientation experienced by users reading and navigating hypermedia documents Getting right information at the right time o Different users are different o Same users are different at different times There is no silver bullet adaptive technology o Adaptation itself is not a silver bullet technology 12

13 Information Tasks Personalization Technologies Web search Adaptive search (IR, from 1980) o Use word-level profile of interests and remedial feedback to adapt search and result presentation Web browsing Adaptive hypermedia (HT, ITS, from 1990) o Use explicit domain models and manual indexing to deliver a range of adaptation effects to different aspects of user models Web shopping Web recommenders (AI, ML, from 1995) o Use explicit and implicit interest indicators, clickstream analysis/log mining to recommend best resources for detected user interests o Content-based recommenders o Collaborative recommenders o Hybrid recommenders

14 Adaptive search 14

15 Why Search Personalization? Different users need different documents in response to the same query o Ajax, Jaguar, Relevance is not enough if the volume of data is high o With the growth of information, even a good query can return thousands of "relevant" documents Personalization is an attempt to find the most relevant documents using information about user s: o goals, o knowledge, o preferences, o navigation history, etc.

16 Adaptive Search How can the search process be adapted to the user? How can we model the user in adaptive search? Which adaptation technologies can be applied?

17 How can Search be Adapted? Results Query Search Engine User profile User profile User profile Before search During search After search

18 Modeling Users in Adaptive Search Most essential feature: user interests Observing user document selection, adaptive IR systems build profile of user interests Keyword-level modeling o A long list of keywords (terms) in place of a domain model o User interests are modeled as weighted vector or terms o More advanced systems use several profiles for different domains or timeframes

19 Keyword-based User Profiles

20 Before: Query Expansion User profile is applied to add terms to the query o Popular terms could be added to introduce context osimilar terms could be added to resolve indexer-user mismatch o Related terms could be added to resolve ambiguity oworks with any IR model or search engine

21 During The user profile is used to organize the results of the retrieval process o Retrieve the most interesting documents o Filter out irrelevant documents In this case the use of the profile adds an extra step to processing Extended profile can be used effectively o Demographics o Location o Social circle

22 After Re-ranking of search results is a typical approach for post-filtering o Each document is rated according to its relevance (similarity) to the user or group profile o This rating is fused with the relevance rating returned by the search engine o The results are ranked by fused rating Annotation of search results o Results are provided with visual cues encoding useful information o Adaptive navigation approach

23 Google adaptive search Milestones: o March, 2004 beta o April, 2005 non-beta extra service o November, 2011 part of normal search o October, 2009 social search added Authenticated users Google user profile Non-authenticated users User profile contains: o Location o Search History (queries and clicks) o Web History (Google tracking scripts from adsense and adwords, Chrome, Android,.) o Social Networks o History from a Multitude of Google services o gender, age, languages o Topics 23

24 Personalized search experiment

25 Adaptive Hypermedia 25

26 Adaptive Hypermedia How hypertext and hypermedia can become adaptive? Which adaptation technologies can be applied? How can we model the user in adaptive hypermedia?

27 Why Adaptive Hypermedia? þdifferent people are different þindividuals are different at different times þ"lost in hyperspace We may need to make hypermedia adaptive where ðthere us a large variety of users ðsame user may need a different treatment at different times ðthe hyperspace is relatively large

28 What Can Be Adapted? Web-based systems = Pages + Links Adaptive presentation o content adaptation Adaptive navigation support o link adaptation

29 Classification Adaptive multimedia presentation Natural language adaptation Inserting/ removing fragments of Adaptive Hypermedia techniques Adaptive presentation Adaptive text presentation Adaptation of modality Canned text adaptation Altering fragments Stretchtext Adaptive hypermedia technologies Direct guidance Sorting fragments Adaptive link sorting Hiding Dimming fragments Adaptive navigation support Adaptive link hiding Disabling Adaptive link annotation Removal Adaptive link generation Map adaptation 29

30 Adaptive Stretchtext (PUSH)

31 Adaptive annotation in InterBook State of concepts (unknown, known,..., learned) 2. State of current section (ready, not ready, nothing new) 3. States of sections behind the links (as above + visited)

32 QuizGuide: Dual Adaptive Annotations

33 User Modeling in Classic AH Classic AH use external models (besides user model) o Domain models, adaptation (pedagogical) modes, etc. Users are modeled in relation to these models o User knowledge of loops is high o User is interested in 19th century architecture styles Resources are connected (indexed) with elements of these models (aka knowledge behind pages) o This section presents while loop and increment o This page is for field-independent learners o This church is built in 1876

34 Domain Model Concept 1 Concept 4 Concept N Concept 2 Concept 3 Concept 5

35 Indexing of Nodes External (domain) model Hyperspace Concept 1 Concept 4 Concept n Concept 2 Concept 3 Concept m

36 Indexing of Fragments Concepts Node Concept 1 Concept 2 Concept 4 Concept N Fragment 1 Fragment 2 Concept 3 Concept 5 Fragment K

37 Concept-Level User Model Concept 1 10 Concept Concept 3 Concept 4 3 Concept N 0 2 Concept 5

38 AH: Known effects Adaptive presentation helps users understand content faster and deeper Adaptive navigation support reduces navigation efforts and brings the users to the right place at the right time Altogether AH techniques can significantly improve the effectiveness of hypertext and hypermedia systems

39 Recommender systems 39

40 Recommender Systems Native adaptive information access approach How can we model the user in recommender systems? Which adaptation technologies can be applied?

41 Recommender Systems Started as extension of work on adaptive information filtering What is filtering? Search without explicit query Started as library notification services user provided profiles Later considered user feedback (yes/no or ratings) to automatically improve profile Modern Recommenders can start without user profile, constructing it by observation and user feedback Clicking, rating, bookmarking, downloading, purchasing

42 Amazon.com Recommendations

43

44 Types of Recommender Systems Collaborative Filtering Recommender System o Recommendations are generated based on the user history and the community of likeminded users Content-based Recommender System o Recommendation generated from the content features associated with products and the ratings of the user Case-based Recommender System o Similar to content-based recommendation. Information about items is represented as cases. The system recommends the cases that are most similar to user s preference. Hybrid Recommender System o Combination of two or more recommendation techniques to gain better performance with fewer of the drawbacks of any individual one 44

45 Recommendation Procedure 1. Understand and model users 2. Collect candidate items to recommend 3. Based on the recommendation method, predict target users preferences for each candidate item 4. Sort candidate items according to the prediction probability and recommend them 45

46 What is Collaborative Filtering? } Traced back to the Information Tapestry project at Xerox PARC } users annotate documents that they have read and the system recommends them new documents to read } Expanded to automatic CF in the works of Resnick, Riedl, Maes } the process of filtering items using the opinions of other people like you } Key idea: people who agreed with me in the past, will also agree in the future } Compare to Content-based recommendation: } items with features similar to those I liked before will be also liked by me 46

47 User-based CF The input for the CF prediction algorithms is a matrix of users ratings on items, referred as the ratings matrix. Target User Item 1 Item 2 Item 3 Item 4 Item 5 Average Alice ??? 16/4 User /4 User /4 User /4 User /4 47

48 User-based CF Alice User1 User2 User4 1 0 Item 1 Item 2 Item 3 Item 4 48

49 Long Tale the key to Amazon s success 49

50 User-Based NN Recommendation 1.Select like-minded peer group for a target user 2. Choose candidate items which are not in the list of the target user but in the list of peer group 3.Score the items by producing a weighted score and predict the ratings for the given items 4.Select the best candidate items and recommend them to a target user Redo all the procedures through 1 ~ 4 on a timely basis 50

51 User-based NN: User Similarity Pearson s Correlation Coefficient for User a and User b for all Products P rated by both users: Pearson correlation takes values from +1 (Perfectly positive correlation) to -1 (Perfectly negative correlation) 51 å å å Î Î Î = ) ( 2, ) ( 2, ) (,, ) ( ) ( ) )( ( ), ( P product p b p b P product p a p a P product p b p b a p a r r r r r r r r b a sim Average rating of user b

52 User-based NN: Rating Prediction All users with high enough sim(a,b) form the likeminded neighborhood of the user a: o b neighbors(n) To compute the predictions for the user a to like a candidate product p P pred ( a, p) = r a + å bîneighbors( n) å sim( a, b) ( r bîneighbors( n) b, p sim( a, b) - r b ) The items with highest prediction scores get recommended 52

53 Personal data Personalization requires personal data The more detailed and reach the better Data is collected Data is integrated Data is reason upon and inferred Privacy o Do we appreciate their our data is collected o Do we agree that it is manipulated, reasoned upon and used to draw conclusions about us Transparency o Who can see the data o How much do we (need to) know Consent o Do we need to give our consent? Location and security o Where the data/model can/should be stored and how they should be protected o Organization/Country? Ownership o Who owns the data? The models? o Who decides how they can be used? o Can we require to be forgotten? Is it always technically possible? o Can data be shared across organizations? Misinterpretation and Accountability o What if the data is misinterpreted by a system? Or a data analyst? o What if a model makes a wrong prediction o What if the model output is misinterpreted by a user 53

54 Filter Bubble Stream of Information News item SN post Search result News item SN post Search result Your clickstream Your search history Your model Your social circle News item SN post Search result News item SN post Search result 54

55 Targeted recommendations 55