Unsupervised Learning. CS 3793/5233 Artificial Intelligence Unsupervised Learning 1
|
|
|
- Marjorie Hodges
- 6 years ago
- Views:
Transcription
1 Unsupervised CS 3793/5233 Artificial Intelligence Unsupervised 1
2 EM k-means Procedure Data Random Assignment Assign 1 Assign 2 Soft k-means In clustering, the target feature is not given. Goal: Construct a natural classification that can be used to predict features of the data. The examples are partitioned in into clusters or classes. Each class predicts values of the features for the examples in the class. In hard clustering, each example is placed definitively in a class. In soft clustering, each example has a probability of belonging to each class. The best clustering minimizes an error measure. CS 3793/5233 Artificial Intelligence Unsupervised 2
3 EM Algorithm EM k-means Procedure Data Random Assignment Assign 1 Assign 2 Soft k-means The EM (Expectation Maximization) algorithm is not an algorithm, but is an algorithm design technique. Start with a hypothesis space for classifying the data and a random hypothesis. Repeat until convergence: E Step. Classify the examples using the current hypothesis. M Step. Learn a new hypothesis from the examples using their current classification. This can get stuck in local optima; different initializations can affect the result. CS 3793/5233 Artificial Intelligence Unsupervised 3
4 k-means Algorithm EM k-means Procedure Data Random Assignment Assign 1 Assign 2 Soft k-means The k-means algorithm is used for hard clustering. Inputs: training examples the number of classes/clusters, k Outputs: Each example is assigned to one class. The average/mean example of each class. If example e = (x 1,..., x n ) is assigned to class i with mean u i = (u i1,..., u in ), error is e u i 2 = n j=1 (x j u ij ) 2 CS 3793/5233 Artificial Intelligence Unsupervised 4
5 k-means Procedure EM k-means Procedure Data Random Assignment Assign 1 Assign 2 Soft k-means Procedure K-Means(E, k) Inputs: set of examples and number of classes Randomly assign each example to a class Let E i be the examples in class i Repeat M-Step: for each class i from 1 to k u[i] e E i e/ E i E-Step: for each example e in E put e in class arg min i u[i] e 2 until no changes in any E i return u and E i clusters CS 3793/5233 Artificial Intelligence Unsupervised 5
6 Data EM k-means Procedure Data Random Assignment Assign 1 Assign 2 Soft k-means CS 3793/5233 Artificial Intelligence Unsupervised 6
7 Random Assignment to Classes EM k-means Procedure Data Random Assignment Assign 1 Assign 2 Soft k-means CS 3793/5233 Artificial Intelligence Unsupervised 7
8 Assign to Closest Mean EM k-means Procedure Data Random Assignment Assign 1 Assign 2 Soft k-means CS 3793/5233 Artificial Intelligence Unsupervised 8
9 Assign to Closest Mean Again EM k-means Procedure Data Random Assignment Assign 1 Assign 2 Soft k-means CS 3793/5233 Artificial Intelligence Unsupervised 9
10 of k-means EM k-means Procedure Data Random Assignment Assign 1 Assign 2 Soft k-means An assignment of examples to classes is stable if running both the M step and the E step does not change the assignment. This algorithm will converge to a stable local minimum. It is not guaranteed to converge to a global minimum. It is sensitive to the relative scale of the dimensions. Increasing k can always decrease error until k is the number of different examples. CS 3793/5233 Artificial Intelligence Unsupervised 10
11 Soft k-means EM k-means Procedure Data Random Assignment Assign 1 Assign 2 Soft k-means To illustrate soft clustering, consider a soft k-means algorithm. E-Step: For each example e, calculate probability distribution P(class i e) P(c i e) exp{ u i e 2 } M-Step: For each class i, determine mean probabilistically. e E u i = P(c i e) e e E P(c i e) CS 3793/5233 Artificial Intelligence Unsupervised 11
12 Soft k-means EM k-means Procedure Data Random Assignment Assign 1 Assign 2 Soft k-means e P 0 (C x e) P 1 (C x e) P 2 (C x e) (0.7, 5.1) (1.5, 6.0) (2.1, 4.5) (2.4, 5.5) (3.0, 4.4) (3.5, 5.0) (4.5, 1.5) (5.2, 0.7) (5.3, 1.8) (6.2, 1.7) (6.7, 2.5) (8.5, 9.2) (9.1, 9.7) (9.5, 8.5) CS 3793/5233 Artificial Intelligence Unsupervised 12
13 of Soft EM k-means Procedure Data Random Assignment Assign 1 Assign 2 Soft k-means Soft clustering often uses a parameterized probability model, e.g., means and standard deviations for normal distribution. Initially, assign random probabilities to the examples: prob. of class i given example e. The M-step updates the values of the parameters from the probabilities. The E-step updates the probabilities of the examples from the probability model. Does not guarantee global minimum. CS 3793/5233 Artificial Intelligence Unsupervised 13
14 Why hard? Temporal Differences Q Review Q- Update Robot Q-Learner Problems SARSA SARSA on Cliff Features What should an agent do given: Prior knowledge: possible states of the world possible actions Observations: current state of world immediate reward/punishment Goal: act to maximize accumulated reward We assume there is a sequence of experiences: state, action, reward, state, action, reward,... At any time agent must decide whether to explore to gain more knowledge, or exploit knowledge it has already discovered CS 3793/5233 Artificial Intelligence Unsupervised 14
15 Why is reinforcement learning hard? Why hard? Temporal Differences Q Review Q- Update Robot Q-Learner Problems SARSA SARSA on Cliff Features What actions are responsible for a reward may have occurred a long time before the reward was received. The long-term effect of an action depends on what the agent will do in the future. The explore-exploit dilemma: at each time should the agent be greedy or inquisitive? The ǫ-greedy strategy is to select what looks like the best action 1 ǫ of the time, and to select a random action ǫ of the time. CS 3793/5233 Artificial Intelligence Unsupervised 15
16 Temporal Differences Why hard? Temporal Differences Q Review Q- Update Robot Q-Learner Problems SARSA SARSA on Cliff Features Suppose we have a sequence of values v 1, v 2, v 3,... Estimating the average with the first k values: A k = v v k k Separating out v k : A k = (v v k 1 )/k + v k /k Let α = 1/k, then A k = (1 α)a k 1 +αv k = A k 1 +α(v k A k 1 ) The TD update is: A A + α(v A) CS 3793/5233 Artificial Intelligence Unsupervised 16
17 Why hard? Temporal Differences Q Review Q- Update Robot Q-Learner Problems SARSA SARSA on Cliff Features Suppose a robot in this environment. One terminal square has +1 reward (recharge station). One terminal square has 1 reward (falling down stairs). An action to stay put always succeeds. An action to move to a neighbor square, succeeds with probability 0.8, stays in the same square with prob. 0.1, goes to another neighbor with prob. 0.1 Should the robot try moving left or right? +1 1 CS 3793/5233 Artificial Intelligence Unsupervised 17
18 Review of Q Values Why hard? Temporal Differences Q Review Q- Update Robot Q-Learner Problems SARSA SARSA on Cliff Features A policy is a function from states to actions. For reward sequence r 1, r 2,..., discounted reward is: V = i=1 γi 1 r i (discount = γ) V (s) is expected value of state s. Q(s, a) is value of action a from s. For optimal policy: V (s) = max a Q(s, a) (value of best action) Q(s, a)= s P(s s, a)(r(s, a, s )+γv (s )) = s P(s s, a)(r(s, a, s )+γ max a Q(s, a )) Learn optimal policy by learning Q values. Use each experience s, a, r, s to update Q[s, a]. CS 3793/5233 Artificial Intelligence Unsupervised 18
19 Q- Why hard? Temporal Differences Q Review Q- Update Robot Q-Learner Problems SARSA SARSA on Cliff Features Procedure Q-learning(S, A, γ, α, ǫ) Inputs: states, actions discount, step size, exploration factor Initialize Q[S, A] to zeros Repeat for multiple episodes: s initial state Repeat until end of episode: Select action a using ǫ-greedy strategy Do action a. Observe reward r and state s Q[s, a] Q[s, a] + α (r + γ max a Q[s, a ] Q[s, a]) s s return Q CS 3793/5233 Artificial Intelligence Unsupervised 19
20 Q- Update Why hard? Temporal Differences Q Review Q- Update Robot Q-Learner Problems SARSA SARSA on Cliff Features Unpacking the Q- update: Q[s, a] Q[s, a] + α (r + γ max a Q[s, a ] Q[s, a]) Q[s, a]: Value of doing action a from state s. r, s : reward received and next state α, γ: learning rate and discount factor max a Q[s, a ]: With current Q values, the value of the optimal action from state s. r + γ max a Q[s, a ]: With current Q values, the discounted reward to be averaged in. CS 3793/5233 Artificial Intelligence Unsupervised 20
21 Robot Q-Learner, γ =0.9, α=ǫ=0.1 Why hard? Temporal Differences Q Review Q- Update Robot Q-Learner Problems SARSA SARSA on Cliff Features CS 3793/5233 Artificial Intelligence Unsupervised 21
22 Problems with Q- Why hard? Temporal Differences Q Review Q- Update Robot Q-Learner Problems SARSA SARSA on Cliff Features Q-learning does off-policy learning: learns value of optimal policy, but does not follow it. This is bad if exploration is dangerous. Below, Q-learning walks off the cliff too much. On-policy learning learns the value of the policy being followed. SARSA uses the experience s, a, r, s, a to update Q[s, a]. CS 3793/5233 Artificial Intelligence Unsupervised 22
23 SARSA Procedure Why hard? Temporal Differences Q Review Q- Update Robot Q-Learner Problems SARSA SARSA on Cliff Features Procedure SARSA(S, A, γ, α, ǫ) Initialize Q[S, A] to zeros Repeat for multiple episodes: s initial state Select action a using ǫ-greedy strategy Repeat until end of episode: Do action a. Observe reward r and state s Select action a for s using ǫ-greedy Q[s, a] Q[s, a] + α (r + γq[s, a ] Q[s, a]) s s and a a return Q CS 3793/5233 Artificial Intelligence Unsupervised 23
24 SARSA on the Cliff Why hard? Temporal Differences Q Review Q- Update Robot Q-Learner Problems SARSA SARSA on Cliff Features Reward per Episode Q learning Sarsa Number of Episodes CS 3793/5233 Artificial Intelligence Unsupervised 24
25 with Features Why hard? Temporal Differences Q Review Q- Update Robot Q-Learner Problems SARSA SARSA on Cliff Features Often, we want to reason in terms of features. Want to take of advantage of similarities between states. Each assignment to the features is a state. Idea: Express Q as a function of the features. Features encode state and action. (s, a) = (x 1, x 2, x 3,...) Q(s, a) = w 0 + w 1 x 1 + w 2 x 2 + w 3 x δ = r + γq(s, a ) Q(s, a) w i w i + αδx i CS 3793/5233 Artificial Intelligence Unsupervised 25
26 CPs Unobserved Variables Network Structure Algorithm I Algorithm II Original 1000 s Learn Probabilities EM for Hidden Learn Structure Learn each probability separately, if you: know the structure of the network observe all the variables have many examples have no missing data Structure Data Probababilities CS 3793/5233 Artificial Intelligence Unsupervised 26
27 Conditional Probabilities CPs Unobserved Variables Network Structure Algorithm I Algorithm II Original 1000 s Learn Probabilities EM for Hidden Learn Structure Use counts for each conditional probability. For example: P(E = t A = t B = f) = count(e = t A = t B = f) + c 1 count(a = t B = f) + c c 1 and c is prior (expert) knowledge (c 1 c). When there are few examples or many parents to a node, there might be little data for probability estimates: Use supervised learning or noisy ORs/ANDs. CS 3793/5233 Artificial Intelligence Unsupervised 27
28 Unobserved Variables CPs Unobserved Variables Network Structure Algorithm I Algorithm II Original 1000 s Learn Probabilities EM for Hidden Learn Structure What if we had no observations of E? Use EM algorithm with probabilistic inference. Randomly assign values to probability tables that include E. Repeat: E-step: Calculate P(E e) for each example e. M-step: Update probabability tables using counts of P(E e). CS 3793/5233 Artificial Intelligence Unsupervised 28
29 Network Structures CPs Unobserved Variables Network Structure Algorithm I Algorithm II Original 1000 s Learn Probabilities EM for Hidden Learn Structure P(M D) = P(D M) P(M) P(D) log P(M D) log P(D M) + log P(M) M is a Bayesian network and D is the data. Assume all variables are observed. A bigger network can have higher P(M D). P(M) can help control the size. (e.g., using the description length). You can search over network structure looking for the most likely model. CS 3793/5233 Artificial Intelligence Unsupervised 29
30 Algorithm I CPs Unobserved Variables Network Structure Algorithm I Algorithm II Original 1000 s Learn Probabilities EM for Hidden Learn Structure Search over total orderings of variables. For each total ordering X 1,..., X n use supervised learning to learn P(X i X 1... X i 1 ). Return the network model found with minimum: log P(D M) log P(M) log P(D M) can be obtained by calculation. Can approximate log P(M) m log(d + 1), where m = # of parameters in M and d = # of examples. CS 3793/5233 Artificial Intelligence Unsupervised 30
31 Algorithm II CPs Unobserved Variables Network Structure Algorithm I Algorithm II Original 1000 s Learn Probabilities EM for Hidden Learn Structure Learn a tree-structured Bayesian network. Compute correlations between all pairs of variables. Do maximum spanning tree maximizing absolute values of correlations. Pick a variable to be root of the tree, then fill in probabilities using data. CS 3793/5233 Artificial Intelligence Unsupervised 31
32 : Original Bayesian Network CPs Unobserved Variables Network Structure Algorithm I Algorithm II Original 1000 s Learn Probabilities EM for Hidden Learn Structure P(A) = 0.8 B A P(B A) t 0.5 f 0.2 A D C A P(C A) t 0.2 f 0.5 B C P(D B,C) t t 0.5 t f 0.8 f t 0.2 f f 0.4 CS 3793/5233 Artificial Intelligence Unsupervised 32
33 : 1000 s CPs Unobserved Variables Network Structure Algorithm I Algorithm II Original 1000 s Learn Probabilities EM for Hidden Learn Structure A B C D Count T T T T 39 T T T F 37 T T F T 300 T T F F 69 T F T T 15 T F T F 67 T F F T 117 T F F F 189 A B C D Count F T T T 6 F T T F 5 F T F T 20 F T F F 10 F F T T 15 F F T F 46 F F F T 27 F F F F 38 CS 3793/5233 Artificial Intelligence Unsupervised 33
34 : Learn Probabilities CPs Unobserved Variables Network Structure Algorithm I Algorithm II Original 1000 s Learn Probabilities EM for Hidden Learn Structure A t f P(A) = 0.83 B P(B A) A D C A P(C A) t 0.19 f 0.43 B C P(D B,C) t t 0.52 t f 0.80 f t 0.21 f f 0.39 CS 3793/5233 Artificial Intelligence Unsupervised 34
35 : EM for Hidden Variable CPs Unobserved Variables Network Structure Algorithm I Algorithm II Original 1000 s Learn Probabilities EM for Hidden Learn Structure P(H) =0.25 B H P(B H) t 0.83 f 0.37 H D C H P(C H) t 0.02 f 0.30 B C P(D B,C) t t 0.5 t f 0.8 f t 0.2 f f 0.4 CS 3793/5233 Artificial Intelligence Unsupervised 35
36 : Learn Structure A P(A) =0.83 CPs Unobserved Variables Network Structure Algorithm I Algorithm II Original 1000 s Learn Probabilities EM for Hidden Learn Structure Correlation Table B C D A B C.23 B D A P(B A) t 0.53 f 0.25 B P(D B) t 0.75 f 0.34 C D P(C D) t 0.14 f 0.34 CS 3793/5233 Artificial Intelligence Unsupervised 36
Algorithms for Solving RL: Temporal Difference Learning (TD) Reinforcement Learning Lecture 10
Algorithms for Solving RL: Temporal Difference Learning (TD) 1 Reinforcement Learning Lecture 10 Gillian Hayes 8th February 2007 Incremental Monte Carlo Algorithm TD Prediction TD vs MC vs DP TD for control:
Markov Decision Processes and Reinforcement Learning
Lecture 14 and Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Slides by Stuart Russell and Peter Norvig Course Overview Introduction Artificial Intelligence
Using Machine Learning to Optimize Storage Systems
Using Machine Learning to Optimize Storage Systems Dr. Kiran Gunnam 1 Outline 1. Overview 2. Building Flash Models using Logistic Regression. 3. Storage Object classification 4. Storage Allocation recommendation
10-701/15-781, Fall 2006, Final
-7/-78, Fall 6, Final Dec, :pm-8:pm There are 9 questions in this exam ( pages including this cover sheet). If you need more room to work out your answer to a question, use the back of the page and clearly
Clustering & Dimensionality Reduction. 273A Intro Machine Learning
Clustering & Dimensionality Reduction 273A Intro Machine Learning What is Unsupervised Learning? In supervised learning we were given attributes & targets (e.g. class labels). In unsupervised learning
COMS 4771 Clustering. Nakul Verma
COMS 4771 Clustering Nakul Verma Supervised Learning Data: Supervised learning Assumption: there is a (relatively simple) function such that for most i Learning task: given n examples from the data, find
Unsupervised Learning: Clustering
Unsupervised Learning: Clustering Vibhav Gogate The University of Texas at Dallas Slides adapted from Carlos Guestrin, Dan Klein & Luke Zettlemoyer Machine Learning Supervised Learning Unsupervised Learning
Introduction to Reinforcement Learning. J. Zico Kolter Carnegie Mellon University
Introduction to Reinforcement Learning J. Zico Kolter Carnegie Mellon University 1 Agent interaction with environment Agent State s Reward r Action a Environment 2 Of course, an oversimplification 3 Review:
The Expectation Maximization (EM) Algorithm
The Expectation Maximization (EM) Algorithm continued! 600.465 - Intro to NLP - J. Eisner 1 General Idea Start by devising a noisy channel Any model that predicts the corpus observations via some hidden
CS Introduction to Data Mining Instructor: Abdullah Mueen
CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen LECTURE 8: ADVANCED CLUSTERING (FUZZY AND CO -CLUSTERING) Review: Basic Cluster Analysis Methods (Chap. 10) Cluster Analysis: Basic Concepts
Machine Learning. Unsupervised Learning. Manfred Huber
Machine Learning Unsupervised Learning Manfred Huber 2015 1 Unsupervised Learning In supervised learning the training data provides desired target output for learning In unsupervised learning the training
Clustering web search results
Clustering K-means Machine Learning CSE546 Emily Fox University of Washington November 4, 2013 1 Clustering images Set of Images [Goldberger et al.] 2 1 Clustering web search results 3 Some Data 4 2 K-means
Probabilistic Graphical Models Part III: Example Applications
Probabilistic Graphical Models Part III: Example Applications Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2014 CS 551, Fall 2014 c 2014, Selim
A Comparative study of Clustering Algorithms using MapReduce in Hadoop
A Comparative study of Clustering Algorithms using MapReduce in Hadoop Dweepna Garg 1, Khushboo Trivedi 2, B.B.Panchal 3 1 Department of Computer Science and Engineering, Parul Institute of Engineering
Q-learning with linear function approximation
Q-learning with linear function approximation Francisco S. Melo and M. Isabel Ribeiro Institute for Systems and Robotics [fmelo,mir]@isr.ist.utl.pt Conference on Learning Theory, COLT 2007 June 14th, 2007
Adaptive Building of Decision Trees by Reinforcement Learning
Proceedings of the 7th WSEAS International Conference on Applied Informatics and Communications, Athens, Greece, August 24-26, 2007 34 Adaptive Building of Decision Trees by Reinforcement Learning MIRCEA
K-means and Hierarchical Clustering
K-means and Hierarchical Clustering Xiaohui Xie University of California, Irvine K-means and Hierarchical Clustering p.1/18 Clustering Given n data points X = {x 1, x 2,, x n }. Clustering is the partitioning
Reinforcement Learning: A brief introduction. Mihaela van der Schaar
Reinforcement Learning: A brief introduction Mihaela van der Schaar Outline Optimal Decisions & Optimal Forecasts Markov Decision Processes (MDPs) States, actions, rewards and value functions Dynamic Programming
A Brief Introduction to Reinforcement Learning
A Brief Introduction to Reinforcement Learning Minlie Huang ( ) Dept. of Computer Science, Tsinghua University aihuang@tsinghua.edu.cn 1 http://coai.cs.tsinghua.edu.cn/hml Reinforcement Learning Agent
Unsupervised Learning
Networks for Pattern Recognition, 2014 Networks for Single Linkage K-Means Soft DBSCAN PCA Networks for Kohonen Maps Linear Vector Quantization Networks for Problems/Approaches in Machine Learning Supervised
CS 1675 Introduction to Machine Learning Lecture 18. Clustering. Clustering. Groups together similar instances in the data sample
CS 1675 Introduction to Machine Learning Lecture 18 Clustering Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square Clustering Groups together similar instances in the data sample Basic clustering problem:
Machine Learning. B. Unsupervised Learning B.1 Cluster Analysis. Lars Schmidt-Thieme
Machine Learning B. Unsupervised Learning B.1 Cluster Analysis Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim, Germany
Artificial Intelligence. Programming Styles
Artificial Intelligence Intro to Machine Learning Programming Styles Standard CS: Explicitly program computer to do something Early AI: Derive a problem description (state) and use general algorithms to
ADAPTIVE TILE CODING METHODS FOR THE GENERALIZATION OF VALUE FUNCTIONS IN THE RL STATE SPACE A THESIS SUBMITTED TO THE FACULTY OF THE GRADUATE SCHOOL
ADAPTIVE TILE CODING METHODS FOR THE GENERALIZATION OF VALUE FUNCTIONS IN THE RL STATE SPACE A THESIS SUBMITTED TO THE FACULTY OF THE GRADUATE SCHOOL OF THE UNIVERSITY OF MINNESOTA BY BHARAT SIGINAM IN
Simple Model Selection Cross Validation Regularization Neural Networks
Neural Nets: Many possible refs e.g., Mitchell Chapter 4 Simple Model Selection Cross Validation Regularization Neural Networks Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University February
ECE 5424: Introduction to Machine Learning
ECE 5424: Introduction to Machine Learning Topics: Unsupervised Learning: Kmeans, GMM, EM Readings: Barber 20.1-20.3 Stefan Lee Virginia Tech Tasks Supervised Learning x Classification y Discrete x Regression
Review: Final Exam CPSC Artificial Intelligence Michael M. Richter
Review: Final Exam Model for a Learning Step Learner initially Environm ent Teacher Compare s pe c ia l Information Control Correct Learning criteria Feedback changed Learner after Learning Learning by
Grundlagen der Künstlichen Intelligenz
Grundlagen der Künstlichen Intelligenz Unsupervised learning Daniel Hennes 29.01.2018 (WS 2017/18) University Stuttgart - IPVS - Machine Learning & Robotics 1 Today Supervised learning Regression (linear
7. Boosting and Bagging Bagging
Group Prof. Daniel Cremers 7. Boosting and Bagging Bagging Bagging So far: Boosting as an ensemble learning method, i.e.: a combination of (weak) learners A different way to combine classifiers is known
Clustering: Classic Methods and Modern Views
Clustering: Classic Methods and Modern Views Marina Meilă University of Washington mmp@stat.washington.edu June 22, 2015 Lorentz Center Workshop on Clusters, Games and Axioms Outline Paradigms for clustering
Unsupervised Learning
Unsupervised Learning Learning without Class Labels (or correct outputs) Density Estimation Learn P(X) given training data for X Clustering Partition data into clusters Dimensionality Reduction Discover
Unsupervised Learning. Clustering and the EM Algorithm. Unsupervised Learning is Model Learning
Unsupervised Learning Clustering and the EM Algorithm Susanna Ricco Supervised Learning Given data in the form < x, y >, y is the target to learn. Good news: Easy to tell if our algorithm is giving the
COMP 551 Applied Machine Learning Lecture 13: Unsupervised learning
COMP 551 Applied Machine Learning Lecture 13: Unsupervised learning Associate Instructor: Herke van Hoof (herke.vanhoof@mail.mcgill.ca) Slides mostly by: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551
arxiv: v1 [cs.ne] 22 Mar 2016
![arxiv: v1 [cs.ne] 22 Mar 2016](/thumbs/91/106539758.jpg "arxiv: v1 [cs.ne] 22 Mar 2016")
Adaptive Parameter Selection in Evolutionary Algorithms by Reinforcement Learning with Dynamic Discretization of Parameter Range arxiv:1603.06788v1 [cs.ne] 22 Mar 2016 ABSTRACT Arkady Rost ITMO University
10601 Machine Learning. Model and feature selection
10601 Machine Learning Model and feature selection Model selection issues We have seen some of this before Selecting features (or basis functions) Logistic regression SVMs Selecting parameter value Prior
CSC 411: Lecture 12: Clustering
CSC 411: Lecture 12: Clustering Raquel Urtasun & Rich Zemel University of Toronto Oct 22, 2015 Urtasun & Zemel (UofT) CSC 411: 12-Clustering Oct 22, 2015 1 / 18 Today Unsupervised learning Clustering -means
PRACTICE FINAL EXAM: SPRING 2012 CS 6375 INSTRUCTOR: VIBHAV GOGATE
PRACTICE FINAL EXAM: SPRING 0 CS 675 INSTRUCTOR: VIBHAV GOGATE May 4, 0 The exam is closed book. You are allowed four pages of double sided cheat sheets. Answer the questions in the spaces provided on
Unsupervised Learning and Data Mining
Unsupervised Learning and Data Mining Unsupervised Learning and Data Mining Clustering Supervised Learning ó Decision trees ó Artificial neural nets ó K-nearest neighbor ó Support vectors ó Linear regression
When Network Embedding meets Reinforcement Learning?
When Network Embedding meets Reinforcement Learning? ---Learning Combinatorial Optimization Problems over Graphs Changjun Fan 1 1. An Introduction to (Deep) Reinforcement Learning 2. How to combine NE
Case-Based Reasoning. CS 188: Artificial Intelligence Fall Nearest-Neighbor Classification. Parametric / Non-parametric.
CS 188: Artificial Intelligence Fall 2008 Lecture 25: Kernels and Clustering 12/2/2008 Dan Klein UC Berkeley Case-Based Reasoning Similarity for classification Case-based reasoning Predict an instance
CS 188: Artificial Intelligence Fall 2008
CS 188: Artificial Intelligence Fall 2008 Lecture 25: Kernels and Clustering 12/2/2008 Dan Klein UC Berkeley 1 1 Case-Based Reasoning Similarity for classification Case-based reasoning Predict an instance
Markov Decision Processes (MDPs) (cont.)
Markov Decision Processes (MDPs) (cont.) Machine Learning 070/578 Carlos Guestrin Carnegie Mellon University November 29 th, 2007 Markov Decision Process (MDP) Representation State space: Joint state x
Machine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves
![Machine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves](/thumbs/93/111796408.jpg "Machine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves")
Machine Learning A 708.064 11W 1sst KU Exercises Problems marked with * are optional. 1 Conditional Independence I [2 P] a) [1 P] Give an example for a probability distribution P (A, B, C) that disproves
CPSC 340: Machine Learning and Data Mining. Principal Component Analysis Fall 2016
CPSC 340: Machine Learning and Data Mining Principal Component Analysis Fall 2016 A2/Midterm: Admin Grades/solutions will be posted after class. Assignment 4: Posted, due November 14. Extra office hours:
CLUSTERING. JELENA JOVANOVIĆ Web:
CLUSTERING JELENA JOVANOVIĆ Email: jeljov@gmail.com Web: http://jelenajovanovic.net OUTLINE What is clustering? Application domains K-Means clustering Understanding it through an example The K-Means algorithm
Marco Wiering Intelligent Systems Group Utrecht University
Reinforcement Learning for Robot Control Marco Wiering Intelligent Systems Group Utrecht University marco@cs.uu.nl 22-11-2004 Introduction Robots move in the physical environment to perform tasks The environment
Machine Learning. Semi-Supervised Learning. Manfred Huber
Machine Learning Semi-Supervised Learning Manfred Huber 2015 1 Semi-Supervised Learning Semi-supervised learning refers to learning from data where part contains desired output information and the other
k-means demo Administrative Machine learning: Unsupervised learning" Assignment 5 out
Machine learning: Unsupervised learning" David Kauchak cs Spring 0 adapted from: http://www.stanford.edu/class/cs76/handouts/lecture7-clustering.ppt http://www.youtube.com/watch?v=or_-y-eilqo Administrative
Machine Learning. B. Unsupervised Learning B.1 Cluster Analysis. Lars Schmidt-Thieme, Nicolas Schilling
Machine Learning B. Unsupervised Learning B.1 Cluster Analysis Lars Schmidt-Thieme, Nicolas Schilling Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim,
Generalized Inverse Reinforcement Learning
Generalized Inverse Reinforcement Learning James MacGlashan Cogitai, Inc. james@cogitai.com Michael L. Littman mlittman@cs.brown.edu Nakul Gopalan ngopalan@cs.brown.edu Amy Greenwald amy@cs.brown.edu Abstract
1) Give decision trees to represent the following Boolean functions:
1) Give decision trees to represent the following Boolean functions: 1) A B 2) A [B C] 3) A XOR B 4) [A B] [C Dl Answer: 1) A B 2) A [B C] 1 3) A XOR B = (A B) ( A B) 4) [A B] [C D] 2 2) Consider the following
Announcements. CS 188: Artificial Intelligence Spring Classification: Feature Vectors. Classification: Weights. Learning: Binary Perceptron
CS 188: Artificial Intelligence Spring 2010 Lecture 24: Perceptrons and More! 4/20/2010 Announcements W7 due Thursday [that s your last written for the semester!] Project 5 out Thursday Contest running
ALTERNATIVE METHODS FOR CLUSTERING
ALTERNATIVE METHODS FOR CLUSTERING K-Means Algorithm Termination conditions Several possibilities, e.g., A fixed number of iterations Objects partition unchanged Centroid positions don t change Convergence
Latent Variable Models and Expectation Maximization
Latent Variable Models and Expectation Maximization Oliver Schulte - CMPT 726 Bishop PRML Ch. 9 2 4 6 8 1 12 14 16 18 2 4 6 8 1 12 14 16 18 5 1 15 2 25 5 1 15 2 25 2 4 6 8 1 12 14 2 4 6 8 1 12 14 5 1 15
Learning. Learning agents Inductive learning. Neural Networks. Different Learning Scenarios Evaluation
Learning Learning agents Inductive learning Different Learning Scenarios Evaluation Slides based on Slides by Russell/Norvig, Ronald Williams, and Torsten Reil Material from Russell & Norvig, chapters
Supervised vs unsupervised clustering
Classification Supervised vs unsupervised clustering Cluster analysis: Classes are not known a- priori. Classification: Classes are defined a-priori Sometimes called supervised clustering Extract useful
Fall 09, Homework 5
5-38 Fall 09, Homework 5 Due: Wednesday, November 8th, beginning of the class You can work in a group of up to two people. This group does not need to be the same group as for the other homeworks. You
Introduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Clustering and EM Barnabás Póczos & Aarti Singh Contents Clustering K-means Mixture of Gaussians Expectation Maximization Variational Methods 2 Clustering 3 K-
Midterm Examination CS 540-2: Introduction to Artificial Intelligence
Midterm Examination CS 54-2: Introduction to Artificial Intelligence March 9, 217 LAST NAME: FIRST NAME: Problem Score Max Score 1 15 2 17 3 12 4 6 5 12 6 14 7 15 8 9 Total 1 1 of 1 Question 1. [15] State
Introduction to Artificial Intelligence
Introduction to Artificial Intelligence COMP307 Machine Learning 2: 3-K Techniques Yi Mei yi.mei@ecs.vuw.ac.nz 1 Outline K-Nearest Neighbour method Classification (Supervised learning) Basic NN (1-NN)
Final Exam. Introduction to Artificial Intelligence. CS 188 Spring 2010 INSTRUCTIONS. You have 3 hours.
CS 188 Spring 2010 Introduction to Artificial Intelligence Final Exam INSTRUCTIONS You have 3 hours. The exam is closed book, closed notes except a two-page crib sheet. Please use non-programmable calculators
Clustering with Reinforcement Learning
Clustering with Reinforcement Learning Wesam Barbakh and Colin Fyfe, The University of Paisley, Scotland. email:wesam.barbakh,colin.fyfe@paisley.ac.uk Abstract We show how a previously derived method of
Function Approximation. Pieter Abbeel UC Berkeley EECS
Function Approximation Pieter Abbeel UC Berkeley EECS Outline Value iteration with function approximation Linear programming with function approximation Value Iteration Algorithm: Start with for all s.
Inference and Representation
Inference and Representation Rachel Hodos New York University Lecture 5, October 6, 2015 Rachel Hodos Lecture 5: Inference and Representation Today: Learning with hidden variables Outline: Unsupervised
CS839: Probabilistic Graphical Models. Lecture 10: Learning with Partially Observed Data. Theo Rekatsinas
CS839: Probabilistic Graphical Models Lecture 10: Learning with Partially Observed Data Theo Rekatsinas 1 Partially Observed GMs Speech recognition 2 Partially Observed GMs Evolution 3 Partially Observed
Based on Raymond J. Mooney s slides
Instance Based Learning Based on Raymond J. Mooney s slides University of Texas at Austin 1 Example 2 Instance-Based Learning Unlike other learning algorithms, does not involve construction of an explicit
Introduction to Graphical Models
Robert Collins CSE586 Introduction to Graphical Models Readings in Prince textbook: Chapters 10 and 11 but mainly only on directed graphs at this time Credits: Several slides are from: Review: Probability
CP365 Artificial Intelligence
CP365 Artificial Intelligence Example Problem Problem: Does a given image contain cats? Input vector: RGB/BW pixels of the image. Output: Yes or No. Example Problem Problem: What category is a news story?
Gene Clustering & Classification
BINF, Introduction to Computational Biology Gene Clustering & Classification Young-Rae Cho Associate Professor Department of Computer Science Baylor University Overview Introduction to Gene Clustering
Network Traffic Measurements and Analysis
DEIB - Politecnico di Milano Fall, 2017 Introduction Often, we have only a set of features x = x 1, x 2,, x n, but no associated response y. Therefore we are not interested in prediction nor classification,
CSE 40171: Artificial Intelligence. Learning from Data: Unsupervised Learning
CSE 40171: Artificial Intelligence Learning from Data: Unsupervised Learning 32 Homework #6 has been released. It is due at 11:59PM on 11/7. 33 CSE Seminar: 11/1 Amy Reibman Purdue University 3:30pm DBART
Contents Machine Learning concepts 4 Learning Algorithm 4 Predictive Model (Model) 4 Model, Classification 4 Model, Regression 4 Representation
Contents Machine Learning concepts 4 Learning Algorithm 4 Predictive Model (Model) 4 Model, Classification 4 Model, Regression 4 Representation Learning 4 Supervised Learning 4 Unsupervised Learning 4
An Introduction to Machine Learning
TRIPODS Summer Boorcamp: Topology and Machine Learning August 6, 2018 General Set-up Introduction Set-up and Goal Suppose we have X 1,X 2,...,X n data samples. Can we predict properites about any given
CS242: Probabilistic Graphical Models Lecture 3: Factor Graphs & Variable Elimination
CS242: Probabilistic Graphical Models Lecture 3: Factor Graphs & Variable Elimination Instructor: Erik Sudderth Brown University Computer Science September 11, 2014 Some figures and materials courtesy
Hierarchical Assignment of Behaviours by Self-Organizing
Hierarchical Assignment of Behaviours by Self-Organizing W. Moerman 1 B. Bakker 2 M. Wiering 3 1 M.Sc. Cognitive Artificial Intelligence Utrecht University 2 Intelligent Autonomous Systems Group University
Reddit Recommendation System Daniel Poon, Yu Wu, David (Qifan) Zhang CS229, Stanford University December 11 th, 2011
Reddit Recommendation System Daniel Poon, Yu Wu, David (Qifan) Zhang CS229, Stanford University December 11 th, 2011 1. Introduction Reddit is one of the most popular online social news websites with millions
CSE 573: Artificial Intelligence Autumn 2010
CSE 573: Artificial Intelligence Autumn 2010 Lecture 16: Machine Learning Topics 12/7/2010 Luke Zettlemoyer Most slides over the course adapted from Dan Klein. 1 Announcements Syllabus revised Machine
Performance Comparison of Sarsa(λ) and Watkin s Q(λ) Algorithms
Performance Comparison of Sarsa(λ) and Watkin s Q(λ) Algorithms Karan M. Gupta Department of Computer Science Texas Tech University Lubbock, TX 7949-314 gupta@cs.ttu.edu Abstract This paper presents a
Uninformed Search Methods. Informed Search Methods. Midterm Exam 3/13/18. Thursday, March 15, 7:30 9:30 p.m. room 125 Ag Hall
Midterm Exam Thursday, March 15, 7:30 9:30 p.m. room 125 Ag Hall Covers topics through Decision Trees and Random Forests (does not include constraint satisfaction) Closed book 8.5 x 11 sheet with notes
Lecture 13: Learning from Demonstration
CS 294-5 Algorithmic Human-Robot Interaction Fall 206 Lecture 3: Learning from Demonstration Scribes: Samee Ibraheem and Malayandi Palaniappan - Adapted from Notes by Avi Singh and Sammy Staszak 3. Introduction
Cluster Evaluation and Expectation Maximization! adapted from: Doug Downey and Bryan Pardo, Northwestern University
Cluster Evaluation and Expectation Maximization! adapted from: Doug Downey and Bryan Pardo, Northwestern University Kinds of Clustering Sequential Fast Cost Optimization Fixed number of clusters Hierarchical
http://www.xkcd.com/233/ Text Clustering David Kauchak cs160 Fall 2009 adapted from: http://www.stanford.edu/class/cs276/handouts/lecture17-clustering.ppt Administrative 2 nd status reports Paper review
WestminsterResearch
WestminsterResearch http://www.westminster.ac.uk/research/westminsterresearch Reinforcement learning in continuous state- and action-space Barry D. Nichols Faculty of Science and Technology This is an
Midterm Examination CS540-2: Introduction to Artificial Intelligence
Midterm Examination CS540-2: Introduction to Artificial Intelligence March 15, 2018 LAST NAME: FIRST NAME: Problem Score Max Score 1 12 2 13 3 9 4 11 5 8 6 13 7 9 8 16 9 9 Total 100 Question 1. [12] Search
10701 Machine Learning. Clustering
171 Machine Learning Clustering What is Clustering? Organizing data into clusters such that there is high intra-cluster similarity low inter-cluster similarity Informally, finding natural groupings among
Semi-supervised Learning
Semi-supervised Learning Piyush Rai CS5350/6350: Machine Learning November 8, 2011 Semi-supervised Learning Supervised Learning models require labeled data Learning a reliable model usually requires plenty
INF4820 Algorithms for AI and NLP. Evaluating Classifiers Clustering
INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Murhaf Fares & Stephan Oepen Language Technology Group (LTG) September 27, 2017 Today 2 Recap Evaluation of classifiers Unsupervised
University of Ghana Department of Computer Engineering School of Engineering Sciences College of Basic and Applied Sciences
University of Ghana Department of Computer Engineering School of Engineering Sciences College of Basic and Applied Sciences CPEN 405: Artificial Intelligence Lab 7 November 15, 2017 Unsupervised Learning
Machine Learning and Data Mining. Clustering (1): Basics. Kalev Kask
Machine Learning and Data Mining Clustering (1): Basics Kalev Kask Unsupervised learning Supervised learning Predict target value ( y ) given features ( x ) Unsupervised learning Understand patterns of
Apprenticeship Learning for Reinforcement Learning. with application to RC helicopter flight Ritwik Anand, Nick Haliday, Audrey Huang
Apprenticeship Learning for Reinforcement Learning with application to RC helicopter flight Ritwik Anand, Nick Haliday, Audrey Huang Table of Contents Introduction Theory Autonomous helicopter control
Topics in AI (CPSC 532L): Multimodal Learning with Vision, Language and Sound. Lecture 12: Deep Reinforcement Learning
Topics in AI (CPSC 532L): Multimodal Learning with Vision, Language and Sound Lecture 12: Deep Reinforcement Learning Types of Learning Supervised training Learning from the teacher Training data includes
Clustering and The Expectation-Maximization Algorithm
Clustering and The Expectation-Maximization Algorithm Unsupervised Learning Marek Petrik 3/7 Some of the figures in this presentation are taken from An Introduction to Statistical Learning, with applications
K Nearest Neighbor Wrap Up K- Means Clustering. Slides adapted from Prof. Carpuat
K Nearest Neighbor Wrap Up K- Means Clustering Slides adapted from Prof. Carpuat K Nearest Neighbor classification Classification is based on Test instance with Training Data K: number of neighbors that
The k-means Algorithm and Genetic Algorithm
The k-means Algorithm and Genetic Algorithm k-means algorithm Genetic algorithm Rough set approach Fuzzy set approaches Chapter 8 2 The K-Means Algorithm The K-Means algorithm is a simple yet effective
International Journal of Scientific Research & Engineering Trends Volume 4, Issue 6, Nov-Dec-2018, ISSN (Online): X
Analysis about Classification Techniques on Categorical Data in Data Mining Assistant Professor P. Meena Department of Computer Science Adhiyaman Arts and Science College for Women Uthangarai, Krishnagiri,
Pattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition
Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant
STAT 598L Probabilistic Graphical Models. Instructor: Sergey Kirshner. Exact Inference
STAT 598L Probabilistic Graphical Models Instructor: Sergey Kirshner Exact Inference What To Do With Bayesian/Markov Network? Compact representation of a complex model, but Goal: efficient extraction of
Day 3 Lecture 1. Unsupervised Learning
Day 3 Lecture 1 Unsupervised Learning Semi-supervised and transfer learning Myth: you can t do deep learning unless you have a million labelled examples for your problem. Reality You can learn useful representations
Planning and Control: Markov Decision Processes
CSE-571 AI-based Mobile Robotics Planning and Control: Markov Decision Processes Planning Static vs. Dynamic Predictable vs. Unpredictable Fully vs. Partially Observable Perfect vs. Noisy Environment What
Pascal De Beck-Courcelle. Master in Applied Science. Electrical and Computer Engineering
Study of Multiple Multiagent Reinforcement Learning Algorithms in Grid Games by Pascal De Beck-Courcelle A thesis submitted to the Faculty of Graduate and Postdoctoral Affairs in partial fulfillment of