CC PROCESAMIENTO MASIVO DE DATOS OTOÑO 2018
|
|
|
- Frederica Sparks
- 5 years ago
- Views:
Transcription
1 CC PROCESAMIENTO MASIVO DE DATOS OTOÑO 2018 Lecture 1: Introduction Aidan Hogan
2 THE VALUE OF DATA
3 Soho, London, 1854
4 Cholera: What we know now
5 Cholera: What we knew in 1854
6 1854: Galen s miasma theory of cholera
7 1854: The hunt for the invisible cholera
8 John Snow:
9 John Snow:
10 The Survey of Soho
11 The Survey of Soho
12 What the data showed
13 What the data showed
14 616 deaths, 8 days later
15 Cholera: What we knew in 1855
16 Cholera boil notice ca. 1866
17 Cholera boil notice ca. 1866
18 Thirty years before discovery of V. cholerae
19 John Snow: Father of Epidemiology
20 Epidemiology s Success Stories
21 Value of data: Not just epidemiology
22 (Paper) Notebooks no longer good enough
23 THE GROWTH OF DATA
24 Big Data English Wikipedia 51 GB of data (2015 dump) (Text; No edit history) (XML, uncompressed) 1 Wiki = 1 Wikipedia
25 Big Data Wikimedia Commons 24 TB of data Wiki (2014 dump)
26 Big Data Sloan Digital Sky Survey 200 GB / day 4 Wiki / day (2013, generated by SDSS)
27 Big Data Twitter 8 TB / day 157 Wiki / day (2013, generated)
28 Big Data Large Hadron Collider 68 TB / day 1,370 Wiki / day (2012, collision data generated)
29 Big Data Facebook 600 TB / day 11,764 Wiki / day (2014, incoming Hive data)
30 Big Data NSA Surveillance 29 PB / day 568,627 Wiki / day (2013, processed)
31 Big Data Google 100 PB / day 2,000,000 Wiki / day (2014, processed)
32 Big Data Internet Traffic 2,417 PB / day 47,000,000 Wiki / day (2014, Cisco estimates)
33 Data: A Modern-day Bottleneck?
34 The V s of Big Data
35 BIG DATA IN ACTION
36 Getting Home (Waze) What s the fastest route to get home right now? Processes journeys as background knowledge Participatory Sensing
37 Predicting Pre-crime (PredPol) What areas of the city are most need of police patrol at 13:55 on Mondays? PredPol system used by Santa Cruz (US) police patrols Predictions based on 8 years of historical crime data
38 Getting Elected President (Narwhal) Who are the undecided voters and how can I convince them to vote for me? User profiles built and integrated from online sources Targeted messages sent to voters based on profile
39 Winning Jeopardy (IBM Watson) Can a computer beat human experts at Jeopardy? Indexed 200 million pages of content An ensemble of 100 processing techniques
40 BIG DATA NEEDS MASSIVE DATA PROCESSING
41 Every Application is Different Data can be (Semi-)Structured data (Relational DBs, JSON, XML, CSV, HTML form data) Unstructured data (text document, comments, tweets) And everything in-between!
42 Every Application is Different Processing can involve: Database Management/Analytics (indexing, querying, joins, aggregation) Natural Language Processing (keyword search, topic extraction, entity recognition, machine translation, sentiment analysis, etc.) Data Mining and Statistics (pattern recognition, classification, event detection, recommendations, etc.) Or something else / A mix
43 So where to start?
44 Scale is a Common Factor I have an algorithm. I have a machine that can process 1,000 input items in an hour. If I buy a machine that is n times as powerful, how many input items can I process in an hour? Depends on what the algorithm is!! Quadratic O(n 2 ) often too much Note: Not the same machine!
45 Scale is a Common Factor One machine that s n times as powerful? n machines that are vs. equally as powerful?
46 Scale is a Common Factor Data-intensive (our focus!) Inexpensive algorithms / Large inputs e.g., Google, Facebook, Twitter Compute-intensive (not our focus!) More expensive algorithms / Smaller inputs e.g., climate simulations, chess games, combinatorials No black and white!
47 MASSIVE DATA PROCESSING NEEDS DISTRIBUTED COMPUTING
48 Distributed Computing Need more than one machine! Google ca. 1998:
49 Distributed Computing Need more than one machine! Google ca. 2014:
50 Data Transport Costs Need to divide tasks over many machines Machines need to communicate but not too much! Data transport costs (simplified): Main Memory Solid-state Disk Hard-disk Network Need to minimise network costs!
51 Data Placement Need to think carefully about where to put what data! I have four machines to run a website. I have 10 million users. Each user has personal profile data, photos, friends and games. How should I split the data up over the machines? Depends on the application! But some general principles and design choices apply.
52 Network/Node Failures Need to think about failures!
53 Network/Node Failures Need to think (even more!) carefully about where to put what data! I have four machines to run a website. I have 10 million users. Each user has personal profile data, photos, friends and games. How should I split the data up over the machines? (Again) Depends on the application! But some general principles and design choices apply.
54 Human Distributed Computation
55 DISTRIBUTED COMPUTING LIMITS & CHALLENGES
56 Distribution Not Always Applicable!
57 Distributed Development Difficult Distributed systems can be complex Multiple machines; need to take care of Data in different locations Logs and messages in different places Different users with different priorities Different network capabilities Need to balance load! Need to handle failures! Tasks may take a long time! Bugs may not become apparent for hours Lots of data = lots of counter-examples
58 Frameworks/Abstractions can Help For Distrib. Processing For Distrib. Storage
59 HOW DOES TWITTER WORK?
60 Based on 2013 slides by Twitter lead architect: Raffi Krikorian Twitter Timelines at Scale
61 Big Data at Twitter 150 million active worldwide users 400 million tweets per day mean: 4,600 tweets/second max: 150,000 tweets/second 300,000 queries/second for user timelines 6,000 queries/second for custom search Which aspect is most important to optimise?
62 Supporting timelines: write mean: 4,000 tweets/second
63 High-fanout
64 Supporting timelines: read 300,000 queries/second
65 Supporting text search Information retrieval Earlybird: Lucene clone Write once Query many
66 Timeline vs. Search 4,600 requests/sec 300,000 requests/sec 4,600 requests/sec 6,000 requests/sec
67 Twitter: Full Architecture
68 PROCESAMIENTO MASIVO DE DATOS ABOUT THE COURSE
69 What the Course Is/Is Not Data-intensive not compute-intensive Distributed tasks not networking Commodity hardware not supercomputers General methods not specific algorithms Practical methods with a little theory
70 What the Course Is Principles of Distributed Computing [1 week] Distributed Processing Frameworks [4 weeks] Information Retrieval [3 weeks] Principles of Distributed Databases [3 weeks] Projects [1 2 weeks]
71 Course Structure ~1.5 hours of lectures per week [Monday] 1.5 hours of labs per week [Wednesday] To be turned in by next Monday evening Mostly Java Auxiliar Session [Friday] Mostly for controls We will announce in the forum if there's a session
72 Course Marking 55% for Weekly Labs (~5% a lab!) 15% for Class Project 30% for 2x Controls Assignments each week Controls Working in groups Only need to pass overall! No final exam! Working in groups!
73 Outcomes!
74 Outcomes!
75 Outcomes!
76 Outcomes!
77 Outcomes!
78 Questions?
27/04/2015 CC PROCESAMIENTO MASIVO DE DATOS OTOÑO Lecture 1: Introduction THE VALUE OF DATA. Aidan Hogan
CC5212-1 PROCESAMIENTO MASIVO DE DATOS OTOÑO 2015 Lecture 1: Introduction Aidan Hogan aidhog@gmail.com THE VALUE OF DATA Soho, London, 1854 The mystery of cholera The Hunt for the invisible cholera Cholera:
CC PROCESAMIENTO MASIVO DE DATOS OTOÑO Lecture 7: Information Retrieval II. Aidan Hogan
CC5212-1 PROCESAMIENTO MASIVO DE DATOS OTOÑO 2017 Lecture 7: Information Retrieval II Aidan Hogan aidhog@gmail.com How does Google know about the Web? Inverted Index: Example 1 Fruitvale Station is a 2013
CC PROCESAMIENTO MASIVO DE DATOS OTOÑO Lecture 12: Conclusion. Aidan Hogan
CC5212-1 PROCESAMIENTO MASIVO DE DATOS OTOÑO 2016 Lecture 12: Conclusion Aidan Hogan aidhog@gmail.com FULL-CIRCLE The value of data Twitter architecture Google architecture Generalise concepts to Working
Big Data - Some Words BIG DATA 8/31/2017. Introduction
BIG DATA Introduction Big Data - Some Words Connectivity Social Medias Share information Interactivity People Business Data Data mining Text mining Business Intelligence 1 What is Big Data Big Data means
CC PROCESAMIENTO MASIVO DE DATOS OTOÑO 2018
CC5212-1 PROCESAMIENTO MASIVO DE DATOS OTOÑO 2018 Lecture 6 Information Retrieval: Crawling & Indexing Aidan Hogan aidhog@gmail.com MANAGING TEXT DATA Information Overload If we didn t have search Contains
Big Data Analytics. Izabela Moise, Evangelos Pournaras, Dirk Helbing
Big Data Analytics Izabela Moise, Evangelos Pournaras, Dirk Helbing Izabela Moise, Evangelos Pournaras, Dirk Helbing 1 Big Data "The world is crazy. But at least it s getting regular analysis." Izabela
Web Search. Lecture Objectives. Text Technologies for Data Science INFR Learn about: 11/14/2017. Instructor: Walid Magdy
Text Technologies for Data Science INFR11145 Web Search Instructor: Walid Magdy 14-Nov-2017 Lecture Objectives Learn about: Working with Massive data Link analysis (PageRank) Anchor text 2 1 The Web Document
CPSC 340: Machine Learning and Data Mining. Density-Based Clustering Fall 2017
CPSC 340: Machine Learning and Data Mining Density-Based Clustering Fall 2017 Assignment 1 is due Friday. Admin Needs Julie 0.6 (you can use JuliaBox if you can t get Julia/PyPlot working). There was a
Challenges for Data Driven Systems
Challenges for Data Driven Systems Eiko Yoneki University of Cambridge Computer Laboratory Data Centric Systems and Networking Emergence of Big Data Shift of Communication Paradigm From end-to-end to data
Topics. Big Data Analytics What is and Why Hadoop? Comparison to other technologies Hadoop architecture Hadoop ecosystem Hadoop usage examples
Hadoop Introduction 1 Topics Big Data Analytics What is and Why Hadoop? Comparison to other technologies Hadoop architecture Hadoop ecosystem Hadoop usage examples 2 Big Data Analytics What is Big Data?
DIGITAL MARKETING AND SOCIAL MEDIA COMMUNICATION FOR THE NSS EPALE CY TEAM AND STAKEHOLDERS
DIGITAL MARKETING AND SOCIAL MEDIA COMMUNICATION FOR THE NSS EPALE CY TEAM AND STAKEHOLDERS 6 και 7 Ιουνίου, 8:00 π.μ.-15:00 μ.μ. Παιδαγωγικό Ινστιτούτο Κύπρου, Αίθουσα Π206 THE SEMINAR WAS HELD DURING
Fall Principles of Knowledge Discovery in Databases. University of Alberta
Principles of Knowledge Discovery in Databases Fall 1999 Dr. Osmar R. Zaïane 2 1 Class and Office Hours Class: Mondays, Wednesdays and Fridays from 10:00 to 10:50 Office Hours: Tuesdays from 11:00 to 11:55
Semantic Web Systems Introduction Jacques Fleuriot School of Informatics
Semantic Web Systems Introduction Jacques Fleuriot School of Informatics 11 th January 2015 Semantic Web Systems: Introduction The World Wide Web 2 Requirements of the WWW l The internet already there
Introduction to Text Mining. Hongning Wang
Introduction to Text Mining Hongning Wang CS@UVa Who Am I? Hongning Wang Assistant professor in CS@UVa since August 2014 Research areas Information retrieval Data mining Machine learning CS@UVa CS6501:
Security analytics: From data to action Visual and analytical approaches to detecting modern adversaries
Security analytics: From data to action Visual and analytical approaches to detecting modern adversaries Chris Calvert, CISSP, CISM Director of Solutions Innovation Copyright 2013 Hewlett-Packard Development
TIM 50 - Business Information Systems
TIM 50 - Business Information Systems Lecture 15 UC Santa Cruz Nov 10, 2016 Class Announcements n Database Assignment 2 posted n Due 11/22 The Database Approach to Data Management The Final Database Design
TIM 50 - Business Information Systems
TIM 50 - Business Information Systems Lecture 15 UC Santa Cruz May 20, 2014 Announcements DB 2 Due Tuesday Next Week The Database Approach to Data Management Database: Collection of related files containing
CSE 544 Principles of Database Management Systems
CSE 544 Principles of Database Management Systems Lecture 1 - Introduction and the Relational Model 1 Outline Introduction Class overview Why database management systems (DBMS)? The relational model 2
STATS Data Analysis using Python. Lecture 7: the MapReduce framework Some slides adapted from C. Budak and R. Burns
STATS 700-002 Data Analysis using Python Lecture 7: the MapReduce framework Some slides adapted from C. Budak and R. Burns Unit 3: parallel processing and big data The next few lectures will focus on big
CC PROCESAMIENTO MASIVO DE DATOS OTOÑO Lecture 6: Information Retrieval I. Aidan Hogan
CC5212-1 PROCESAMIENTO MASIVO DE DATOS OTOÑO 2017 Lecture 6: Information Retrieval I Aidan Hogan aidhog@gmail.com Postponing MANAGING TEXT DATA Information Overload If we didn t have search Contains all
CS 6240: Parallel Data Processing in MapReduce: Module 1. Mirek Riedewald
CS 6240: Parallel Data Processing in MapReduce: Module 1 Mirek Riedewald Why Parallel Processing? Answer 1: Big Data 2 How Much Information? Source: http://www2.sims.berkeley.edu/research/projects/ho w-much-info-2003/execsum.htm
Knowledge Discovery & Data Mining
Announcements ISM 50 - Business Information Systems Lecture 17 Instructor: Magdalini Eirinaki UC Santa Cruz May 29, 2007 News Folio #3 DUE Thursday 5/31 Database Assignment DUE Tuesday 6/5 Business Paper
microsoft
70-775.microsoft Number: 70-775 Passing Score: 800 Time Limit: 120 min Exam A QUESTION 1 Note: This question is part of a series of questions that present the same scenario. Each question in the series
Web Scraping XML/JSON. Ben McCamish
Web Scraping XML/JSON Ben McCamish We Have a Lot of Data 90% of the world s data generated in last two years alone (2013) Sloan Sky Server stores 10s of TB per day Hadron Collider can generate 500 Exabytes
Introduction to Database Systems CSE 414
Introduction to Database Systems CSE 414 Lecture 24: Parallel Databases CSE 414 - Spring 2015 1 Announcements HW7 due Wednesday night, 11 pm Quiz 7 due next Friday(!), 11 pm HW8 will be posted middle of
CS / Cloud Computing. Recitation 3 September 9 th & 11 th, 2014
CS15-319 / 15-619 Cloud Computing Recitation 3 September 9 th & 11 th, 2014 Overview Last Week s Reflection --Project 1.1, Quiz 1, Unit 1 This Week s Schedule --Unit2 (module 3 & 4), Project 1.2 Questions
Introduction April 27 th 2016
Social Web Mining Summer Term 2016 1 Introduction April 27 th 2016 Dr. Darko Obradovic Insiders Technologies GmbH Kaiserslautern d.obradovic@insiders-technologies.de Outline for Today 1.1 1.2 1.3 1.4 1.5
ISM 50 - Business Information Systems
ISM 50 - Business Information Systems Lecture 17 Instructor: Magdalini Eirinaki UC Santa Cruz May 29, 2007 Announcements News Folio #3 DUE Thursday 5/31 Database Assignment DUE Tuesday 6/5 Business Paper
Introduction to Data Management CSE 344
Introduction to Data Management CSE 344 Lecture 25: Parallel Databases CSE 344 - Winter 2013 1 Announcements Webquiz due tonight last WQ! J HW7 due on Wednesday HW8 will be posted soon Will take more hours
3 Data, Data Mining. Chengkai Li
CSE4334/5334 Data Mining 3 Data, Data Mining Chengkai Li Department of Computer Science and Engineering University of Texas at Arlington Fall 2018 (Slides partly courtesy of Pang-Ning Tan, Michael Steinbach
Prof. Dr. Christian Bizer
STI Summit July 6 th, 2011, Riga, Latvia Global Data Integration and Global Data Mining Prof. Dr. Christian Bizer Freie Universität ität Berlin Germany Outline 1. Topology of the Web of Data What data
CS639: Data Management for Data Science. Lecture 1: Intro to Data Science and Course Overview. Theodoros Rekatsinas
CS639: Data Management for Data Science Lecture 1: Intro to Data Science and Course Overview Theodoros Rekatsinas 1 2 Big science is data driven. 3 Increasingly many companies see themselves as data driven.
Computing Yi Fang, PhD
Computing Yi Fang, PhD Department of Computer Engineering Santa Clara University yfang@scu.edu http://www.cse.scu.edu/~yfang/ 1 This Talk Part I Computing Part II Computing at SCU Part III Data Science
DATA MINING INTRO LECTURE. Introduction
DATA MINING INTRO LECTURE Introduction Instructors Aris (Aris Anagnostopoulos, lectures) Yiannis (Ioannis Chatzigiannakis, lab) Adriano (Adriano Fazzone, Teaching Assistant) Mailing list Register to the
High Performance Computing on MapReduce Programming Framework
International Journal of Private Cloud Computing Environment and Management Vol. 2, No. 1, (2015), pp. 27-32 http://dx.doi.org/10.21742/ijpccem.2015.2.1.04 High Performance Computing on MapReduce Programming
Edward A. Fox, Yinlin Chen, Monika Akbar Virginia Tech. Edward A. Fox, Yinlin Chen, Monika Akbar. Virginia Tech
Edward A. Fox, Yinlin Chen, Monika Akbar Virginia Tech Edward A. Fox, Yinlin Chen, Monika Akbar Virginia Tech Currently, users have to login to their account and go to My Unread to see the recent post.
High Performance Computing in C and C++
High Performance Computing in C and C++ Rita Borgo Computer Science Department, Swansea University WELCOME BACK Course Administration Contact Details Dr. Rita Borgo Home page: http://cs.swan.ac.uk/~csrb/
Data Intensive Scalable Computing. Thanks to: Randal E. Bryant Carnegie Mellon University
Data Intensive Scalable Computing Thanks to: Randal E. Bryant Carnegie Mellon University http://www.cs.cmu.edu/~bryant Big Data Sources: Seismic Simulations Wave propagation during an earthquake Large-scale
Demystifying Machine Learning
Demystifying Machine Learning Dmitry Figol, WW Enterprise Sales Systems Engineer - Programmability @dmfigol CTHRST-1002 Agenda Machine Learning examples What is Machine Learning Types of Machine Learning
Embedded Technosolutions
Hadoop Big Data An Important technology in IT Sector Hadoop - Big Data Oerie 90% of the worlds data was generated in the last few years. Due to the advent of new technologies, devices, and communication
Towards a Cross- Disciplinary Pedagogy for Big Data. Joshua Eckroth Math/CS Department Stetson University CCSC- Eastern 2015
Towards a Cross- Disciplinary Pedagogy for Big Data Joshua Eckroth Math/CS Department Stetson University CCSC- Eastern 2015 What is big data? Data mining and analysis require big data techniques when
CIS : Scalable Data Analysis
CIS 602-01: Scalable Data Analysis Visualization Dr. David Koop Growth of Data 2 Usefulness of Data 3 Analyzed Data 4 Example Data Sources Radio Telescopes Twitter Wind Turbine Sensors Surveillance Cameras
The amount of data increases every day Some numbers ( 2012):
1 The amount of data increases every day Some numbers ( 2012): Data processed by Google every day: 100+ PB Data processed by Facebook every day: 10+ PB To analyze them, systems that scale with respect
2/26/2017. The amount of data increases every day Some numbers ( 2012):
The amount of data increases every day Some numbers ( 2012): Data processed by Google every day: 100+ PB Data processed by Facebook every day: 10+ PB To analyze them, systems that scale with respect to
CSE6331: Cloud Computing
CSE6331: Cloud Computing Leonidas Fegaras University of Texas at Arlington c 2019 by Leonidas Fegaras Cloud Computing Fundamentals Based on: J. Freire s class notes on Big Data http://vgc.poly.edu/~juliana/courses/bigdata2016/
The Fusion Distributed File System
Slide 1 / 44 The Fusion Distributed File System Dongfang Zhao February 2015 Slide 2 / 44 Outline Introduction FusionFS System Architecture Metadata Management Data Movement Implementation Details Unique
Turning Lookers into Bookers!
Turning Lookers into Bookers! Rachel Howes, Great National Hotels Ian Sloan, Avvio Avvio Operates Worldwide 450 4 280m Clients Offices Globally Hotel revenue processed annually Great National Hotels Acquisition
Databases and Information Retrieval Integration TIETS42. Kostas Stefanidis Autumn 2016
+ Databases and Information Retrieval Integration TIETS42 Autumn 2016 Kostas Stefanidis kostas.stefanidis@uta.fi http://www.uta.fi/sis/tie/dbir/index.html http://people.uta.fi/~kostas.stefanidis/dbir16/dbir16-main.html
CMPS 182: Introduction to Database Management Systems. Instructor: David Martin TA: Avi Kaushik. Syllabus
CMPS 182: Introduction to Database Management Systems Instructor: David Martin TA: Avi Kaushik Syllabus Course Content Relational database features & operations Data definition, queries and update Indexes,
Apache Openoffice Writer
Apache Openoffice Writer 1 / 6 2 / 6 3 / 6 Apache Openoffice Writer Apache OpenOffice Writer. Letting you concentrate on writing your ideas. Writer has everything you would expect from a modern, fully
Lecture 11. Lecture 11: External Sorting
Lecture 11 Lecture 11: External Sorting Lecture 11 Announcements 1. Midterm Review: This Friday! 2. Project Part #2 is out. Implement CLOCK! 3. Midterm Material: Everything up to Buffer management. 1.
CISC 7610 Lecture 2b The beginnings of NoSQL
CISC 7610 Lecture 2b The beginnings of NoSQL Topics: Big Data Google s infrastructure Hadoop: open google infrastructure Scaling through sharding CAP theorem Amazon s Dynamo 5 V s of big data Everyone
Data Analytics at Logitech Snowflake + Tableau = #Winning
Welcome # T C 1 8 Data Analytics at Logitech Snowflake + Tableau = #Winning Avinash Deshpande I am a futurist, scientist, engineer, designer, data evangelist at heart Find me at Avinash Deshpande Chief
Course Introduction. CSC343 - Introduction to Databases Manos Papagelis
Course Introduction CSC343 - Introduction to Databases Manos Papagelis Thanks to Ryan Johnson, John Mylopoulos, Arnold Rosenbloom and Renee Miller for material in these slides Overview 2 What is a database?
Networked Systems and Services, Fall 2018 Chapter 1. Jussi Kangasharju Markku Kojo Lea Kutvonen
Networked Systems and Services, Fall 2018 Chapter 1 Jussi Kangasharju Markku Kojo Lea Kutvonen Course topic and goals Networked systems and services In particular, their reliability from various points
Code Analysis Via Version Control History
Code Analysis Via Version Control History Justin Mclean Class Software Email: justin@classsoftware.com Twitter: @justinmclean Blog: http://blog.classsoftware.com Who am I? Freelance Developer Programming
CSE 124: Networked Services Fall 2009 Lecture-19
CSE 124: Networked Services Fall 2009 Lecture-19 Instructor: B. S. Manoj, Ph.D http://cseweb.ucsd.edu/classes/fa09/cse124 Some of these slides are adapted from various sources/individuals including but
Exam Questions
Exam Questions 70-775 Perform Data Engineering on Microsoft Azure HDInsight (beta) https://www.2passeasy.com/dumps/70-775/ NEW QUESTION 1 You are implementing a batch processing solution by using Azure
What is the maximum file size you have dealt so far? Movies/Files/Streaming video that you have used? What have you observed?
Simple to start What is the maximum file size you have dealt so far? Movies/Files/Streaming video that you have used? What have you observed? What is the maximum download speed you get? Simple computation
Cloud Computing 2. CSCI 4850/5850 High-Performance Computing Spring 2018
Cloud Computing 2 CSCI 4850/5850 High-Performance Computing Spring 2018 Tae-Hyuk (Ted) Ahn Department of Computer Science Program of Bioinformatics and Computational Biology Saint Louis University Learning
CSE 124: THE DATACENTER AS A COMPUTER. George Porter November 20 and 22, 2017
CSE 124: THE DATACENTER AS A COMPUTER George Porter November 20 and 22, 2017 ATTRIBUTION These slides are released under an Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0) Creative
CSCI 5417 Information Retrieval Systems! What is Information Retrieval?
CSCI 5417 Information Retrieval Systems! Lecture 1 8/23/2011 Introduction 1 What is Information Retrieval? Information retrieval is the science of searching for information in documents, searching for
Hitachi Visualization Predictive Crime Analytics
Hitachi Visualization Predictive Crime Analytics Creating Safe Cities - The Problem Public safety budgets are tightening Violent crimes e.g., assault are increasing in many cities along with the shifting
Nowcasting. D B M G Data Base and Data Mining Group of Politecnico di Torino. Big Data: Hype or Hallelujah? Big data hype?
Big data hype? Big Data: Hype or Hallelujah? Data Base and Data Mining Group of 2 Google Flu trends On the Internet February 2010 detected flu outbreak two weeks ahead of CDC data Nowcasting http://www.internetlivestats.com/
Introduction to Databases Fall-Winter 2010/11. Syllabus
Introduction to Databases Fall-Winter 2010/11 Syllabus Werner Nutt Syllabus Lecturer Werner Nutt, nutt@inf.unibz.it, Room POS 2.09 Office hours: Tuesday, 14:00 16:00 and by appointment (If you want to
Science 2.0 VU Big Science, e-science and E- Infrastructures + Bibliometric Network Analysis
W I S S E N n T E C H N I K n L E I D E N S C H A F T Science 2.0 VU Big Science, e-science and E- Infrastructures + Bibliometric Network Analysis Elisabeth Lex KTI, TU Graz WS 2015/16 u www.tugraz.at
Microsoft. Exam Questions Perform Data Engineering on Microsoft Azure HDInsight (beta) Version:Demo
Microsoft Exam Questions 70-775 Perform Data Engineering on Microsoft Azure HDInsight (beta) Version:Demo NEW QUESTION 1 HOTSPOT You install the Microsoft Hive ODBC Driver on a computer that runs Windows
DS595/CS525: Urban Network Analysis --Urban Mobility Prof. Yanhua Li
Welcome to DS595/CS525: Urban Network Analysis --Urban Mobility Prof. Yanhua Li Time: 6:00pm 8:50pm Wednesday Location: Fuller 320 Spring 2017 2 Team assignment Finalized. (Great!) Guest Speaker 2/22 A
Introduction to Database Systems CSE 344
Introduction to Database Systems CSE 344 Lecture 10: Basics of Data Storage and Indexes 1 Student ID fname lname Data Storage 10 Tom Hanks DBMSs store data in files Most common organization is row-wise
/ Cloud Computing. Recitation 7 October 10, 2017
15-319 / 15-619 Cloud Computing Recitation 7 October 10, 2017 Overview Last week s reflection Project 3.1 OLI Unit 3 - Module 10, 11, 12 Quiz 5 This week s schedule OLI Unit 3 - Module 13 Quiz 6 Project
The Future of High Performance Computing
The Future of High Performance Computing Randal E. Bryant Carnegie Mellon University http://www.cs.cmu.edu/~bryant Comparing Two Large-Scale Systems Oakridge Titan Google Data Center 2 Monolithic supercomputer
Today. An Animated Introduction to Programming. Prerequisites. Computer programming
Today 1 2 3 4 Computer programming What is this course about? We re making several assumptions about you as a student. In particular, we assume that you have: Never taken a programming course before. Have
Programming. Dr Ben Dudson University of York
Programming Dr Ben Dudson University of York Outline Last lecture covered the basics of programming and IDL This lecture will cover More advanced IDL and plotting Fortran and C++ Programming techniques
Large-Scale Data Engineering. Overview and Introduction
Large-Scale Data Engineering Overview and Introduction Administration Blackboard Page Announcements, also via email (pardon html formatting) Practical enrollment, Turning in assignments, Check Grades Contact:
DEC Computer Technology LESSON 6: DATABASES AND WEB SEARCH ENGINES
DEC. 1-5 Computer Technology LESSON 6: DATABASES AND WEB SEARCH ENGINES Monday Overview of Databases A web search engine is a large database containing information about Web pages that have been registered
Speech 2 Part 2 Transcript: The role of DB2 in Web 2.0 and in the IOD World
Speech 2 Part 2 Transcript: The role of DB2 in Web 2.0 and in the IOD World Slide 1: Cover Welcome to the speech, The role of DB2 in Web 2.0 and in the Information on Demand World. This is the second speech
The Google File System
The Google File System Sanjay Ghemawat, Howard Gobioff and Shun Tak Leung Google* Shivesh Kumar Sharma fl4164@wayne.edu Fall 2015 004395771 Overview Google file system is a scalable distributed file system
Introduction to CS 4604
Introduction to CS 4604 T. M. Murali August 23, 2010 Course Information Instructor T. M. Murali, 2160B Torgerson, 231-8534, murali@cs.vt.edu Office Hours: 9:30am 11:30am Mondays and Wednesdays Teaching
CC PROCESAMIENTO MASIVO DE DATOS OTOÑO Lecture 7: Information Retrieval I. Aidan Hogan
CC5212-1 PROCESAMIENTO MASIVO DE DATOS OTOÑO 2016 Lecture 7: Information Retrieval I Aidan Hogan aidhog@gmail.com MANAGING TEXT DATA Information Overload If we didn t have search Contains all books with
Webinar Series TMIP VISION
Webinar Series TMIP VISION TMIP provides technical support and promotes knowledge and information exchange in the transportation planning and modeling community. Today s Goals To Consider: Parallel Processing
Project Revision. just links to Principles of Information and Database Management 198:336 Week 13 May 2 Matthew Stone
Project Revision Principles of Information and Database Management 198:336 Week 13 May 2 Matthew Stone Email just links to mdstone@cs Link to code (on the web) Link to writeup (on the web) Link to project
MASSIVE DATA NEEDS DISTRIBUTED SYSTEMS
CC5212-1 PROCESAMIENTO MASIVO DE DATOS OTOÑO 2015 Lecture 2: Distributed Systems I Aidan Hogan aidhog@gmail.com MASSIVE DATA NEEDS DISTRIBUTED SYSTEMS Monolithic vs. Distributed Systems Parallel vs. Distributed
Introduction to Database S ystems Systems CSE 444 Lecture 1 Introduction CSE Summer
Introduction to Database Systems CSE 444 Lecture 1 Introduction 1 Staff Instructor: Hal Perkins CSE 548, perkins@cs.washington.edu Office hours: labs tba, office drop-ins and appointments welcome TA: David
Modern Database Systems CS-E4610
Modern Database Systems CS-E4610 Aristides Gionis Michael Mathioudakis Spring 2017 what is a database? a collection of data what is a database management system?... a.k.a. database system software to store,
CSE 113 A. Announcements - Lab
CSE 113 A February 21-25, 2011 Announcements - Lab Lab 1, 2, 3, 4; Practice Assignment 1, 2, 3, 4 grades are available in Web-CAT look under Results -> Past Results and if looking for Lab 1, make sure
DATA MINING INTRO LECTURE. Introduction
DATA MINING INTRO LECTURE Introduction Instructors Aris (Aris Anagnostopoulos) Yiannis (Ioannis Chatzigiannakis) Evimaria (Evimaria Terzi) What is data mining? After years of data mining there is still
Introduction to Data Management CSE 344. Lecture 1: Introduction
Introduction to Data Management CSE 344 Lecture 1: Introduction CSE 344 - Winter 2014 1 Staff Instructor: Sudeepa Roy sudeepa@cs.washington.edu Office hours: Wednesdays, 3:30-4:20, in CSE 344 (my office)
From Internet Data Centers to Data Centers in the Cloud
From Internet Data Centers to Data Centers in the Cloud This case study is a short extract from a keynote address given to the Doctoral Symposium at Middleware 2009 by Lucy Cherkasova of HP Research Labs
Mining of Massive Datasets
Mining of Massive Datasets Krzysztof Dembczyński Intelligent Decision Support Systems Laboratory (IDSS) Poznań University of Technology, Poland Software Development Technologies Master studies, second
SCU SEEDs Workshop Angela Musurlian
SCU SEEDs Workshop Angela Musurlian Lecturer Department of Computer Engineering Santa Clara University amusurlian@scu.edu 1 This Talk Part I Computing Part II Computing at SCU Part III Today s activity
Data-Intensive Distributed Computing
Data-Intensive Distributed Computing CS 451/651 431/631 (Winter 2018) Part 5: Analyzing Relational Data (1/3) February 8, 2018 Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo
CS490W: Web Information Search & Management. CS-490W Web Information Search and Management. Luo Si. Department of Computer Science Purdue University
CS490W: Web Information Search & Management CS-490W Web Information Search and Management Luo Si Department of Computer Science Purdue University Overview Web: Growth of the Web The world produces between
Improved Solutions for I/O Provisioning and Application Acceleration
1 Improved Solutions for I/O Provisioning and Application Acceleration August 11, 2015 Jeff Sisilli Sr. Director Product Marketing jsisilli@ddn.com 2 Why Burst Buffer? The Supercomputing Tug-of-War A supercomputer
CIS 4930/6930 Spring 2014 Introduction to Data Science /Data Intensive Computing. University of Florida, CISE Department Prof.
CIS 4930/6930 Spring 2014 Introduction to Data Science /Data Intensive Computing University of Florida, CISE Department Prof. Daisy Zhe Wang Text To Knowledge IR and Boolean Search Text to Knowledge (IE)
Database Systems CSE 414
Database Systems CSE 414 Lecture 10: Basics of Data Storage and Indexes 1 Reminder HW3 is due next Tuesday 2 Motivation My database application is too slow why? One of the queries is very slow why? To
Distributed Systems Intro and Course Overview
Distributed Systems Intro and Course Overview COS 418: Distributed Systems Lecture 1 Wyatt Lloyd Distributed Systems, What? 1) Multiple computers 2) Connected by a network 3) Doing something together Distributed
Information Retrieval
Multimedia Computing: Algorithms, Systems, and Applications: Information Retrieval and Search Engine By Dr. Yu Cao Department of Computer Science The University of Massachusetts Lowell Lowell, MA 01854,
Online Communication. Chat Rooms Instant Messaging Blogging Social Media
Online Communication E-mail Chat Rooms Instant Messaging Blogging Social Media { Advantages: { Reduces cost of postage Fast and convenient Need an email address to sign up for other online accounts. Eliminates
COMP283-Lecture 3 Applied Database Management
COMP283-Lecture 3 Applied Database Management Introduction DB Design Continued Disk Sizing Disk Types & Controllers DB Capacity 1 COMP283-Lecture 3 DB Storage: Linear Growth Disk space requirements increases
CAS CS 460/660 Introduction to Database Systems. Fall
CAS CS 460/660 Introduction to Database Systems Fall 2017 1.1 About the course Administrivia Instructor: George Kollios, gkollios@cs.bu.edu MCS 283, Mon 2:30-4:00 PM and Tue 1:00-2:30 PM Teaching Fellows: