Privacy Overview and Data Mining CSC 301 Spring 2018 Howard Rosenthal

Transcription

1 Privacy Overview and Data Mining CSC 301 Spring 2018 Howard Rosenthal

2 Course Notes: Much of the material in the slides comes from the books and their associated support materials, below as well as many of the references at the class web site Baase, Sara and Henry, Timothy, A Gift of Fire: Social, Legal, and Ethical Issues for Computing Technology (5th Edition) Pearson, March 9, 2017, ISBN-13: Quinn, Michael, Ethics for the Information Age (7th Edition), Pearson, Feb. 21, 2016,ISBN

3 Lesson Goals Basic principles in privacy Defining privacy Threats to privacy Impacts of technology on privacy Securing personal privacy Technology excursion Data Mining 3

4 4

5 There Are Many Aspects To Security and Privacy 5

6 What Is Privacy? Is privacy a Natural Right Is privacy a type of property? If you invade a person s privacy it can be a major coercive force Privacy used to be fairly simple Your home could not be invaded, nor your property seized, without due process Today your private information is everywhere On the net On your phone On your computer In the cloud In your employer s databases With the government Even if the people you give information to do not misuse that information, the information is more susceptible to theft via hacking or other mischief than ever before Recently the Federal Government s Office of Personal Management was hacked and detailed information on everyone with a security clearance was stolen Government accepted very little responsibility for this theft 6

7 There Are Three Key Aspects To Privacy Freedom from intrusion Control over information about oneself Freedom from surveillance (physical, electronic, etc.) 7

8 Our Privacy Is Always Being Threatened There are many threats to our privacy Intentional use or misuse of information by businesses or government Unauthorized release to insiders by information maintainers Theft of information by criminals or hostile governments Inadvertent leakage through negligence or carelessness Our own actions, such as posting too much data on the Internet for either benign (B) or malicious (M) purposes Give to one charity and ten others will come knocking (B) List of off color movies you may have watched (M) - used to discredit you Divorce proceeding (M) sometimes used by politicians Stealing financial data (M) used to open loans, buy homes, etc. all in your name 8

9 9

10 New Technology Creates Many New Opportunities To Invade Our Privacy Some of these threats combine both low tech techniques, such as eavesdropping or looking over a shoulder, with high tech techniques Government and private databases Sophisticated tools for surveillance and data analysis Vulnerability of data Search engines collect many terabytes of data daily. Data is analyzed to target advertising and develop new services. Who gets to see this data? Why should we care? This same data, when aggregated, creates a detailed biography of you Data collected for one purpose will find other uses Assume that everything in cyberspace is recorded and replicated You create new potential security leaks every day Facebook s Texts Map instructions Twitter If information is on a public Web site, it is available to everyone If you post pictures of your vacation while you are on it you may come home to an empty house 10

11 Re-identification Re-identification is the process of identifying individuals using anonymous data. Re-identification has become much easier due to the quantity of information and power of data search and analysis tools A collection of small items can be aggregated to provide a detailed picture Your search history could identify who you are. Working backwards from the metadata is often possible with enough computing power and data. Reporters often use anonymous data as they work towards identifying individuals. If information is on a public Web site, it is often available to everyone 11

12 Personal Security and Privacy Are Often Threatened By Our Own Actions 12

13 Everything You Access May Be Monitored Search Engines May record all your searches If you search for a book on Amazon you ll get s about that book or others every few days Some of your searches you may want to keep private Looking for a new job Searching for certain specific products Medical searches Smartphones Are often transmitting location data Great if a phone is lost or stolen Horrible if a house thief gets the data Passwords and codes for key accounts are often stored without your knowledge and then uploaded to the cloud with other data If the cloud is hacked your information may be on the market without your knowledge Contact lists can be compromised Photos may be gathered and subjected to various forms of analysis Software Many pieces of software record all types of data This data may ultimately be collected and analyzed Sometimes it simply sits forgotten until someone decides to see what s there 13

14 Managing Personal Data Terminology and Principles (1) Personal information is any information relating to an individual person Invisible information gathering Data collected without your knowledge Always read the fine print How often do you click agree when downloading? This is an ethical issue Cookies Files a Web site stores on a visitor s computer Secondary use Use of personal information for a purpose other than the purpose for which it was provided Sale of consumer information you marketers or other businesses Use of information in various databases to deny someone a job Use of vehicle registrations by the IRS to find persons with high incomes Use of text messages to prosecute for a crime Using your information in an illegal manner after stealing or gleaning it from legitimate sources 14

15 Managing Personal Data Terminology and Principles (2) Data mining Searching and analyzing masses of data to find patterns and develop new information or knowledge Computer matching Combining and comparing information from different databases (using social security number, for example) to match records. Computer profiling Analyzing data to determine characteristics of people most likely to engage in a certain behavior 15

16 Informed Consent Provides An Ethical Framework For Information Collection Informed Consent You must agree before your information can be collected or used Could be used to pressure you if you are denied a service without agreeing to share this data LoJack collects information about your car location continuously was this informed consent The AAA tried collecting information by asking you if you d like to hook data collectors into your car then they reported that data to the insurance side of the house Two common forms for providing informed consent are opt out and opt in: In opt out a person must request (usually by checking a box) that an organization not use information. In opt in the collector of the information may use information only if person explicitly permits use (usually by checking a box). Discussion Question: Have you seen opt-in and opt-out choices? Where? How were they worded? Were any of them deceptive? 16

17 Fair Information Principles A basic set of principles for businesses to handle data in an ethical way Inform people when you collect data Collect only the data that is needed Make opt in your default Offer opt out methods that can be used at any time It is harder to ensure if all data is deleted if you opt in and then opt out Keep data only as long as is need Maintain accuracy of data Protect the data. Use all reasonable security methods to do so. Develop policies for responding to law enforcement requests Many government organizations are developing guidelines FTC Fair Information Practice Principles.pdf 17

18 Data Mining 18

19 What Is Data Mining? Data mining is defined as extracting information from huge sets of data. In other words, we can say that data mining is the procedure of mining knowledge from data. Data mining can integrate many different data sets The information or knowledge extracted from data mining can be used for any of the following applications Profiling This is where privacy really gets involved Customer Retention Pattern Analysis Market Analysis Fraud Detection Production Control Science Exploration 19

20 Data Mining Tasks Data mining deals with the kind of patterns that can be mined. On the basis of the kind of data to be mined, there are two categories of functions involved in Data Mining The Descriptive Function deals with the general properties of data in the database. Class/Concept Description Mining of Frequent Patterns Mining of Associations Mining of Correlations Mining of Clusters Classification and Prediction is the process of finding a model that describes the data classes or concepts. The purpose is to be able to use this model to predict the class of objects whose class label is unknown. This derived model is based on the analysis of sets of training data. The derived model can be presented in the following forms Classification (IF-THEN) Rules Decision Trees Mathematical Formulae Neural Networks 20

21 Descriptive Tasks In Data Mining (1) The Class/Concept Description refers to the data to be associated with the classes or concepts. For example, in a company, the classes of items for sales include computer and printers, and concepts of customers include big spenders and budget spenders. Such descriptions of a class or a concept are called class/concept descriptions. These descriptions can be derived by the following two ways Data Characterization refers to summarizing data of class under study. This class under study is called as Target Class. Data Discrimination refers to the mapping or classification of a class with some predefined group or class. Mining of Frequent Patterns looks at patterns are those patterns that occur frequently in transactional data. The list of kind of frequent patterns includes The Frequent Item Set is a set of items that frequently appear together, for example, milk and bread. The Frequent Subsequence is a sequence of patterns that occur frequently such as purchasing a camera is followed by memory card. The Frequent Sub Structure refers to different structural forms, such as graphs, trees, or lattices, which may be combined with item sets or subsequences. 21

22 Descriptive Tasks In Data Mining (2) Mining of Association This process refers to the process of uncovering the relationship among data and determining association rules. Associations are used in retail sales to identify patterns that are frequently purchased together, helping to identify potential buyers For example, a retailer generates an association rule that shows that 70% of time milk is sold with bread while only 30% of times are biscuits sold with bread. Mining of Correlations Additional analysis performed to uncover interesting statistical correlations between associated-attribute value pairs or between two item sets to analyze that if they have positive, negative or no effect on each other. Want to understand if there is actual causation Mining of Clusters Cluster refers to a group of similar kind of objects. Cluster analysis refers to forming group of objects that are very similar to each other but are highly different from the objects in other clusters. Can group by gender, age, home location, language,. 22

23 Classification and Prediction Functions Classification It predicts the class of objects whose class label is unknown. Its objective is to find a derived model that describes and distinguishes data classes or concepts. The Derived Model is based on the analysis set of training data i.e. the data object whose class label is well known. Prediction It is used to predict missing or unavailable numerical data values rather than class labels. Regression Analysis is generally used for prediction. Prediction can also be used for distribution trends based on available data. Outlier Analysis Outliers may be defined as the data objects that do not comply with the general behavior or model of the data available. Evolution Analysis Evolution analysis refers to the description and model regularities or trends for objects whose behavior changes over time. 23

24 Data Warehousing Data warehousing is the process of constructing and using the data warehouse. A data warehouse is constructed by integrating the data from multiple heterogeneous sources. It supports analytical reporting, structured and/or ad hoc queries, and decision making. Data warehousing involves data cleaning, data integration, and data consolidations. To integrate heterogeneous databases, we have the following two approaches Query Driven Approach Update Driven Approach 24

25 Query Driven Approach This is the traditional approach to integrate heterogeneous databases. Builds wrappers and integrators on top of multiple heterogeneous databases. These integrators are also known as mediators. The process of the Query Driven Approach When a query is issued to a client side, a metadata dictionary translates the query into one or more queries, appropriate for the individual heterogeneous site involved. Now these queries are mapped and sent to the local query processor. The results from heterogeneous sites are integrated into a global answer set. Advantages Government doesn t get to keep a large data base of information on permanent file Don t need to maintain a large IT infrastructure Disadvantages The Query Driven Approach needs complex integration and filtering processes. It is very inefficient and very expensive for frequent queries. This approach is expensive for queries that require aggregations (constant regrouping) of data 25

26 Update Driven Approach Today's data warehouse systems follow update-driven approach rather than the traditional approach discussed earlier. In the update-driven approach, the information from multiple heterogeneous sources is integrated in advance and stored in a warehouse. This includes data scrubbing the process of validating data for correctness in advance This information is available for direct querying and analysis. Advantages This approach provides high performance. The data can be copied, processed, integrated, annotated, summarized and restructured in the semantic data store in advance. In other words, we store data in the way(s) we want to look at it Query processing does not require an interface with the processing at the local original data sources. Much less intrusive and resource intensive to pull the data once, rather than whenever you want to query Disadvantages Must maintain a large infrastructure to import, store and maintain data Privacy concerns since the government now has access to so much data The whole debate on the Patriot Act centered around whether or not the government could continuously collect and store metadata from the ISPs and cell/land-line phone providers A political/privacy argument conflicted with a technical argument 26

27 Data Warehousing and Data Mining Online Analytical Mining integrates with Online Analytical Processing to discover knowledge across multidimensional databases. 27

28 On-line Analytical Mining On-line Analytical Mining (OLAM) has the following important attributes High quality of data in data warehouses The data mining tools are required to work on integrated, consistent, and cleaned data which are very costly in the preprocessing of data. The data warehouses constructed by such preprocessing are valuable sources of high quality data for OLAP and data mining as well. A complex information processing infrastructure surrounds each data warehouses Information processing infrastructure refers to accessing, integration, consolidation, and transformation of multiple heterogeneous databases, webaccessing and service facilities, reporting and OLAP analysis tools. On-line Analytical Processing (OLAP) based exploratory data analysis Exploratory data analysis is required for effective data mining. OLAP provides facilities for data mining on various subset of data and at different levels of abstraction. Online selection of data mining functions Integrating OLAP with multiple data mining functions and online analytical mining provides users with the flexibility to select desired data mining functions and swap data mining tasks dynamically. 28

29 Steps In Data Mining Data Cleaning The noise and inconsistent data is removed. Data Integration Multiple data sources are combined. Data Selection Data relevant to the analysis task are retrieved from the database. Data Transformation Data is transformed or consolidated into forms appropriate for mining by performing summary or aggregation operations. Data Mining Intelligent methods are applied in order to extract data patterns. Pattern Evaluation Data patterns are evaluated. Knowledge Presentation Knowledge is represented, often graphically 29

30 The Process of Knowledge Discovery 30

31 Multi-Dimensional Databases Multidimensional structures use a variation of the relational model to organize data and express the relationships between data. More complex than the typical row/column relational database. Each cell within a multidimensional structure contains aggregated data related to elements along each of its dimensions Time is an additional dimension used in the analysis of data 31

32 Example Of A Multi-Dimensional Database Structure 32