/ Computational Genomics. Normalization

Size: px
Start display at page:

Download "/ Computational Genomics. Normalization"

Transcription

1 / Computational Genomics Normalization

2 Genes and Gene Expression Technology Display of Expression Information

3 Yeast cell cycle expression Experiments (over time) baseline expression program gene genes Higher expression compared to baseline Lower expression compared to baseline Spellman et al Mol. Biol. Cell 1998

4 ROS response

5 Visualization: Relative vs. absolute expression

6 Exercising the Genome 600 Conditions/Mutations 6200 Genes Environment Single-gene Mutations

7 Using annotation databases Statistical tests to identify the overlap with various functional categories

8

9 Genome wide binding experiments (transcription factors) no binding by this TF gene 1 TF1 TF8 genes Probably bound by this TF Lee et al Science 2002

10 What you should know The basic idea behind microarray profiling The two different microarray technologies Pros and cons for each Noise factors in microarray experiments (more next)

11 Gene expression analysis

12 Gene Expression Analysis Model Computational information fusion Biological regulatory networks Pattern Recognition Data Analysis clustering, classification normalization, miss. value estimation functional assignment, response programs diff. expressed genes Experimental Design array design, number of repeats experiment selection

13 Experiment design A number of computational issues should be addressed: Selecting short subsequences for oligo arrays to minimize cross hybridizations Determining the number of replicates for each sample Sampling rates for time series experiments

14 Data analysis Normalization Combining results from replicates Identifying differentially expressed genes Dealing with missing values Static vs. time series

15 Data analysis Normalization Combining results from replicates Identifying differentially expressed genes Dealing with missing values Static vs. time series

16 Typical experiment: replicates healthy cancer Technical replicates: same sample using multiple arrays Dye swap: reverse the color code between arrays Clinical replicates: samples from different individuals Many experiments have all three kinds of replicates

17 Normalization Single Channel Background Correction Probe Set Summarization (Affymetrix) Between Array Normalization Normalization Two Channel Array

18 Background Correction On Affymetrix major normalization software (e.g. MAS 5, dchip, RMA, gcrma) have different approaches to background correction On Agilent the feature extraction software gives various options on ways to do background correction Image from Agilent User Manual

19 Agilent Background Correction Table from Agilent User Manual

20 Normalization Single Channel Background Correction Between Array Normalization Normalization Probe Set Summarization (Affymetrix) Two Channel Array

21 In theory these really should have the same distribution Image from Terry Speed s Slides Ideally normalization should remove variation between microarray arrays unrelated to the biologly of interest, without removing the variation due to the biology of interest

22 Two repeats from Agilent mouse expression data

23 Normalization: Assumptions In order to normalize between arrays we need to make certain assumptions. Methods we will discuss will assume (with decreasing order of assumption strength): 1. Values are exactly the same between the arrays (though different genes may be assigned different values in each array). 2. Values are normally distributed with the same mean and variance across arrays. 3. Some of the genes do not change between arrays and thus should have relatively similar values (rank invariant).

24 Three Major Approaches to Between Array Normalization Quantile Normalization Scale Factor Normalization Invariant Set Normalization 1. Values are exactly the same between arrays (though different genes may be assigned different values in each array). 2. Values are normally distributed with the same mean and variance across arrays. 3. Some of the genes do not change between arrays and thus should have relatively similar values (rank invariant).

25 Three Major Approaches to Between Array Normalization Quantile Normalization Scale Factor Normalization Invariant Set Normalization 1. Values are exactly the same between arrays (though different genes may be assigned different values in each array). 2. Values are normally distributed with the same mean and variance across arrays. 3. Some of the genes do not change between arrays and thus should have relatively similar values (rank invariant).

26 Normalizing across arrays Consider the following two sets of values:

27 Lets put them together

28 Normalizing between arrays The first step in the analysis of microarray data in a given experiment is to normalize between the different arrays. Simple assumption: mrna quantity is the same for all arrays M j = 1 n n! i= 1 y i j Where n and T are the total number of genes and arrays, respectfully. M j is known as the sample mean M Next we transform each value to make all arrays have the same mean: j j j yˆ i = yi! M + M = 1 T T! j= 1 M j

29 Normalizing the mean

30 Variance normalization In many cases normalizing the mean is not enough. We may further assume that the variance should be the same for each array Implicitly we assume that the expression distribution is the same for all arrays (though different genes may change in each of the arrays) Here V j is the sample variance. Next, we transform each value as follows:!! = = = " = T j j n i j j j V T V M y n V i ) ( 1 ( ) j j j i j i V V M M y y +! = ˆ

31 Normalizing mean and variance

32 Mean Scaling Before After

33 Distribution is still not identical

34 Three Major Approaches to Between Array Normalization Quantile Normalization Scale Factor Normalization Invariant Set Normalization 1. Values are exactly the same between arrays (though different genes may be assigned different values in each array). 2. Values are normally distributed with the same mean and variance across arrays. 3. Some of the genes do not change between arrays and thus should have relatively similar values (rank invariant).

35 Quantile Normalization Maps the distribution of expression values for every microarray experiment to the same target distribution This is the approach of Terry Speed s group s RMA software Also an option in Agilent s Gene Spring Software

36 Mapping raw data to normalized data Image from Terry Speed s Slides

37 Normalized distribution For each order all the values in the array Take the mean (or median) value in each position Sort Mean C B A C B A Normalized

38 Quantile Example Image from Terry Speed s Slides

39 Three Major Approaches to Between Array Normalization Quantile Normalization Scale Factor Normalization Invariant Set Normalization 1. Values are exactly the same between arrays (though different genes may be assigned different values in each array). 2. Values are normally distributed with the same mean and variance across arrays. 3. Some of the genes do not change between arrays and thus should have relatively similar values (rank invariant).

40 Invariant Set Normalization Normalization based on probes that do not change between data Housekeeping Genes Control Probes Rank invariant genes This is the approach of Wing Wong s dchip software

41 dchip Invariant Set Normalization Select one baseline array Default Median Intensity All other arrays are normalized against baseline Computationally finds rank invariant probes Different genes may be used for different arrays Running Median curve through rank invariant probes becomes the new y=x curve, all values adjust accordingly

42 dchip on Replicate Data Spline on All Unnormalized Data Running Median Line Normalized Data Q-Q Plot Image From Li and Wong, Genome Biology 2001

43 dchip on data with true differentially expressed genes Spline on All Unnormalized Data Running Median Line Normalized Data Q-Q Plot Differently expressed genes not selected as rank invariant Image From Li and Wong, Genome Biology 2001

44 Normalizing Across Very Different Distributions can be Problematic May want to normalize subsets from different tissues separately Image From dchip manual

45 cdna arrays: ratios healthy cancer In many experiments we are interested in the ratio between two samples For example - Cancer vs. healthy - Progression of disease (ratio to time point 0)

46 Transformation While ratios are useful, they are not symmetric. If R = 2*G then R/G = 2 while G/R = ½ This makes it hard to visualize the different changes Instead, we use a log transform, and focus on the log ratio: y i = R i log = log Ri! Gi logg i Empirical studies have also shown that in microarray experiments the log ratio of (most) genes tends to be normally distributed

47 Normalizing between array: Locally weighted linear regression Normalizing the mean and the variance works well if the variance is independent of the measured value. However, this is not the case in gene expression. For microarrays it turns out that the variance is value dependent.

48 Locally weighted linear regression Instead of computing a single mean and variance for each array, we can compute different means and variances for different expression values. Given two arrays, R and G, or two channels on the same array, we plot on the x axis the (log) of their intensity and on the y axis their ratio We are interested in normalizing the average (log) expression ratio for the different intensity values

49 Computing local mean and variance Setting may work, however, it requires that many genes have the same x value, which is usually not the case Instead, we can use a weighted sum where the weight is propotional to the distance of the point from x:!! = = " = = x x i x x i i i x m y k x v y k x m 2 )) ( ( 1 ) ( 1 ) (!!!! " = = i i i i i i i i i i x m y x w x w x v y x w x w x m 2 )) ( ) ( ( ) ( 1 ) ( ) ( ) ( 1 ) ( ( ) ) ( ) ( ) ( ) ˆ( x v V M x m x y x y +! =

50 Determining the weights There are a number of ways to determine the weights Here we will use a Gaussian centered at x, such that 2 ( x# x i ) # 2 w( ) = 2! x i 1 2"! e σ 2 is a parameter that should be selected by the user

51 Locally weighted regression: Results Original values normalized values

52 Oligo arrays: Negative values In many cases oligo array can return values that are less than 0 (Why?) There are a number of ways to handle these values The most common is to threshold at a certain positive value A more sophisticated way is to use the negative values to learn something about the variance of the specific gene

53 Data analysis Normalization Combining results from replicates Identifying differentially expressed genes Dealing with missing values Static vs. time series

54 Motivation In many cases, this is the goal of the experiment. Such genes can be key to understanding what goes wrong / or get fixed under certain condition (cancer, stress etc.). In other cases, these genes can be used as features for a classifier. These genes can also serve as a starting point for a model for the system being studied (e.g. cell cycle, phermone response etc.).

55 Problems As mentioned in the previous lecture, differences in expression values can result from many different noise sources. Our goal is to identify the real differences, that is, differences that can be explained by the various errors introduced during the experimental phase. Need to understand both the experimental protocol and take into account the underlying biology / chemistry

56 The wrong way During the early days (though some continue to do this today) the common method was to select genes based on their fold change between experiments. The common value was 2 (or absolute log of 1). Obviously this method is not perfect

57 Significance bands for Affy arrays

58 Value dependent variance

59 Typical experiment: replicates healthy cancer Technical replicates: same sample using multiple arrays Dye swap: reverse the color code between arrays Clinical replicates: samples from different individuals Many experiments have all three kinds of replicates

60 What you should know The different noise factors that influence microarray results The assumptions and corresponding methods for normalizing oligo (single channel) arrays. Methods for normalizing two channel (cdna) arrays.

Course on Microarray Gene Expression Analysis

Course on Microarray Gene Expression Analysis Course on Microarray Gene Expression Analysis ::: Normalization methods and data preprocessing Madrid, April 27th, 2011. Gonzalo Gómez ggomez@cnio.es Bioinformatics Unit CNIO ::: Introduction. The probe-level

More information

Preprocessing -- examples in microarrays

Preprocessing -- examples in microarrays Preprocessing -- examples in microarrays I: cdna arrays Image processing Addressing (gridding) Segmentation (classify a pixel as foreground or background) Intensity extraction (summary statistic) Normalization

More information

Microarray Data Analysis (V) Preprocessing (i): two-color spotted arrays

Microarray Data Analysis (V) Preprocessing (i): two-color spotted arrays Microarray Data Analysis (V) Preprocessing (i): two-color spotted arrays Preprocessing Probe-level data: the intensities read for each of the components. Genomic-level data: the measures being used in

More information

Introduction to GE Microarray data analysis Practical Course MolBio 2012

Introduction to GE Microarray data analysis Practical Course MolBio 2012 Introduction to GE Microarray data analysis Practical Course MolBio 2012 Claudia Pommerenke Nov-2012 Transkriptomanalyselabor TAL Microarray and Deep Sequencing Core Facility Göttingen University Medical

More information

Microarray Data Analysis (VI) Preprocessing (ii): High-density Oligonucleotide Arrays

Microarray Data Analysis (VI) Preprocessing (ii): High-density Oligonucleotide Arrays Microarray Data Analysis (VI) Preprocessing (ii): High-density Oligonucleotide Arrays High-density Oligonucleotide Array High-density Oligonucleotide Array PM (Perfect Match): The perfect match probe has

More information

Gene Expression an Overview of Problems & Solutions: 1&2. Utah State University Bioinformatics: Problems and Solutions Summer 2006

Gene Expression an Overview of Problems & Solutions: 1&2. Utah State University Bioinformatics: Problems and Solutions Summer 2006 Gene Expression an Overview of Problems & Solutions: 1&2 Utah State University Bioinformatics: Problems and Solutions Summer 2006 Review DNA mrna Proteins action! mrna transcript abundance ~ expression

More information

Gene signature selection to predict survival benefits from adjuvant chemotherapy in NSCLC patients

Gene signature selection to predict survival benefits from adjuvant chemotherapy in NSCLC patients 1 Gene signature selection to predict survival benefits from adjuvant chemotherapy in NSCLC patients 1,2 Keyue Ding, Ph.D. Nov. 8, 2014 1 NCIC Clinical Trials Group, Kingston, Ontario, Canada 2 Dept. Public

More information

Towards an Optimized Illumina Microarray Data Analysis Pipeline

Towards an Optimized Illumina Microarray Data Analysis Pipeline Towards an Optimized Illumina Microarray Data Analysis Pipeline Pan Du, Simon Lin Robert H. Lurie Comprehensive Cancer Center, Northwestern University August 06, 2007 Outline Introduction of Illumina Beadarray

More information

Gene Clustering & Classification

Gene Clustering & Classification BINF, Introduction to Computational Biology Gene Clustering & Classification Young-Rae Cho Associate Professor Department of Computer Science Baylor University Overview Introduction to Gene Clustering

More information

MATH3880 Introduction to Statistics and DNA MATH5880 Statistics and DNA Practical Session Monday, 16 November pm BRAGG Cluster

MATH3880 Introduction to Statistics and DNA MATH5880 Statistics and DNA Practical Session Monday, 16 November pm BRAGG Cluster MATH3880 Introduction to Statistics and DNA MATH5880 Statistics and DNA Practical Session Monday, 6 November 2009 3.00 pm BRAGG Cluster This document contains the tasks need to be done and completed by

More information

SVM Classification in -Arrays

SVM Classification in -Arrays SVM Classification in -Arrays SVM classification and validation of cancer tissue samples using microarray expression data Furey et al, 2000 Special Topics in Bioinformatics, SS10 A. Regl, 7055213 What

More information

MICROARRAY IMAGE SEGMENTATION USING CLUSTERING METHODS

MICROARRAY IMAGE SEGMENTATION USING CLUSTERING METHODS Mathematical and Computational Applications, Vol. 5, No. 2, pp. 240-247, 200. Association for Scientific Research MICROARRAY IMAGE SEGMENTATION USING CLUSTERING METHODS Volkan Uslan and Đhsan Ömür Bucak

More information

Drug versus Disease (DrugVsDisease) package

Drug versus Disease (DrugVsDisease) package 1 Introduction Drug versus Disease (DrugVsDisease) package The Drug versus Disease (DrugVsDisease) package provides a pipeline for the comparison of drug and disease gene expression profiles where negatively

More information

Analyzing ICAT Data. Analyzing ICAT Data

Analyzing ICAT Data. Analyzing ICAT Data Analyzing ICAT Data Gary Van Domselaar University of Alberta Analyzing ICAT Data ICAT: Isotope Coded Affinity Tag Introduced in 1999 by Ruedi Aebersold as a method for quantitative analysis of complex

More information

Analysis of (cdna) Microarray Data: Part I. Sources of Bias and Normalisation

Analysis of (cdna) Microarray Data: Part I. Sources of Bias and Normalisation Analysis of (cdna) Microarray Data: Part I. Sources of Bias and Normalisation MICROARRAY ANALYSIS My (Educated?) View 1. Data included in GEXEX a. Whole data stored and securely available b. GP3xCLI on

More information

Expander Online Documentation

Expander Online Documentation Expander Online Documentation Table of Contents Introduction...1 Starting EXPANDER...2 Input Data...4 Preprocessing GE Data...8 Viewing Data Plots...12 Clustering GE Data...14 Biclustering GE Data...17

More information

BIOINFORMATICS. A comparison of normalization methods for high density oligonucleotide array data based on variance and bias

BIOINFORMATICS. A comparison of normalization methods for high density oligonucleotide array data based on variance and bias BIOINFORMATICS Vol. 19 no. 2 2003 Pages 185 193 A comparison of normalization methods for high density oligonucleotide array data based on variance and bias B. M. Bolstad 1,,R.A.Irizarry 2,M.Åstrand 3

More information

Exploratory data analysis for microarrays

Exploratory data analysis for microarrays Exploratory data analysis for microarrays Jörg Rahnenführer Computational Biology and Applied Algorithmics Max Planck Institute for Informatics D-66123 Saarbrücken Germany NGFN - Courses in Practical DNA

More information

How do microarrays work

How do microarrays work Lecture 3 (continued) Alvis Brazma European Bioinformatics Institute How do microarrays work condition mrna cdna hybridise to microarray condition Sample RNA extract labelled acid acid acid nucleic acid

More information

Expander 7.2 Online Documentation

Expander 7.2 Online Documentation Expander 7.2 Online Documentation Introduction... 2 Starting EXPANDER... 2 Input Data... 3 Tabular Data File... 4 CEL Files... 6 Working on similarity data no associated expression data... 9 Working on

More information

Computer Exercise - Microarray Analysis using Bioconductor

Computer Exercise - Microarray Analysis using Bioconductor Computer Exercise - Microarray Analysis using Bioconductor Introduction The SWIRL dataset The SWIRL dataset comes from an experiment using zebrafish to study early development in vertebrates. SWIRL is

More information

Class Discovery and Prediction of Tumor with Microarray Data

Class Discovery and Prediction of Tumor with Microarray Data Minnesota State University, Mankato Cornerstone: A Collection of Scholarly and Creative Works for Minnesota State University, Mankato Theses, Dissertations, and Other Capstone Projects 2011 Class Discovery

More information

Micro-array Image Analysis using Clustering Methods

Micro-array Image Analysis using Clustering Methods Micro-array Image Analysis using Clustering Methods Mrs Rekha A Kulkarni PICT PUNE kulkarni_rekha@hotmail.com Abstract Micro-array imaging is an emerging technology and several experimental procedures

More information

ROTS: Reproducibility Optimized Test Statistic

ROTS: Reproducibility Optimized Test Statistic ROTS: Reproducibility Optimized Test Statistic Fatemeh Seyednasrollah, Tomi Suomi, Laura L. Elo fatsey (at) utu.fi March 3, 2016 Contents 1 Introduction 2 2 Algorithm overview 3 3 Input data 3 4 Preprocessing

More information

Introduction to the xps Package: Comparison to Affymetrix Power Tools (APT)

Introduction to the xps Package: Comparison to Affymetrix Power Tools (APT) Introduction to the xps Package: Comparison to Affymetrix Power Tools (APT) Christian Stratowa April, 2010 Contents 1 Introduction 1 2 Comparison of results for the Human Genome U133 Plus 2.0 Array 2 2.1

More information

Frozen Robust Multi-Array Analysis and the Gene Expression Barcode

Frozen Robust Multi-Array Analysis and the Gene Expression Barcode Frozen Robust Multi-Array Analysis and the Gene Expression Barcode Matthew N. McCall November 9, 2017 Contents 1 Frozen Robust Multiarray Analysis (frma) 2 1.1 From CEL files to expression estimates...................

More information

Review of feature selection techniques in bioinformatics by Yvan Saeys, Iñaki Inza and Pedro Larrañaga.

Review of feature selection techniques in bioinformatics by Yvan Saeys, Iñaki Inza and Pedro Larrañaga. Americo Pereira, Jan Otto Review of feature selection techniques in bioinformatics by Yvan Saeys, Iñaki Inza and Pedro Larrañaga. ABSTRACT In this paper we want to explain what feature selection is and

More information

Using FARMS for summarization Using I/NI-calls for gene filtering. Djork-Arné Clevert. Institute of Bioinformatics, Johannes Kepler University Linz

Using FARMS for summarization Using I/NI-calls for gene filtering. Djork-Arné Clevert. Institute of Bioinformatics, Johannes Kepler University Linz Software Manual Institute of Bioinformatics, Johannes Kepler University Linz Using FARMS for summarization Using I/NI-calls for gene filtering Djork-Arné Clevert Institute of Bioinformatics, Johannes Kepler

More information

Genomics - Problem Set 2 Part 1 due Friday, 1/26/2018 by 9:00am Part 2 due Friday, 2/2/2018 by 9:00am

Genomics - Problem Set 2 Part 1 due Friday, 1/26/2018 by 9:00am Part 2 due Friday, 2/2/2018 by 9:00am Genomics - Part 1 due Friday, 1/26/2018 by 9:00am Part 2 due Friday, 2/2/2018 by 9:00am One major aspect of functional genomics is measuring the transcript abundance of all genes simultaneously. This was

More information

Organizing, cleaning, and normalizing (smoothing) cdna microarray data

Organizing, cleaning, and normalizing (smoothing) cdna microarray data Organizing, cleaning, and normalizing (smoothing) cdna microarray data All product names are given as examples only and they are not endorsed by the USDA or the University of Illinois. INTRODUCTION The

More information

Applying Data-Driven Normalization Strategies for qpcr Data Using Bioconductor

Applying Data-Driven Normalization Strategies for qpcr Data Using Bioconductor Applying Data-Driven Normalization Strategies for qpcr Data Using Bioconductor Jessica Mar April 30, 2018 1 Introduction High-throughput real-time quantitative reverse transcriptase polymerase chain reaction

More information

Nature Publishing Group

Nature Publishing Group Figure S I II III 6 7 8 IV ratio ssdna (S/G) WT hr hr hr 6 7 8 9 V 6 6 7 7 8 8 9 9 VII 6 7 8 9 X VI XI VIII IX ratio ssdna (S/G) rad hr hr hr 6 7 Chromosome Coordinate (kb) 6 6 Nature Publishing Group

More information

PROCEDURE HELP PREPARED BY RYAN MURPHY

PROCEDURE HELP PREPARED BY RYAN MURPHY Module on Microarray Statistics for Biochemistry: Metabolomics & Regulation Part 2: Normalization of Microarray Data By Johanna Hardin and Laura Hoopes Instructions and worksheet to be handed in NAME Lecture/Discussion

More information

Guide to Microarray Analysis

Guide to Microarray Analysis Application Note Microarray Analysis Guide to Microarray Analysis Damian Verdnik, Ph.D. Last updated: January 1, 1 Principles 1 Data Types 1 3 Normalization 3 5 Clustering 7 6 Statistical Analysis 13 7

More information

SEEK User Manual. Introduction

SEEK User Manual. Introduction SEEK User Manual Introduction SEEK is a computational gene co-expression search engine. It utilizes a vast human gene expression compendium to deliver fast, integrative, cross-platform co-expression analyses.

More information

Statistical Analysis of Metabolomics Data. Xiuxia Du Department of Bioinformatics & Genomics University of North Carolina at Charlotte

Statistical Analysis of Metabolomics Data. Xiuxia Du Department of Bioinformatics & Genomics University of North Carolina at Charlotte Statistical Analysis of Metabolomics Data Xiuxia Du Department of Bioinformatics & Genomics University of North Carolina at Charlotte Outline Introduction Data pre-treatment 1. Normalization 2. Centering,

More information

Preprocessing Affymetrix Data. Educational Materials 2005 R. Irizarry and R. Gentleman Modified 21 November, 2009, M. Morgan

Preprocessing Affymetrix Data. Educational Materials 2005 R. Irizarry and R. Gentleman Modified 21 November, 2009, M. Morgan Preprocessing Affymetrix Data Educational Materials 2005 R. Irizarry and R. Gentleman Modified 21 November, 2009, M. Morgan 1 Input, Quality Assessment, and Pre-processing Fast Track Identify CEL files.

More information

How to use the DEGseq Package

How to use the DEGseq Package How to use the DEGseq Package Likun Wang 1,2 and Xi Wang 1. October 30, 2018 1 MOE Key Laboratory of Bioinformatics and Bioinformatics Division, TNLIST /Department of Automation, Tsinghua University. 2

More information

Weka ( )

Weka (  ) Weka ( http://www.cs.waikato.ac.nz/ml/weka/ ) The phases in which classifier s design can be divided are reflected in WEKA s Explorer structure: Data pre-processing (filtering) and representation Supervised

More information

Application of Hierarchical Clustering to Find Expression Modules in Cancer

Application of Hierarchical Clustering to Find Expression Modules in Cancer Application of Hierarchical Clustering to Find Expression Modules in Cancer T. M. Murali August 18, 2008 Innovative Application of Hierarchical Clustering A module map showing conditional activity of expression

More information

Bayesian Analysis of Differential Gene Expression

Bayesian Analysis of Differential Gene Expression Bayesian Analysis of Differential Gene Expression Biostat Journal Club Chuan Zhou chuan.zhou@vanderbilt.edu Department of Biostatistics Vanderbilt University Bayesian Modeling p. 1/1 Lewin et al., 2006

More information

CS6716 Pattern Recognition

CS6716 Pattern Recognition CS6716 Pattern Recognition Prototype Methods Aaron Bobick School of Interactive Computing Administrivia Problem 2b was extended to March 25. Done? PS3 will be out this real soon (tonight) due April 10.

More information

Supervised Clustering of Yeast Gene Expression Data

Supervised Clustering of Yeast Gene Expression Data Supervised Clustering of Yeast Gene Expression Data In the DeRisi paper five expression profile clusters were cited, each containing a small number (7-8) of genes. In the following examples we apply supervised

More information

User s Guide for R Routines to Perform Reference Marker Normalization

User s Guide for R Routines to Perform Reference Marker Normalization User s Guide for R Routines to Perform Reference Marker Normalization Stan Pounds and Charles Mullighan St. Jude Children s Research Hospital Memphis, TN 38135 USA Version Date: January 29, 2008 Purpose

More information

Evaluating Machine-Learning Methods. Goals for the lecture

Evaluating Machine-Learning Methods. Goals for the lecture Evaluating Machine-Learning Methods Mark Craven and David Page Computer Sciences 760 Spring 2018 www.biostat.wisc.edu/~craven/cs760/ Some of the slides in these lectures have been adapted/borrowed from

More information

Genomics - Problem Set 2 Part 1 due Friday, 1/25/2019 by 9:00am Part 2 due Friday, 2/1/2019 by 9:00am

Genomics - Problem Set 2 Part 1 due Friday, 1/25/2019 by 9:00am Part 2 due Friday, 2/1/2019 by 9:00am Genomics - Part 1 due Friday, 1/25/2019 by 9:00am Part 2 due Friday, 2/1/2019 by 9:00am One major aspect of functional genomics is measuring the transcript abundance of all genes simultaneously. This was

More information

Incorporating Known Pathways into Gene Clustering Algorithms for Genetic Expression Data

Incorporating Known Pathways into Gene Clustering Algorithms for Genetic Expression Data Incorporating Known Pathways into Gene Clustering Algorithms for Genetic Expression Data Ryan Atallah, John Ryan, David Aeschlimann December 14, 2013 Abstract In this project, we study the problem of classifying

More information

Feature selection. LING 572 Fei Xia

Feature selection. LING 572 Fei Xia Feature selection LING 572 Fei Xia 1 Creating attribute-value table x 1 x 2 f 1 f 2 f K y Choose features: Define feature templates Instantiate the feature templates Dimensionality reduction: feature selection

More information

FMA901F: Machine Learning Lecture 3: Linear Models for Regression. Cristian Sminchisescu

FMA901F: Machine Learning Lecture 3: Linear Models for Regression. Cristian Sminchisescu FMA901F: Machine Learning Lecture 3: Linear Models for Regression Cristian Sminchisescu Machine Learning: Frequentist vs. Bayesian In the frequentist setting, we seek a fixed parameter (vector), with value(s)

More information

From microarray images to Biological knowledge. Junior Barrera BIOINFO-USP DCC/IME-USP

From microarray images to Biological knowledge. Junior Barrera BIOINFO-USP DCC/IME-USP From microarray images to Biological knowledge Junior Barrera BIOINFO-USP DCC/IME-USP Team Hugo A. Armelin Junior Barrera Helena Brentaini Marcel Brun Y. Chen Edward R. Dougherty Roberto M. Cesar Jr. Daniel

More information

Acquisition Description Exploration Examination Understanding what data is collected. Characterizing properties of data.

Acquisition Description Exploration Examination Understanding what data is collected. Characterizing properties of data. Summary Statistics Acquisition Description Exploration Examination what data is collected Characterizing properties of data. Exploring the data distribution(s). Identifying data quality problems. Selecting

More information

CARMAweb users guide version Johannes Rainer

CARMAweb users guide version Johannes Rainer CARMAweb users guide version 1.0.8 Johannes Rainer July 4, 2006 Contents 1 Introduction 1 2 Preprocessing 5 2.1 Preprocessing of Affymetrix GeneChip data............................. 5 2.2 Preprocessing

More information

Evaluation Measures. Sebastian Pölsterl. April 28, Computer Aided Medical Procedures Technische Universität München

Evaluation Measures. Sebastian Pölsterl. April 28, Computer Aided Medical Procedures Technische Universität München Evaluation Measures Sebastian Pölsterl Computer Aided Medical Procedures Technische Universität München April 28, 2015 Outline 1 Classification 1. Confusion Matrix 2. Receiver operating characteristics

More information

High throughput Data Analysis 2. Cluster Analysis

High throughput Data Analysis 2. Cluster Analysis High throughput Data Analysis 2 Cluster Analysis Overview Why clustering? Hierarchical clustering K means clustering Issues with above two Other methods Quality of clustering results Introduction WHY DO

More information

Microarray data analysis

Microarray data analysis Microarray data analysis Computational Biology IST Technical University of Lisbon Ana Teresa Freitas 016/017 Microarrays Rows represent genes Columns represent samples Many problems may be solved using

More information

COSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor

COSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor COSC160: Detection and Classification Jeremy Bolton, PhD Assistant Teaching Professor Outline I. Problem I. Strategies II. Features for training III. Using spatial information? IV. Reducing dimensionality

More information

Min Wang. April, 2003

Min Wang. April, 2003 Development of a co-regulated gene expression analysis tool (CREAT) By Min Wang April, 2003 Project Documentation Description of CREAT CREAT (coordinated regulatory element analysis tool) are developed

More information

Gene expression & Clustering (Chapter 10)

Gene expression & Clustering (Chapter 10) Gene expression & Clustering (Chapter 10) Determining gene function Sequence comparison tells us if a gene is similar to another gene, e.g., in a new species Dynamic programming Approximate pattern matching

More information

Clustering Techniques

Clustering Techniques Clustering Techniques Bioinformatics: Issues and Algorithms CSE 308-408 Fall 2007 Lecture 16 Lopresti Fall 2007 Lecture 16-1 - Administrative notes Your final project / paper proposal is due on Friday,

More information

Evaluation of different biological data and computational classification methods for use in protein interaction prediction.

Evaluation of different biological data and computational classification methods for use in protein interaction prediction. Evaluation of different biological data and computational classification methods for use in protein interaction prediction. Yanjun Qi, Ziv Bar-Joseph, Judith Klein-Seetharaman Protein 2006 Motivation Correctly

More information

Package SCAN.UPC. October 9, Type Package. Title Single-channel array normalization (SCAN) and University Probability of expression Codes (UPC)

Package SCAN.UPC. October 9, Type Package. Title Single-channel array normalization (SCAN) and University Probability of expression Codes (UPC) Package SCAN.UPC October 9, 2013 Type Package Title Single-channel array normalization (SCAN) and University Probability of expression Codes (UPC) Version 2.0.2 Author Stephen R. Piccolo and W. Evan Johnson

More information

Big Data Methods. Chapter 5: Machine learning. Big Data Methods, Chapter 5, Slide 1

Big Data Methods. Chapter 5: Machine learning. Big Data Methods, Chapter 5, Slide 1 Big Data Methods Chapter 5: Machine learning Big Data Methods, Chapter 5, Slide 1 5.1 Introduction to machine learning What is machine learning? Concerned with the study and development of algorithms that

More information

Title: Optimized multilayer perceptrons for molecular classification and diagnosis using genomic data

Title: Optimized multilayer perceptrons for molecular classification and diagnosis using genomic data Supplementary material for Manuscript BIOINF-2005-1602 Title: Optimized multilayer perceptrons for molecular classification and diagnosis using genomic data Appendix A. Testing K-Nearest Neighbor and Support

More information

Fuzzy C-means with Bi-dimensional Empirical Mode Decomposition for Segmentation of Microarray Image

Fuzzy C-means with Bi-dimensional Empirical Mode Decomposition for Segmentation of Microarray Image www.ijcsi.org 316 Fuzzy C-means with Bi-dimensional Empirical Mode Decomposition for Segmentation of Microarray Image J.Harikiran 1, D.RamaKrishna 2, M.L.Phanendra 3, Dr.P.V.Lakshmi 4, Dr.R.Kiran Kumar

More information

1 Principles 1 2 Data Types 1 3 Normalization 3. 5 Clustering 7 6 Statistical Analysis 13 7 Conclusion 16 4 Dataset Filtering and Management 4

1 Principles 1 2 Data Types 1 3 Normalization 3. 5 Clustering 7 6 Statistical Analysis 13 7 Conclusion 16 4 Dataset Filtering and Management 4 » fi fifi Damian Verdnik, Ph.D. Last updated: January 1, 1 Principles 1 Data Types 1 3 Normalization 3 5 Clustering 7 6 Statistical Analysis 13 7 Conclusion 16 Dataset Filtering and Management 1 Principles

More information

CompClustTk Manual & Tutorial

CompClustTk Manual & Tutorial CompClustTk Manual & Tutorial Brandon King Copyright c California Institute of Technology Version 0.1.10 May 13, 2004 Contents 1 Introduction 1 1.1 Purpose.............................................

More information

Support Vector Machines: Brief Overview" November 2011 CPSC 352

Support Vector Machines: Brief Overview November 2011 CPSC 352 Support Vector Machines: Brief Overview" Outline Microarray Example Support Vector Machines (SVMs) Software: libsvm A Baseball Example with libsvm Classifying Cancer Tissue: The ALL/AML Dataset Golub et

More information

EECS 730 Introduction to Bioinformatics Microarray. Luke Huan Electrical Engineering and Computer Science

EECS 730 Introduction to Bioinformatics Microarray. Luke Huan Electrical Engineering and Computer Science EECS 730 Introduction to Bioinformatics Microarray Luke Huan Electrical Engineering and Computer Science http://people.eecs.ku.edu/~jhuan/ GeneChip 2011/11/29 EECS 730 2 Hybridization to the Chip 2011/11/29

More information

Description of gcrma package

Description of gcrma package Description of gcrma package Zhijin(Jean) Wu, Rafael Irizarry October 30, 2018 Contents 1 Introduction 1 2 What s new in version 2.0.0 3 3 Options for gcrma 3 4 Getting started: the simplest example 4

More information

Identifying differentially expressed genes with siggenes

Identifying differentially expressed genes with siggenes Identifying differentially expressed genes with siggenes Holger Schwender holger.schw@gmx.de Abstract In this vignette, we show how the functions contained in the R package siggenes can be used to perform

More information

Supplementary information: Detection of differentially expressed segments in tiling array data

Supplementary information: Detection of differentially expressed segments in tiling array data Supplementary information: Detection of differentially expressed segments in tiling array data Christian Otto 1,2, Kristin Reiche 3,1,4, Jörg Hackermüller 3,1,4 July 1, 212 1 Bioinformatics Group, Department

More information

Announcements. CS 188: Artificial Intelligence Spring Generative vs. Discriminative. Classification: Feature Vectors. Project 4: due Friday.

Announcements. CS 188: Artificial Intelligence Spring Generative vs. Discriminative. Classification: Feature Vectors. Project 4: due Friday. CS 188: Artificial Intelligence Spring 2011 Lecture 21: Perceptrons 4/13/2010 Announcements Project 4: due Friday. Final Contest: up and running! Project 5 out! Pieter Abbeel UC Berkeley Many slides adapted

More information

Biosphere: the interoperation of web services in microarray cluster analysis

Biosphere: the interoperation of web services in microarray cluster analysis Biosphere: the interoperation of web services in microarray cluster analysis Kei-Hoi Cheung 1,2,*, Remko de Knikker 1, Youjun Guo 1, Guoneng Zhong 1, Janet Hager 3,4, Kevin Y. Yip 5, Albert K.H. Kwan 5,

More information

Things you ll know (or know better to watch out for!) when you leave in December: 1. What you can and cannot infer from graphs.

Things you ll know (or know better to watch out for!) when you leave in December: 1. What you can and cannot infer from graphs. 1 2 Things you ll know (or know better to watch out for!) when you leave in December: 1. What you can and cannot infer from graphs. 2. How to construct (in your head!) and interpret confidence intervals.

More information

Resampling Methods. Levi Waldron, CUNY School of Public Health. July 13, 2016

Resampling Methods. Levi Waldron, CUNY School of Public Health. July 13, 2016 Resampling Methods Levi Waldron, CUNY School of Public Health July 13, 2016 Outline and introduction Objectives: prediction or inference? Cross-validation Bootstrap Permutation Test Monte Carlo Simulation

More information

Exercise 1 Review. --outfiltermismatchnmax : max number of mismatch (Default 10) --outreadsunmapped fastx: output unmapped reads

Exercise 1 Review. --outfiltermismatchnmax : max number of mismatch (Default 10) --outreadsunmapped fastx: output unmapped reads Exercise 1 Review Setting parameters STAR --quantmode GeneCounts --genomedir genomedb -- runthreadn 2 --outfiltermismatchnmax 2 --readfilesin WTa.fastq.gz --readfilescommand zcat --outfilenameprefix WTa

More information

Release Notes. JMP Genomics. Version 4.0

Release Notes. JMP Genomics. Version 4.0 JMP Genomics Version 4.0 Release Notes Creativity involves breaking out of established patterns in order to look at things in a different way. Edward de Bono JMP. A Business Unit of SAS SAS Campus Drive

More information

Noise-based Feature Perturbation as a Selection Method for Microarray Data

Noise-based Feature Perturbation as a Selection Method for Microarray Data Noise-based Feature Perturbation as a Selection Method for Microarray Data Li Chen 1, Dmitry B. Goldgof 1, Lawrence O. Hall 1, and Steven A. Eschrich 2 1 Department of Computer Science and Engineering

More information

Dimension reduction : PCA and Clustering

Dimension reduction : PCA and Clustering Dimension reduction : PCA and Clustering By Hanne Jarmer Slides by Christopher Workman Center for Biological Sequence Analysis DTU The DNA Array Analysis Pipeline Array design Probe design Question Experimental

More information

Stat 342 Exam 3 Fall 2014

Stat 342 Exam 3 Fall 2014 Stat 34 Exam 3 Fall 04 I have neither given nor received unauthorized assistance on this exam. Name Signed Date Name Printed There are questions on the following 6 pages. Do as many of them as you can

More information

Cluster Analysis for Microarray Data

Cluster Analysis for Microarray Data Cluster Analysis for Microarray Data Seventh International Long Oligonucleotide Microarray Workshop Tucson, Arizona January 7-12, 2007 Dan Nettleton IOWA STATE UNIVERSITY 1 Clustering Group objects that

More information

COMPUTATIONAL INTELLIGENCE SEW (INTRODUCTION TO MACHINE LEARNING) SS18. Lecture 6: k-nn Cross-validation Regularization

COMPUTATIONAL INTELLIGENCE SEW (INTRODUCTION TO MACHINE LEARNING) SS18. Lecture 6: k-nn Cross-validation Regularization COMPUTATIONAL INTELLIGENCE SEW (INTRODUCTION TO MACHINE LEARNING) SS18 Lecture 6: k-nn Cross-validation Regularization LEARNING METHODS Lazy vs eager learning Eager learning generalizes training data before

More information

Evaluating Classifiers

Evaluating Classifiers Evaluating Classifiers Reading for this topic: T. Fawcett, An introduction to ROC analysis, Sections 1-4, 7 (linked from class website) Evaluating Classifiers What we want: Classifier that best predicts

More information

CANCER PREDICTION USING PATTERN CLASSIFICATION OF MICROARRAY DATA. By: Sudhir Madhav Rao &Vinod Jayakumar Instructor: Dr.

CANCER PREDICTION USING PATTERN CLASSIFICATION OF MICROARRAY DATA. By: Sudhir Madhav Rao &Vinod Jayakumar Instructor: Dr. CANCER PREDICTION USING PATTERN CLASSIFICATION OF MICROARRAY DATA By: Sudhir Madhav Rao &Vinod Jayakumar Instructor: Dr. Michael Nechyba 1. Abstract The objective of this project is to apply well known

More information

Classification by Nearest Shrunken Centroids and Support Vector Machines

Classification by Nearest Shrunken Centroids and Support Vector Machines Classification by Nearest Shrunken Centroids and Support Vector Machines Florian Markowetz florian.markowetz@molgen.mpg.de Max Planck Institute for Molecular Genetics, Computational Diagnostics Group,

More information

Normalization: Bioconductor s marray package

Normalization: Bioconductor s marray package Normalization: Bioconductor s marray package Yee Hwa Yang 1 and Sandrine Dudoit 2 October 30, 2017 1. Department of edicine, University of California, San Francisco, jean@biostat.berkeley.edu 2. Division

More information

SVM in Analysis of Cross-Sectional Epidemiological Data Dmitriy Fradkin. April 4, 2005 Dmitriy Fradkin, Rutgers University Page 1

SVM in Analysis of Cross-Sectional Epidemiological Data Dmitriy Fradkin. April 4, 2005 Dmitriy Fradkin, Rutgers University Page 1 SVM in Analysis of Cross-Sectional Epidemiological Data Dmitriy Fradkin April 4, 2005 Dmitriy Fradkin, Rutgers University Page 1 Overview The goals of analyzing cross-sectional data Standard methods used

More information

GS Analysis of Microarray Data

GS Analysis of Microarray Data GS01 0163 Analysis of Microarray Data Keith Baggerly and Bradley Broom Department of Bioinformatics and Computational Biology UT MD Anderson Cancer Center kabagg@mdanderson.org bmbroom@mdanderson.org 19

More information

MIT 801. Machine Learning I. [Presented by Anna Bosman] 16 February 2018

MIT 801. Machine Learning I. [Presented by Anna Bosman] 16 February 2018 MIT 801 [Presented by Anna Bosman] 16 February 2018 Machine Learning What is machine learning? Artificial Intelligence? Yes as we know it. What is intelligence? The ability to acquire and apply knowledge

More information

Metrics for Performance Evaluation How to evaluate the performance of a model? Methods for Performance Evaluation How to obtain reliable estimates?

Metrics for Performance Evaluation How to evaluate the performance of a model? Methods for Performance Evaluation How to obtain reliable estimates? Model Evaluation Metrics for Performance Evaluation How to evaluate the performance of a model? Methods for Performance Evaluation How to obtain reliable estimates? Methods for Model Comparison How to

More information

mirnet Tutorial Starting with expression data

mirnet Tutorial Starting with expression data mirnet Tutorial Starting with expression data Computer and Browser Requirements A modern web browser with Java Script enabled Chrome, Safari, Firefox, and Internet Explorer 9+ For best performance and

More information

All About PlexSet Technology Data Analysis in nsolver Software

All About PlexSet Technology Data Analysis in nsolver Software All About PlexSet Technology Data Analysis in nsolver Software PlexSet is a multiplexed gene expression technology which allows pooling of up to 8 samples per ncounter cartridge lane, enabling users to

More information

A Reliable and Distributed LIMS for Efficient Management of the Microarray Experiment Environment

A Reliable and Distributed LIMS for Efficient Management of the Microarray Experiment Environment A Reliable and Distributed LIMS for Efficient Management of the Microarray Experiment Environment Hee-Jeong Jin BK Center for U-Port IT Research Education, Pusan National University, Busan, South Korea,

More information

CS 229 Final Project Report Learning to Decode Cognitive States of Rat using Functional Magnetic Resonance Imaging Time Series

CS 229 Final Project Report Learning to Decode Cognitive States of Rat using Functional Magnetic Resonance Imaging Time Series CS 229 Final Project Report Learning to Decode Cognitive States of Rat using Functional Magnetic Resonance Imaging Time Series Jingyuan Chen //Department of Electrical Engineering, cjy2010@stanford.edu//

More information

Gene Expression Clustering with Functional Mixture Models

Gene Expression Clustering with Functional Mixture Models Gene Expression Clustering with Functional Mixture Models Darya Chudova, Department of Computer Science University of California, Irvine Irvine CA 92697-3425 dchudova@ics.uci.edu Eric Mjolsness Department

More information

Image Processing. Bilkent University. CS554 Computer Vision Pinar Duygulu

Image Processing. Bilkent University. CS554 Computer Vision Pinar Duygulu Image Processing CS 554 Computer Vision Pinar Duygulu Bilkent University Today Image Formation Point and Blob Processing Binary Image Processing Readings: Gonzalez & Woods, Ch. 3 Slides are adapted from

More information

Evaluation. Evaluate what? For really large amounts of data... A: Use a validation set.

Evaluation. Evaluate what? For really large amounts of data... A: Use a validation set. Evaluate what? Evaluation Charles Sutton Data Mining and Exploration Spring 2012 Do you want to evaluate a classifier or a learning algorithm? Do you want to predict accuracy or predict which one is better?

More information

Exon Probeset Annotations and Transcript Cluster Groupings

Exon Probeset Annotations and Transcript Cluster Groupings Exon Probeset Annotations and Transcript Cluster Groupings I. Introduction This whitepaper covers the procedure used to group and annotate probesets. Appropriate grouping of probesets into transcript clusters

More information

TAIR User guide. TAIR User Guide Version 1.0 1

TAIR User guide. TAIR User Guide Version 1.0 1 TAIR User guide TAIR User Guide Version 1.0 1 Getting Started... 3 Browser compatibility and configuration.... 3 Additional Resources... 3 Finding help documents for TAIR tools... 3 Requesting Help....

More information

Cross-validation and the Bootstrap

Cross-validation and the Bootstrap Cross-validation and the Bootstrap In the section we discuss two resampling methods: cross-validation and the bootstrap. These methods refit a model of interest to samples formed from the training set,

More information