TUTORIAL. HCS- Tools + Scripting Integrations

Transcription

1 TUTORIAL HCS- Tools + Scripting Integrations

2 HCS- Tools... 3 Setup... 3 Task and Data ) Data Input Opera Reader ) Meta data integration Expand barcode ) Meta data integration Join Layout ) Visualization Plate Viewer ) Normalization Normalize Plates (POC) ) Quality Control SSMD (PC x NC) ) Reshape table Pivoting ) Filter dataset Row Splitter ) Reshape table Pivoting ) Utilities Range Splitter ) Normalization Normalize Plates (Z- score) ) Filtering - Reference Row Filter ) Filtering Row Filter ) Filtering Row Filter ) Utilities Range Filter (R) Scripting Integration Setup Task ) Use templates R Snippet ) Use templates R Plot ) How to use flow variables ) Writing templates... 39

3 HCS- Tools Setup Update- Site Location (nightly build) contributions/nightly Preferences (HCS- Tools) Minimal number of samples required to calculate the mean or median Minimal number of samples required to calculate variance or MAD... at least [x] samples per group should be present to provide a descent estimate of the statistic important for normalization and QC nodes less samples will result in a warning Scaling factor for MAD statistic set to the factor proposed for normal distributed data Barcode patterns regular expressions (separated by ; ) which describes a barcode standard used for Expand Barcode node to automatically retrieve meta data from the barcode, and Plate Viewer node possible placeholders: projectcode, libcode, libnumber, date, replicate, assay, description, concentration, concunit, timepoint, customa, customb, customc, custom

4 Example regex for barcode: (?<libplatenumber>[0-9]{3}) (?<projectcode>[a- z]{2}) (?<date>[0-9]{6}) (?<replicate>[a- z]{1})- (?<libcode>[_a- z\d]{3}) (?<assay>[- _\s\w\d]*) Preferences (Knime Helpers) Data example resources Semicolon separates list of text file URLs which contain references to a set of.table files (exported tables from KNIME) Task and Data RNAi screen has been performed with a stable cell line. Cells were fixed and stained Nuclei and cytoplasm staining Marker 1 and Marker 2 39 x 384- well plates Results from an image analysis (Acapella) of images taken by the Opera - an automated confocal microscope from PerkinElmer Each RES- file (XML- format) contains the results of one 384 well plate Measurements Number of cells in the well Several quality control measurements (intensities of different channels) The Layout is given as an Excel- sheet

5 An Excel- sheet of a certain format provides information on the treatment Positive Transfection Controls (Tox1, Tox, Tox3) Negative Controls (Mock, Untreated) RNAi library Big goal: Extract RNAi s which are not toxic but show a significant increase of the signals in both marker channels Step by step Data Input: Load the raw data Metadata integration: Add meta data from both barcode and layout Visualization: o Inspect data visually o Strength of cell number reduction for transfection controls seems to be different, has to be quantified o Readouts show a plate to plate variation Quality control: o Did the transfection work well? o well = at least 80% transfection efficiency Normalization: o Plate wise normalization / Percent of control (POC) o plate data has to be normalized on the Mock wells o two effects: o Mock wells will be centered around 100% (eliminates plate wise variation) o Transfection controls are represented as percentage which makes it easy to judge about transfection efficiency Quality Control: o SSMD o Measure of how well positive control and negative control are separated from each other. It s better interpretable than z prime factor Filtering: o Remove all screening plate where none of the transfection controls shows an efficiency < 80 % o keep, if at least on transfection control has less then 20% cells Normalization for hit selection and filter criteria: o consider only wells with a cell number comparable to Mock o ( comparable = standard deviation away from the median) o consider wells which show more than 2 standard deviation increase of both marker channel signals o Z- score normalization of the whole screen based on Mock o centralize Mock values around 0 with standard deviation 1

6

7 1) Data Input Opera Reader Where to find this node? HCS Tools IO How to configure the node? Click on the Button Select Files or Directory and select the folder containing the Opera result files Result files are located in you workflow folder [Path to your workspace]/hcs- Tools- Tutorial/data/OperaExampleData/ Note: If a folder is selected, subfolders will be searched recursively for Opera result files. Otherwise, single or multiple files within a folder can be selected. What does the output table contain? Table contains well based image analysis results of a set of screening plates and the corresponding metadata (barcode, well position, acquisition time)

8 2) Meta data integration Expand barcode Where to find this node? HCS Tools Utilities How to configure the node? Select the column which contains the plate barcode How does it work? The barcode string of each row will be matched against all patterns given by the regular expressions in the preferences of HCS- Tools Each placeholder (see preferences) of the matching pattern then will result in a column

9 What does the output table contain? The table contains as much additional columns as there are placeholders from the corresponding barcode pattern. 3) Meta data integration Join Layout Where to find this node? HCS Tools Utilities How to configure the node? Click on the Button Browse and select the Excel file containing the plate layout

10 [Path to your workspace]/hcs- Tools- Tutorial/data/Join_Layout_for_KNIME_HCStools_Example.xls Note: The node can read XLS files as well as XLSX files. It s important that the name of the Excel sheet containing the layout information is set correctly in the text field Sheet name How does it work? The Excel sheet has to follow some rules: The layout has to be in a plate format (matrix of plate rows and plate columns) There can be as much layouts as needed but layout names have to be unique and their format has to be the same Row 1-4 can contain comments (ignored by the node) The first layout starts at cell C5 with the layout name Row 6 has to contain the plate column header (1,2,3,4, ) Column C has to contain the plate row header (A,B,C, ) The node reads all plate formats After the every layout two empty rows have to be inserted before the next layout starts (layout name, row and column header, content) Formatting (colors, fonts, borders, ) supports readability of the Excel sheet and will be ignored by the node

11 What does the output table contain? The table contains as much additional columns as there are layouts in the Excel sheet.

12 4) Visualization Plate Viewer Where to find this node? HCS Tools Data Views How to configure the node? There are three tabs to configure! Readouts tab: Select ( Include ) the columns containing the numerical measurements

13 Factors tab: Select ( Include ) the columns containing meta data (assay,, concentration, left_right, top_bottom, inner_outer) Plate Defintion tab: It s already configured correctly if the table contains columns named barcode, platerow and platecolumn What does the node view ScreenExplorer show? The view shows plate heatmaps of the selected readout Note: Color scale is applied globally not plate wise

14 Play around: Select different readouts Select different overlays, they show color coded layout information. The color legend is available in Options > Show overlay legend Filter plates by a search string (e.g. CON or 003 ) Select transfection as overlay and then activate Options > Hide most frequent overlay A double click on one plate opens a single plate view Play around: Enlarge the window until you can see the labels of the overlay (overlay has to be selected before) Move your mouse cursor over the wells the see a little window with all readout values for the current well

15 Double click on wells opens these windows permanently What does the output table contain? Unmodified input table 5) Normalization Normalize Plates (POC) Where to find this node? HCS Tools Normalization How to configure the node? Group wells by barcode Select ( Include ) some readouts which should be normalized Select the column transfection as treatment attribute Select Mock as the reference treatment you would like to normalize on

16 How does it work? The input table is split into subsets of data according to the grouping column. In this example each subset will contain the data of one plate. For each subset all data points (rows) are selected which are labeled with Mock in the transfection column (the Mock wells of the plate). For each selected readout the median of these wells is calculated. Then each data point of the plate is POC normalized with these median values. The result will be a percentage. What does the output table contain? For each selected readout will be an additional column with the suffix.poc 6.1) Quality Control SSMD (PC x NC) Where to find this node? HCS Tools Quality Control How to configure the node? Select the column transfection as treatment attribute Select Tox1, Tox2 and Tox3 as positive controls Select Untreated and Mock as negative controls Select ( Include ) Number of Cells as readout

17 How does it work? The input table is split into subsets of data according to the grouping column. In this example each subset will contain the data of one plate. Within each plate the SSMD values for each readout and on all combinations of positive and negative controls are calculated. In our example the SSMD value of the Number of cells will be calculated for the following combinations: Positive Control Tox1 Tox1 Tox2 Tox2 Tox3 Tox3 Negative Control Mock Untreated Mock Untreated Mock Untreated

18 What does the output table contain? The table contains the three columns barcode (grouping column), Positive Control and Negative Control as a unique combination for the SSMD value of the Number of Cells (last column) 6.2) Reshape table Pivoting Purpose Each plate should show all SSMD- values in a row (easier to read) How to configure the node? Groups tab: Select ( Include ) column barcode

19 Pivots tab: Select ( Include ) columns Positive Control and Negative Control Options tab: Select column Number of Cells and use Mean as aggregation method Keep original column names

20 7.1) Filter dataset Row Splitter Purpose Extract all transfection controls (labeled as Tox ) How to configure the node? Select transfection as column to test Use pattern matching (check contains wild cards!) Pattern: Tox*

21 7.2) Reshape table Pivoting Purpose The result table should contain for each plate one row which shows the mean of POC normalized values of the cell count for each transfection control How to configure the node? Groups tab: Select ( Include ) column barcode Pivots tab: Select ( Include ) column transfection Options tab: Select column Number of Cells.poc and use Mean as aggregation method

22 What does the output table contain? 7.3) Utilities Range Splitter Where to find this node? HCS Tools Data Manipulation Row Filter How to configure the node? Lower bound is set to 0 and upper bound is set to 20 Rows match if at least one value is in range Select ( Include ) all numeric columns available

23 How does it work? For each row the values of all chosen column are checked whether they fall in the given range. If this is true for at least one colum, the row is kept, otherwise it is discarded What does the output table contain? The output consists of two tables The table keep now contains all plates (43) where at least one transfection control showed a cell number decrease (maximal 20% of the negative control) The table discard contains the remaining plates (3)

24 8) Normalization Normalize Plates (Z- score) Where to find this node? HCS- Tools Normalization How to configure the node? Group wells by project code Select ( Include ) the POC- normalized readouts Select the column transfection as treatment attribute Select Mock as the reference treatment you would like to normalize on How does it work? The input table is split into subsets of data according to the grouping column. In this example each subset will contain the data of all plates (so it s not a subset at all but the whole dataset).

25 For each subset all data points (rows) are selected which are labeled with Mock in the transfection column (the Mock wells of all plates). For each selected readout the median of these wells is calculated. Then each data point is POC normalized with these median values. The result will be a factor, which expresses how many standard deviations the data point is away from the reference. What does the output table contain? For each selected readout will be an additional column with the suffix.zscore 9.1) Filtering - Reference Row Filter Purpose Exclude all plates with bad transfection efficiency How to configure the node? Select the column barcode for both data table and reference table

26 9.2) Filtering Row Filter Purpose Extract all library wells How to configure the node? Select transfection as column to test Use pattern matching Pattern: library

27 9.3) Filtering Row Filter Purpose Extract all wells whose cell numbers are comparable to Mock wells ( standard deviation away) to eliminate toxic and proliferating treatments How to configure the node? Select Number of Cells.poc.zscore as column to test Use range checking with lower bound and upper bound 0.5

28 10) Utilities Range Filter Where to find this node? HCS Tools Data Manipulation Row Filter How to configure the node? Lower bound is set to 2.0 and upper bound is set to Infinity Rows match if values are in range for all columns Select ( Include ) the columns containing the z- score normalized values for both marker channel signals

29 How does it work? For each row the values of all both columns are checked whether they fall in the given range. If this is true for both columns, the row is kept, otherwise it is discarded What does the output table contain? The wells, which are considered as our hits! Congratulation! You are done!

30 (R) Scripting Integration Setup Update- Site Location (nightly build) contributions/nightly Preferences (R scripting) The host where Rserve is running Host name or IP address The port on which Rserve is listening Port number (standard 6311) Repaint on resize If the view window is resized the plot can be created again to avoid an interpolation of the picture Snippet template resource List of snippet template URLs (can also be local files), which can be enabled or disabled Plot template resource List of plot template URLs (can also be local files), which can be enabled or disabled Location of the local R installation Used by OpenInR

31 Task

32 1) Use templates R Snippet Where to find this node? R Scripting R Snippet How to configure the node? Select the Templates tab In the template tree view browse to the snippet Shapiro- Wilk Normality Test (QQ- Plot)

33 Click Use this template and the tab will automatically change to Script Editor with the template view. Now the template itself can be configured. Select the columns containing the POC- normalized readouts.

34 How it works? The template contains an R- script that is initialized with the configuration of the user. Both the script and the input table are sent to the R- server. There, the script is executed and the result data frame is sent back and converted into an output KNIME table. What does the output table contain? Whatever the R snippet returns. In this example, it contains the result of the Shapiro- Wilk test for every parameter.

35 2) Use templates R Plot Where to find this node? R Scripting R Snippet How to configure the node? Select the Templates tab In the template tree view browse to the template QQ- Plot Grid Click Use this template and the tab will automatically change to Script Editor with the template view. Now the template itself can be configured. Select the columns containing the POC- normalized readouts.

36 How it works? The data transfer follows the same concept as for the snippet node. Instead of a table, the server returns a picture of the given dimensions and type (JPEG or PNG) defined in the Output Options tab. What does the node view show? It displays the returned picture of the R plot. What does the output port contain? It s a KNIME image port containing the R plot

37 3) How to use flow variables 3.1 Column List Loop Start Select all POC- normalized readouts The flow variable is named currentcolumnname 3.2 R Plot Use the script editor tab to enter the following R script var <- "FLOWVAR(currentColumnName)" plot(density(kin[,var]), main = var) Use the Output Options tab Set image dimensions to 500 x 500 Select a path and filename containing the current flow variable value Check Overwrite existing file How it works? The placeholder FLOWVAR(currentColumnName) is replaced by the actual value of this flow variable before the script is sent to the server The same replacement takes place for the output file name.

38 Image to table The node puts the R plot into a table cell Variable to table column Select the currentcolumnname to add the readout name to the image in the table Loop end It s time to run the loop and don t forget to look at the output folder which should contain 5 different PNG files after execution.

39 4) Writing templates Main architecture RGG- XML contains the description of the user interface as well as the script itself <rgg> some interface elements definitions <![CDATA[ R- script ]]> </rgg> The syntax of several elements can be found at forge.r- project.org/ Additionally the scripting integrations contains the column selection element <panellistbox label="label" items="item1,item2,item3" span="full"/> And the scripting integrations provide some placeholders $$$NUM_ATTRIBUTES$$$ - for all numeric columns $$$STR_ATTRIBUTES$$$ - for all string columns $$$ALL_ATTRIBUTES$$$ - for all columns The placeholders are replaced by a comma separated list of the column names Example: <panellistbox label="parameters of interest" items="$$$num_attributes$$$" span="full"/> would create this interface: Note: the label of each element has t be unique! Example template Take an R Plot node Select any template; say Use this template Open the template editor window by clicking Edit template

40 <rgg> Delete everything and add the following template code <!--1. Title and short description --> <h3 text="normal distributed data" aligment="center" span="full"/> <separator label="description" span="full"/> <labelarea span="full">snippet creates normal distributed data with mean and standard deviation taken from the estimates of a numeric column of the input table </labelarea> <gaprow height="1"/> <!-- 2. Configuration--> <separator label="options" span="full"/> <gaprow height="2"/> #1. Parameter selection <group> <combobox var = "selectedcolumn" label = "Select a column" items = "$$$NUM_ATTRIBUTES$$$"/> n = as.numeric(c(<textfield label="number of datapoints" datatype="number" default-value = "100" size="5"/>)) </group> <![CDATA[ meanvalue <- mean(kin[,selectedcolumn]) sdvalue <- sd(kin[,selectedcolumn]) rout <- data.frame(data = rnorm(n, meanvalue, sdvalue)) hist(rout$data) ]]> </rgg> Congratulation! You are done!