Appendix I. TACTICS Toolbox v3.x. Interactive MATLAB Platform For Bioimaging informatics. User Guide TRACKING MODULE

Transcription

1 TACTICS Toolbox v3.x Interactive MATLAB Platform For Bioimaging informatics User Guide TRACKING MODULE

2 -17- Cell Tracking Module 1 (user interface) Once the images were successfully segmented, the next step in the processing scheme, is the cell tracking and manual correction interface: From the main menu, go to >Cell Tracking Module The Cell Tracking Module is used to correct segmentation, label cells and tracking. The GUI has several selection methods for manual image segmentation that allow the user to inspect and correct cell segmentation manually, and allows for multiple objects tracking correction. Video tutorial 3 Screen capture of the Cell Tracking Module interface Secondary Screen Editing tracks panel Trajectory panel Primary Screen Segmentation panel Navigation panel Functions supported in the icon bar- Load experimental data file Save experimental data file Drag zoom rectangle Capture ROI to secondary screen Zoom in Zoom 100% Zoom out Pan Error Zoom (temporary solution for Zoom issues) Open new MATLAB figure 1. Run the Linker 2. Split cells after divisions 3. Save manual correction of cell associations 4. Show lineage tree 5. Mark cell division 6. Mark cell death Mark cell division Mark cell death

3 -18- Cell Tracking Module 1 (user interface) Screen capture of the Cell Tracking Module interface Channel and display mode of the secondary screen Change the channel and the display mode (raw, filtered etc.) of the Primary screen using this pull down menu Read and upload images within selected range to RAM (dependent on the computer memory) Keyboard shortcuts: Uparrow - Next frame Downarrow - Previous frame Space - label frame again Enter - associate frame overpassing selective operator F - Format painting S - Segmentation mode M - Mark division mode E - Next frame T - Trajectory mode A - Association mode D - Dead cell mode E - Next frame R - Remove segment mode U - uncheck/check P - Paint Tool mode

4 -19- Cell Tracking Module 2 (cell labeling and tracking) Instructions for cell labeling and tracking: 1.Load an experiment data file to TACTICS. 2.To label the cells- Go to > Label cells >> Label cells. A labelling function will label each segment from n to m (n and m are user input). Reference channel can be used to help to interpret normalized pixel intensities in the quantified channel and allow to track two types of cells at the same time. After running the Label Cells function, the cells will be detected, identified and labeled including feature extraction. A bounding box will mark each segment, and all labeled frames will be marked in red tab under the sliderbar: 3. To connect between the cells over time- Go to > Label cells >> Label association. An input box will ask for the range of frames to be associated and maximum distance that the cells can travel (a new cell will be defined if the distance to a closer point in time (t-1) is above this criteria. The input units are currently in pixels. Once finished, all associated frames will be marked in green tab under the sliderbar (note that the first frame cannot be associated): 4.To track the cells, click on the tracking icon or Go to >Track cells>> Hungarian algorithm>>>tracking linker. The associated points will be connected to create the cell`s tracks. The user must input a cut off value that sets the possible minimum track length (a value of 1 is frequently acceptable) As an alternative to label association followed by Hungarian algorithm, the user can utilize the Crocker algorithm for tracking multiple objects Go to >Track cells>> Crocker algorithm. The advantage is the ability to recover for missing points. However, the algorithm is slower and fails when the number of cells or time points is large (dozens of cells for 1000 frames will fail the algorithm) Once cells are tracked, trajectories of cell over time can be visualized. The location of the cells in different time points is marked with a colored point, and the size of the point is proportional to the timeframe of the track. To mark dividing cells, go to cell division mode and click on the cell at the point of division (one time point before the appearance of two separated cells) with the middle mouse button. X sign will be added/removed correspondingly. To mark dying cell, go to cell death mode and click on the cell at the last point of appearance with the middle mouse button. + sign will be added/removed correspondingly. IMPORTANT. Save the experiment file (overwrite or create new file) before exiting TACTICS (or before loading new experiment file). Otherwise, any operation will be lost.

5 -20- Cell Tracking Module 3 (cell labeling corrections) A. Improving segmentation of a single frame using the secondary screen view 1. Click on the Capture ROI to secondary screen icon. Once you have clicked on the icon a window will come up with an image of the screen you have been working on. Hold down mouse (left click) anywhere on the new screen. This will create a second figure from the right side of the screen containing a magnified image of the ROI within the images that is chosen for zoom. 2. While still holding down left click use the arrow keys to form a box around the cells that you would like to work with in the secondary screen (Make sure the aspect ratio is 1:1 or you will not be able to see the cells clearly or correct them with good accuracy). Once you are happy with the size and placement of your box, double left click to confirm. The selected region will be magnified to the secondary screen view and the borders of this region will be appeared in the primary screen view. 3. When using the mouse-wheel up and down, the ROI will be shown in the primary screen view. User can use the Drag zoom rectangle icon to move the ROI keeping the current dimensions. 4. Go to Segmentation mode, and select Segment current frame and save to HD option in the popup menu. 5. To load the segmentation settings go to the File menu > import > (import segmentation settings OR segmentation settings for current frame). The user can then adjust those settings to improve segmentation of the current frame by altering values on the bottom right frame.. 6. The user can remove individual pixels by clicking with middle mouse button (or mouse-wheel) while in the segmentation mode. Screen capture of the Cell Tracking Module interface Secondary Screen showing the zoomed-in ROI Box containing the ROI Primary screen view Secondary screen view

6 -21- Cell Tracking Module 3 (cell labeling corrections) Separating merged cells. TACTICS provides multiple options for manually separating merged cells, including: 1. Splitting a single object using watershed or intensity shell algorithms. 2. Splitting a single object by drawing a line through it or deleting individual pixels. 3. Improving segmentation of a single frame (as per previous page). 1. Splitting objects using watershed or intensity shell algorithms. If a labeled object is clearly two or more adjoined cells, move the cursor over the box surrounding the object, and left mouse click (watershed) or right mouse click (intensity shell) in the primary screen view. If the algorithm succeeds, the bounding box should split into two boxes. If the algorithm fails, the alternative algorithm can be attempted. After separating all the adjoined cells in a frame scroll forward or back through the movie to save the changes. 2. Splitting objects by drawing a line or deleting pixels. a. To separate between cells by drawing a line of deleting pixels use the Secondary Screen (See the previous page for instructions on how to access the secondary screen using the capture ROI icon ). b. Draw a line where cells need to be divided. To do this, left click at the first point of where your line should be and a yellow dot will appear. Left click again at the same point and drag the blue line to divide your cells. An intensity line will go between the cells to give two labelled cells. c. Double left click on the blue line to accept changes. d. Click middle button (wheel) or CTRL+A to save changes. e. The properties of each segment that are saved in the.dat experiment file (described in Appendix 1), and bounding box will appear for each segment (in the Primary Screen). In the secondary screen, outlining of the cells by different colors indicates successful segmentation. Initial problem cell First point identified Blue line drawn Final product (two defined cells) f. Run tracking algorithm again. This is required to incorporate the newly segmented cells, and can be performed after resegmenting one or several frames. I M P O R T A N T: Once this data file is created and loaded into Cell Tracking Module, any modification in the data file will be logged in the loaded data file. Therefore it is necessary to save the data file once processing is applied.

7 -22- Cell Tracking Module 3 (cell labeling corrections) 3. Splitting objects by altering segmentation parameters. Resegmenting the entire frame can also help with splitting two cells, although care must be taken that further segmentation problems do not occur in other areas of the frame. Below is an example where two joined cells were separated by increasing the threshold value as per instructions in A (p 18). Changed threshold value Optional. Changing the LUT is a standard procedure that helps to distinguish between pixel intensities. An example shown below shows that pixel intensities in the cell perimeter have low value, causing over-segmentation by Otsu s method.

8 -23- Cell Tracking Module 3 (cell labeling corrections) Removal of segmented pixels that are falsely identified as cells: Sometimes a new object appears during segmentation correction that is incorrectly labeled as a cell. An example of this is shown below: Cell Tracking Module incorrectly recognizes this as a cell Instructions for removal: 1. Zoom-in on the problem area in the secondary screen until each pixel is clearly visible. 2. Right click on one of the pixels to remove the colored dots indicating the segment edge. 3. You can now remove each unwanted pixel by right clicking on it. To replace a pixel that you have deleted right click in the same place. Similarly, right clicking on a black pixel converts it from 0 to1, and so can be used to link separated cells. Shot of secondary screen with pixels clearly visible After one right click After pixels had been removed.

9 -24- Cell Tracking Module 4 (cell tracking corrections) The tracking accuracy depends on the quality of the raw data, the imaging system, the biological system, and the computational approach to analyze the data. Common problems with tracking are when a cell suddenly moves a large distance. Tracks can be changed manually using several approaches, that either involve moving track vectors on the primary screen, or use a pull down menu in which cells can be connected by numbers. To correct cell association tracks by shifting connecting vectors: 1.Choose (n-1) in the secondary axes. 2.Click association button. 3.Drag the lines between the middle f the cell bounding-box (n-1) to its origin in n. 4.Click on the green icon in the icon panel to accept changes. Cell without match between n and (n-1) marked by blue centroid. Cells with match are marked in red and id number. Watch the video tutorial for more information. Instructions for manual corrections for tracking: Video tutorial 4 1.Go to edit tracks mode. 2.Point with the mouse curser on required cell and click on the middle button. A selection window will give several different options. IMPORTANT: the manual corrections are applied on the matrix of trajectories. Therefore, when applying the linker again, the manual corrections are deleted. For this reason, it is always better to apply manual corrections for cell association.

10 -25- Cell Tracking Module 5 (marking cell division and death) Marking cell division Go to the (Mark cell division) mode Go to the last frame before division, where only one cells in visible In the primary screen, wheel click on the dividing cell Click on the (tracking splitter) icon Parental cells will have name ##-**, whereas ## is the name chosen by the user and ** is a number by order of appearance. Daughter cells of parental cells will have name ##-**.1 and ##-**.2, whereas ##-** is the name of the parental, and 1 and 2 symbolise daughter 1 and 2. For each additional cell division a dot and number 1 or 2 will be added. For instance: ##-** Screen capture of the Cell Tracking Module interface 4-click tracking splitter 1- Cell division mode Frame n Frame n+1 2- cell about to divide 3- wheel click on cell, Adding red x annotate the cell as dividing cell Frame n Frame n+1 5- (next frame) two cells identified as two related daughter cells and the information is saved in the experimental file Frame n : Final frame before division

11 -26- Cell Tracking Module 6 (drawing tool) The painting tool allows manually to draw, fill, and remove pixels on any selected channels. This can be very useful to correct segmentation when automated segmentation fails. To apply the drawing tool: 1.Go to drawing tool mode. 2. Select show segmentation to on. 3.Select channel to apply segmentation, channel for visualization, and channel for tracking. Zooming-in is recommended, but not a recruitment. The blue net will appear for each existing pixel in the binary matrix. 4. Select the size of painting point. 5. Press mouse left click. Hold it and move the mouse curser to fill new pixels. 6. To remove pixels- Press mouse right click. Hold it and move the mouse curser. 7. To accept: click mouse wheel button. This will save the modified image as segmented one (overwrite) and will label again (relabeling should be saved to experimental file otherwise this will be lost). 8. To cancel (before acceptance): just use mouse wheel to refresh frame (go to the next or previous frames) 9. To cancel (after acceptance): currently TACTICS doesn't have this feature (is that useful?). Therefore need to segment the frame again. 4- select point size 1- drawing tool mode 3- select channels 2- click show segment on

12 -27- Cell Tracking Module 7 (Parameter selection) The parameter selection window ddisplay parameters such as intensity, and velocity for a particular selected cell, which allow to improve the cell tracking. To activate the parameter selection window : 1. Select cell. 2. Choose- plot data for selected cell option. 3. New Manu will popup. Choose parameter to be plotted. (more parameters can be easily added). 4. When using the mouse scroll wheel, the plot will be shown in the secondary figure view for the relevant time points. 3- select parameters in popup menu 4- parameter is plotted 2- select mode 1- select cell Display number of objects in the current frame

13 -28- Cell Tracking Module 8 (corrections of lineage trees) Complicated switch in tracking can accrues when cells drift and tracking is incorrect for the following tracks. This can strongly influence the lineage tree and its analysis. To correct tracks for lineage trees: 1. From main Go to > Tools >> Generate Lineage tree. A popup window shows the lineage for any selected cell. In this particular example it can be seen that a problem accrues in the tracking. Click on the lineage window interactively goes to the frame of selection in the Cell Tracking Module. 2. In some cases other supporting tools can help to allocate events suspected as wrong. For instance the cell tracks window. From main Go to > Tools >> open cell tracks window. Other tools can be useful Go to > Data inspection, such as show number of objects over time, show trajectories matrix and search for undefined association. 3. Adjust the current settings showing for frame n and (n-1). 4. To visualize the association between cells at particular frame click on Centy button. In this particular example, because (n-x) is set to -1, the cells in DIC in frame 382 but the tracks and bounding boxes are actually shown for frame 381. This provides the locations of the cells in the previous frame. The Centy button shows that a new cell was defined (blue), and that the second daughter cell was wrongly labelled as cell Generate Lineage tree 2- (optional) open cell tracks window or use other inspection tools 3.a.- input frame to show to be -1 3.b.- input channels to visualize frame n-1 3.c.- input channels to track and visualize frame n 4- Click on centy

14 -29- Cell Tracking Module 8 (corrections of lineage trees) cell association tracks by shifting connecting vectors 5. The user can move the pink lines, so the pink star * touch the destination location in frame 383. In this case a line was stretched from cell number 7 to , while the line that start. In this example, from the cell between and was pointed back on itself. This mean that this cell doesn't Have a much and will therefore recognized as a new cell. To accept click the cell association correction button. 6. After the correction, run the linker again. The lineage tree is now corrected. If the user regrets or want to repeat the process, labelling the frame again goes to initial association. 5- drag lines to link between cells 6- lineage tree is corrected Two-dimensional graphic visualization of lineage tree. Bifurcations in the tree represent cell divisions. X-axis represents the cell index. Y-axis represents the frame (or time) that each cell appear.

15 -30- Cell Tracking Module 9 (selective operator) Selective operator is method that allows the user to select what frames will be associate or skipped. The user can select by frequency to be skipped (for instance every 3 rd frame) or/and certain frames in the movie. The associated frames appear in green bars under the movie sliderbar. Only the frames between following green bar coded frames are associated. Selective operator Modes Show frames Show tracks Show tracks annotations Non-Selective All frames Only in selected frames Only in selected frames Selective All frames Only in selected frames Only in selected frames Only Selective Skip to selected frames Skip to selected frames Skip to selected frames Partly Selective All frames All frames Only in selected frames Trajectories matrix shows association between every 10 frames Green bars shows the associated frames Number of cells in frame in selective mode Selective operator