User Guide for beginners: Drishti Prayog

Transcription

1 User Guide for beginners: Drishti Prayog Content 1. Introduction to Drishti and Drishti Prayog Drishti Downloading Drishti Loading a volume into Drishti Starting with Sliced Images Starting with 'pvl.nc' files Starting with '.xml' files Choosing transfer functions to see different components of the volume Interpreting the 2D histogram panel Interpreting the transfer functions Tips for choosing good transfer functions Interpreting the Transfer Function Editor panel Importing transfer functions from existing projects Changing colour and opacity Setting Key-frames and Creating an Animation Interpreting key-frames Setting Key-frames Generating an animation Adding a picture/video/text to the current project Save Files for Drishti Prayog Save project Save information for Drishti Prayog Drishti Prayog Installing Prayog Setting up Project Path Creating configuration files for Prayog projects Playing a video Useful tools in Prayog Common problems with solutions Additional Links

2 1. Introduction to Drishti and Drishti Prayog Drishti is software developed by Ajay Limaye at NCI VizLab that allows users to input reconstructed datasets and output the files that can be displayed on Drishti Prayog. Drishti allows users to render a set of slice images into a 3D volume, and to do a range of operations to the rendered volume. Drishti includes two tools for working with 3D volumes. The first tool is Drishti-import, this tool enables users to load volume datasets from selected formats such as tiff, raw etc; into the tool and convert them to a format that Drishti can load efficiently. The second tool is Drishti-paint, this tool enables segmenting a 3D volume and exporting the segmented volume into Drishti. Drishti Prayog is a user-friendly presentation tool designed for users to visualise and explore the 3D datasets on a touch screen. This guide shows only the process of how to create a display on Drishti Prayog, given a set of sliced images. 2

3 2. Drishti 2.1 Downloading Drishti Download Drishti from In the releases section there is operating system specific download link for the software. This will lead you to webpage where the.zip file can be downloaded. Extract the files. Go to bins folder, and you should be able to run drishti(.exe) now. 2.2 Loading a volume into Drishti Starting with sliced images Open 'DrishtiImport' 'Files' 'Load' 'Files' Choose your file type Choose all the sliced images >> needs instructions for export as a.pvl.nc file Starting with '.pvl.nc' file Open 'Drishti' 'File' 'Load Volume' 'Load 1 Volume' choose the '.pvl.nc' file. You can also drag and drop the file to the main interface Starting with '.xml' file Open 'Drishti' 'File' 'Load Project' choose the '.xml' file. This only works if you have saved this project file ('.xml' file) on this machine. Now you have your volume loaded in Drishti. 2.3 Choosing transfer functions to see different components of the volume When you first load a volume into Drishti, usually you cannot see the details of the object of interest, because the surrounding air (also scanned as a part of the volume) is by default opaque (example of this case shown in fig 1). You can apply a function (called 'transfer function') which allows you to see only one part of the volume, e.g. removing the low density gas, so you can see the object (same object with air made transparent in fig 2 (left)). Similarly, you can remove layers of density for example the outer shell (or set them as partially transparent) and so you can see the inner structure (fig 2 (right)). This section describes how to apply transfer functions to isolate the components that you want to see. 3

4 Fig 1. Opaque air hides inner stucture Fig 2. Removal of outer shells reveals internal structure Interpreting the 2D histogram panel When you load a volume, the 2D histogram panel appears on the right hand side of the window (Fig.3). Understanding this panel is crucial if one wants to segment a volume into different components. 4

5 Fig 3. 2D histogram panel A 3D volume is formed by many small elements called voxels, similar to pixels which form a 2D image. Each voxel corresponds to a tiny portion of the scanned object and can be described using two parameters: (1) the average density at this point on the object, and (2) the gradient of density at that point. 'Gradient of density' means the (absolute value of) rate of change of density near this point, so it is zero within a pure material, and is large on an intersection surface between two different materials. For example, suppose you are scanning an uniform rubber sphere, then any voxel corresponding to a point inside the sphere will have a constant density x, and a zero gradient of density; Any point exactly half on the sphere surface will have an average density of about x/2 (assuming the air has zero density) and a very large gradient; Any point outside the sphere will have zero density and zero gradient. Now we can set up a binned coordinate system with x axis representing the average density near a voxel point, and y axis representing the gradient of density at that point. Then any voxel can find its position in the 2D plane. Of course, there can be more than one voxels at any coordinate (x,y). Now if there are no voxels at a particular coordinate (x,y), we paint this point in the 2D coordinate system as black. If there are a maximum number of voxels at a particular coordinate (x,y), we paint this point as white. Any specific voxel can then be described using a gray-scale colour. In the example below, there are a lot of voxels with parameters (x B,y B ) and very few voxels with parameters (x A,y A ), as point A is dark, and point B is bright. 5

6 Fig 4. There are many voxels at position B but very few voxels at position A This gives us our 2D histogram. In reality, however, the density is rarely uniform due to the presence of impurities, and the transition between densities is not absolutely sharp, so you do not usually get a single bright point with completely black surroundings. The patterns are usually more diffused, so you get white patches (An artificial demonstration is given in Fig 5.). Fig 5. A scanning of uniform rubber sphere gives a few patches on 2D histogram (An artificial histogram for demonstration purpose only) Remember, any coordinate (x,y) on the histogram represents the set of points with the density x and gradient of density y. For example, near the boundary between two materials, the gradient of density (i.e. the rate of change of density with respect to spatial distances) is usually large. So when you select points with large y values, you tend to get the voxels on the boundaries where two materials intersect. 6

7 Fig 5. Upper part of the histogram gives intersection surfaces. Note the yellow box above means only those voxels bounded in the box are displayed on the left figure. For example, the voxels in the white patch near the bottom of the coordinate system has been hidden by this transfer function. This bounding box is called a 'transfer function', and you can choose its size and shape however you wish. This is explained in the next section Understanding the Transfer Functions A transfer function is a tool that allows you to select a subset of voxels in the volume. By doing so you can isolate or remove certain parts of the object. For example, the low density part of the volume is usually the air which shows minimal information. The way you choose a smaller set of voxels is by placing a bounding region on the 2D histogram. Any voxel inside the bounding region will be selected under this transfer function. The bounding region is defined by at least 6 points (Fig 6). We specify the two central big points 'spine points', and the four smaller ones 'vertexes'. The difference is that spine points can be moved to anywhere you desire, while position of any vertex is constrained on a line. This complication is needed for mathematical purposes, which we will not go further into. You can add further points to create a more complicated bounding region, but for beginners an easier way is to use 6 points per transfer function, and apply multiple transfer functions. 7

8 Fig 6. An easiest transfer function is defined by 6 points on the histogram The shape and size of the bounding region can be manually adjusted using the following list of operations: To move a point, left click on it and drag it to where you desire To add a spine point, left click. (Two vertexes will be created simultaneously.) To remove a spine point, right click on that point To move the entire transfer function (i.e. all the points at the same time), hold down the middle mouse button (scroll wheel) and drag the function to where you desire. To move two vertexes near a spine point simultaneously and make them symmetrical about a spine point, hold 'shift' and drag one of them. To move a spine point horizontally, press 'Alt' and drag To move a spine point vertically, press 'shift' and drag To undo an action, press 'Ctrl + Z' To redo an action, press 'Ctrl + Y' Tips for choosing a good transfer function A good choice of transfer function allows you to see one component or a few specific components of the volume clearly, while minimizing the noises. See pictures below for examples of good and bad transfer functions. Fig 7. A well-chosen transfer function shows internal structure of an object 8

9 Fig 8. A not-so-good transfer function does not show the component of interest. Choosing a good transfer functions can be a tedious task that can be a 'trial and error' process. However there are some tips which can be helpful to find a good transfer function: Avoid including the low-density points, these are usually the surrounding air If 2D histogram is not very informative, you can switch to 1D mode by clicking the left circle here. The height of the curve at any density x indicates the number of voxels with that particular density. These peaks thus tell you which densities the object is at and usually placing the transfer function box surrounding these peaks highlights that object.. The white patches (2D) or high peaks(1d) indicate strong signals possibly indicating a large piece of material with similar density values. Thus it is either something you definitely want to hide (e.g. soil when you want to observe roots) or something you definitely want to see. You can apply multiple colours to facilitate the observation, so you can easily see which parts have higher density, and which parts have lower density. In particular, a continuous colour spectrum usually gives a fancy picture. (Colour settings are explained in section 2.3.5) Placing a box at high gradient (i.e. high y values) usually allows you to see the boundaries where different materials intersect. This allows you to see the shape of objects. The actual manipulation can be very tedious if there are a lot of impurities in the sample. Sometimes it is impossible to segment the objects from the backgrounds, so we need to use a more advanced segmentation tool named 'Drishti Paint'. We will not go further into that for now but there is a demonstration given by the author that can be found on Youtube: Interpretation of Transfer Function Editor panel 9

10 Fig 9. Transfer Function Editor panel The 'new' button allows you to create a new transfer function. The 'remove' button allows you to remove any unwanted transfer functions. Double-click on any transfer function button to change the name, it is clearer to name it by the component this transfer function allows you to see. When you put a in the box following a transfer function, you make the voxels enclosed by this transfer function visible. Therefore, if you untick all boxes, the volume should completely vanish. The 0,1,2,3 in the table allows you to pick a few transfer functions and put them into a group, so that you can easily apply a few transfer functions simultaneously. To make things easier, beginners can simply ignore all boxes under 1,2, 3 and only tick or untick boxes under Importing transfer functions from existing projects If you have already saved other projects (how to save a project will be covered in ) then you can import the transfer functions that you have saved in other projects. This is simply done by 'Files' 'Import transfer functions from project' select the project file (it should be an'.xml' file) Changing colour and opacity The colour-opacity panel (lower part of Fig 10) allows you to set the colour and opacity of the voxels that are selected by the transfer function. By setting parts of different densities to different colours, you can see various components of a volume more clearly. The selected region colors and opacity are mapped to the transfer function. 10

11 Fig 10. The panel which allows users to change the colour and opacity of selected voxels In the above diagram (fig 10), three points A, B, C dictate the opacity and colour of zones A, B, and C respectively. Any voxel outside the bounding region of transfer function will not be seen. There is no general rule about what opacity you should choose. But in general, if you want to see the inner structure of an object, you should make the outer parts less opaque. See below for demonstration. Fig 11. Highly opaque outer shell hides inner structure 11

12 Fig 12. Less opaque outer shell allows users to see the thinner metal tube inside. 2.4 Setting key-frames and creating an animation In order to present different layers of volume in Drishti Prayog, you need to save the key-frames data. This section explains what a key-frame is, how to set a key-frame, and how to create an animation using Drishti Interpreting key-frames A 'frame' can be simply understood as a snapshot in a film or animation. If you browse through a number of snapshots quickly enough, the human brain regards this as a continuous animation. 'Keyframes' are something that define the starting and ending snapshots of a transition. If you set a few key-frames, Drishti will automatically generate the frames (i.e. snapshots) in between and create a continuous animation. This section explains how you can create a quick animation using Drishti Setting key-frames Go to 'View' 'KeyFrame Editor', you should now see a key-frame editor panel: Fig 13. KeyFrame Editor Interface 12

13 You can add transfer functions, change orientation and select different colours to make the volume look as aesthetic as you wish. Once you are happy with the volume, switch to high-resolution mode by pressing F2, and click 'Set KeyFrame' to save it as Frame 1. Fig 14. KeyFrame 1 is now defined When you hit 'Set KeyFrame', quite literally Drishti saves a snapshot of the volume displayed, including its size, colour, and orientation. Now you can edit the volume again, say applying another transfer function and change the orientation slightly. Once you are happy, simply click at another position on the frame axis, and click 'Set KeyFrame' again. Fig 15. Two Key-frames have been defined To change the title of these tabs, click on a picture and hit spacebar, and the following interface allows you to change the parameters of the tab in Drishti Prayog. 13

14 Fig 16. An interface which allows you to change the name and parameters of a Key-Frame These two key-frames you chose will be the different tabs you see on Drishti Prayog. Fig 17. Different Key-frames on Drishti will turn into different tabs on Prayog Generating an animation Now you have two key-frames, and Drishti can generate an animation automatically. Click the button, Drishti will generate an animation showing a smooth transition from one key-frame to another. Now you can save the video into '.wmv' file (only Windows) using 'File' 'Save movie'. You will need to manually change the resolution to make sure both width and height are multiples of 16. One valid example is shown below: 14

15 Fig 18. 'Save movie' interface. The 'Step Frame' can be used to change the interval between one frame and another, thus changing the time of a transition. If you want to play this video in Prayog (see sections and 3.2.1), then you have to save the video under the same directory as other project files, such as '.xml' and '.drishtiprayog' files. Of course you can add more key-frames into the project. But avoid putting too many, because it may look cluttered on Drishti Prayog Adding a picture/video/text to the current project This section tells you how to add a video/picture/text to your project. By adding a button on the screen, you simply click on it to show a video, an image, or a text description in a pop-up window. Fig 19. An info-icon which allows you to click and show a video, text or picture 15

16 Step (1): select a point: switch to high-resolution mode by pressing F2 place mouse on the volume 'Shift + left click', a blue dot appears you can move it by simply left clicking and dragging Step (2): hover the mouse on the point, the point turns green press the 'i' key choose the video/image/text file you want to show save key-frame Now you should be able to play the video by pressing the button. When you save the project, and save the information for Drishti Prayog, you can show the video/image/text in Prayog as well. Fig 20. A video is played in a pop-up window when you click on the info-icon 2.5 Save files for Drishti Prayog This section explains how to save the information needed for display on Prayog Save project You need to save the project, so you can easily come back to the current Drishti project and make amendments. Go to 'File' click 'Save project as'. This will: (1) save the volume information and transfer functions in an '.xml' file, and (2) save key-frames in a '.keyframe' file Save information for Drishti Prayog You also need to save the information in a file accessible to Drishti Prayog. So click 'File' 'Save information for Drishti Prayog'. This generates a '.drishtiprayog' file. There is one more file required (i.e. 'configuration file') before you can open a Prayog project. This is explained in section

17 3. Drishti Prayog 3.1 Installing Prayog Download Prayog from Extract the zip file, and everything you need is in the 'bin' folder Setting up project path You have to let Prayog know where you have stored the project files. This can be done using the 'drishtiprayog.conf' file. Fig 21.Prayog Configuration file which allows you to set the project path Simply copy and paste the address of the folder where you saved your project files. Fig 22. The line where you can change the project path This folder should contain all of the projects you want to display using Prayog: 17

18 Fig 23. All projects should be saved under this path 3.2 Creating configuration files for Prayog projects From section 2.5 you should already have three files in your project folder, namely an '.xml' file, a '.keyframe' file, and a '.drishtiprayog' file. To open the program in Drishti Prayog you need to have another file, namely configuration file('.conf' file). This can simply be done by opening a notepad and copy and paste the following: tag : astronomy stereo : false title_size : 50 text_size : 20 image: dens.png title : Astronomy simulation description : This simulation presents jets going through an interstellar medium of galaxies. Usually jets are two-sided, but because of geometry and physical symmetry only one side is simulated. The contents after each colon ':' are the parameters of that particular page in Drishti Prayog. Here is a table explaining what these items mean: tag stereo title description title_size text_size image Projects can be categorised using this item. You can select the category at first page of Drishti Prayog. true/false quadbuffer active stereo Title of the project to be put on cover page Description of the project on project's cover page Project title size Description text size Image file name of cover page background. (Make sure this image file is in the same directory as other project files.) A full list of items you can put into configuration file can be found here: 18

19 Now simply change the parameters such as the title and background image, then save the file as 'abc.conf'.(fig.) Note the extension '.conf' is important: if you saved it as '.txt', you will not find this project in Drishti Prayog. Fig 24. Configuration file has to be saved as the same name as all other files and the folder Essentially, you will need at least four files to open a project in Drishti Prayog: an '.xml' file, a '.keyframe' file, a '.drishtiprayog' file, and a '.conf' file. Make sure all four files have the same name (with different extensions) as the folder. If not the Prayog will not recognise those files. You should end up having something like Fig 25. Fig 25. A valid folder structure for a Prayog project The project is now ready for viewing on Drishti Prayog Playing a video 19

20 You can play a video in full-screen mode, instead of showing a 3D volume. To do this, simply add the following lines into your configuration file: executable : drishtimovie.exe movie : abc.wmv This instructs Prayog to start playing the video once you entered the project. Fig 26. A screen-shot for a project which contains only a video However, if you want to show a video using a small pop-up window attached to a displayed 3D volume, refer to the section Useful tools in Prayog Open Prayog by double-clicking in the 'bin' folder. Most of the features should be intuitive. Some of the useful tools are explained below. Clipping tool : This allows users to observe the cross-section of a volume. On a touch screen you can tap and drag with three fingers to get a smooth transition of cross-sections. Light source : This allows users to add additional light sources of different colours. Colour panel : This allows users to change the colour of the volume. 20

21 4. Common problems with solutions 1. When you try to load a project using the '.xml' file, an error message popped up: ' is not a valid preprocessed volume file'. This error occurs if you are trying to load a project (using the '.xml' file) which was not saved on this machine. You need to load the volume (using the '.pvl.nc' file), then import the transfer functions and key-frames. Even if the '.xml' file is saved on this machine, if its name or location has been changed, you will not be able to load the project. Again you have to load the volume instead. 2. After you have correctly set the key-frames on Drishti and imported it on Drishti Prayog, some frames could not render properly in Drishti Prayog. When you go back to Drishti and click on the key-frames, it takes a long time to reload the slices. You should delete the keyframes which did not work, and reset the keyframes that did not render properly. It should now work. 3. Drishti Prayog is running slow/lagging. You should lower the resolution. Always make sure Prayog is interactive, so that the audience can explore the sample. You can do so by going to 'View' 'Preferences' 'Tab 1' Drag the 'Still Image Quality' to the left. 4. Ineffective segmentation: You cannot find any transfer function which gives you a satisfactory view, i.e. the region of interest doesn t have much phase contrast and requires a manual segmentation. Under the 'bin' folder of Drishti, there is a tool called 'Drishti Paint', which is designed specifically for segmentation. Videos on how to use Drishti Paint can be found on Youtube: 5. Noisy data: If the data is noisy then you can apply noise filters using ImageJ ( 21

22 5. Additional links A few useful links which allow users to further explore Drishti and Drishti Prayog: An detailed summary by Bailey Lovett, elaborating on useful Drishti commands for visualisation and quantification: Presentation-Tool Online help page with documentations: A list of demonstration videos created by the author, Ajay Limaye, can be found on his Youtube channel: A slightly older version of Drishti-Prayog user-guide written by Alice Moyle, Diamond Light Source Ltd, can be found here: 22