IMS database application manual

Transcription

1 IMS database application manual The following manual includes standard operation procedures (SOP) for installation and usage of the IMS database application. Chapter 1 8 refer to Windows 7 operating systems and the Chapters 8 and 9 refer to a UNIX environment. Contents 1. Installation Java environment Installation PostgreSQL server Include PostgreSQL tools directory into your system path Installation: first IMSDB application run Upload a project Patient view Machine learning Backup and restore Creating a backup Restoring the database Vacuuming Installation Linux (Ubuntu) Installation Java environment Installation PostgreSQL database server Ubuntu (Linux): first IMSDB application run... 40

2 Chapter: Installation Java environment 1. Installation Java environment In order to use the IMS database application you need to install the Java Runtime environment (JRE) version 6 or later. For more experienced users, the JRE can be downloaded manually from the page which offers different versions for different operating systems. A more convenient option is to test for the java version on the following page This page gives you information about old versions which have to be removed, etc. If you do not have java installed or the version is below 6 then click on Downloads ( and get the installation file and execute it. Then click on install (installieren). Uncheck the installation of browser addons.. After the installation the following window appears. 2

3 Chapter: Installation PostgreSQL server 2. Installation PostgreSQL server Go to In section One click installer click on Download and choose the latest version (version 9.0/9.1/9.2 have been tested). If you use a 64 bit Windows 7 operating system, it is recommended to choose Win x If you use a 64 bit Windows 7 operating system, it is recommended to choose Win x Start the installer, click on next. 3

4 Chapter: Installation PostgreSQL server Give a password for the superuser postgres. This login is required later if you want to connect to your database. Specify the following port 5432 and click next. Click next. 4

5 Chapter: Installation PostgreSQL server Uncheck the box and click finish. The server is installed. 5

6 Chapter: Installation PostgreSQL server Start the following application "C:\Program Files\PostgreSQL\[YourVersion]\bin\pgAdmin3.exe" Connect to the database server 6

7 Chapter: Installation PostgreSQL server Create a new database 7

8 Chapter: Installation PostgreSQL server Call your database imsdb. Then your connection url is jdbc:postgresql://localhost:5432/imsdb The imsdb application will ask for this url later. 8

9 Chapter: Installation PostgreSQL server Your database should be visible in the tree: 9

10 Chapter: Include PostgreSQL tools directory into your system path 3. Include PostgreSQL tools directory into your system path In order to include the backup and restore functionality, the PostgreSQL bin directory must be included into your system path. Open the system control panel Click on system. 10

11 Chapter: Include PostgreSQL tools directory into your system path Click on environment variables. Select the variable Path and click on modify. 11

12 Chapter: Include PostgreSQL tools directory into your system path Add the following directory (depending on your version) ; C:\Program Files\PostgreSQL\[your version]\bin Do not forget the semicolon ; as separator. Press OK. Now you can start the imsdb application. 12

13 Chapter: Installation: first IMSDB application run 4. Installation: first IMSDB application run The config file general.properties will be created after clicking on close (schließen). 13

14 Chapter: Installation: first IMSDB application run After clicking on close (schließen), you have to choose a directory in which the root directory Dump for log files, dump files and backup files is created. In this example the directory test1 is chosen. The Dump directory as well as subdirectories of Dump are created. 14

15 Chapter: Installation: first IMSDB application run The application then asks for a database connection url. A default url is given as shown in the next snapshot. If you followed the previous installation manual for the database server, this should be Okay and you can leave it unchanged and just click on ok. After giving the database connection url which is automatically inserted into the config file general.properties, the application starts and you can click on the button Connect to IMS database 15

16 Chapter: Installation: first IMSDB application run You have to enter the login information of the database super user you created when installing the database server. This user is typically named postgres. The database application will detect that the necessary database tables are not yet created. It asks if it should create them. Click on yes! After all the database objects (e.g. tables) are created, the following progress bar shows 100% progress. Now you might want to create a backup of the clean database (version 0). See Creating a backup on page

17 Chapter: Upload a project 5. Upload a project Click on Upload a project and specify the directory (here: HemerProjectDatabase ) where the files of your project are located in. Note that files in subdirectories of the project directory are ignored. Click on open. The next dialog will ask you if you want to include a Patient table-file which includes diseases, medication, or any other attributes related to patient ids. Those ids are expected to show up in the related IMS measurement files in the field sample ID. Click yes. 17

18 Chapter: Upload a project Choose the patient table PatientDataTable.xls. Click open. In order to have categories (ConceptClass names) for the uploaded measurements, you can specify where these names are taken from. Here you can choose the first option (or the 3 rd option, same effect) Click on the first option, click on choose (Auswählen). Marginal information: The first option will assign alpha-numerical ids to hemer_control and numerical ids to hemer_patient and hemer_patient_case (case for several measurements of the same patient). 18

19 Chapter: Upload a project A dialog shows you which new measurements will be stored in the database and which measurements are already stored. You could cancel the upload project now, but we want to upload, so click on Yes, please. The following progress monitors will show you the progress state. 19

20 Chapter: Upload a project You are asked if the system should overwrite already backupped IMS measurement files. Choose yes to skip the overwriting. Then the information of the patient table is stored into the generalized data structure componend (concept, values, attribute and associations to the IMS measurements). 20

21 Chapter: Upload a project The next dialog shows you that the uploading process is finished. Now you might want to create a backup. See Creating a backup on page

22 Chapter: Patient view 6. Patient view Click on the Patient View tab of the application. Click on Refresh to show the patients. You can switch the categories (concept class names) by using the select field next to the Refresh button! 22

23 Chapter: Patient view Click on a particular patient in the tree to visualize her attributes and values. In this example, we picked Patient This patient has two entries, meaning two corresponding IMS measurements. Furthermore, the id of this patient changed. Therefore there is an id1 attribute to collect the first id the patient was registered with, the id2 attribute contains the new id of the patient. If the patient got another new id, this id would be reflected in the id2 attribute in another entry for the patient and the id1 attribute would stay unchanged for this new entry. 23

24 Chapter: Machine learning 7. Machine learning Click on the Machine Learning tab. Click on Refresh. On the left side, a list of attributes of the class hemer_patient are shown. Click one attribute to compare instances having this attribute with instances which reveal a false value for this attribute. Here we choose insulin and press the 24

25 Chapter: Machine learning Query button. It is also possible to select several attributes by holding shift to do comparisons between e.g. control versus disease. Note that if you wanted to see the control attribute also you have to switch to all in the combobox Choose a class for categorical attributes. Tell the application which classes of instances you want to include in the comparison. Hold the shift key to select several concept classes! Then press OK. A dialog pops up which shows you about removed instances because they do not belong to your chosen classes. Click OK. 25

26 Chapter: Machine learning The next dialog tells you about which instances are removed because they do not have at least one true value for the given attributes. IMPORTANT: This removal is only performed if two or more attributes were chosen. If then for both attributes are false values, it makes no sense to include them into the result dataset for the machine learning. In the case of only one attribute is given, samples with false and true values for this attribute should be included, so nothing is removed here. Click OK. The next dialog is used to exclude peaks (peak regions). Hold the shift key to select several peaks. Press OK. 26

27 Chapter: Machine learning The following table includes all parameters of the peaks. RT and IM denote the coordinate of the maximum signal intensity, target-rt and target-im denote the center point of the target region used to retrieve the peaks. Target-radRT and target-radim denote the radius of the target region. PeakNo denotes an artificial index of the peaks which are ordered according to the increasing RT. 27

28 Chapter: Machine learning The decision tree which was trained on the overall dataset pops up. You can click on a node to enter another visualization window. Right click gives you the choice of changing the font size etc. We click on the root node. 28

29 Chapter: Machine learning After clicking on the root node of the tree a data visualization window opens. This window includes all instances of the dataset. If you click on a lower node, only the instances which are included in THIS particular node are shown! You can change the axes according to the particular features or you choose the nominal class as axis, etc. 29

30 Chapter: Machine learning The corresponding WEKA dataset is shown in the area on the right side of the application. The evaluation result of the ten fold cross validation (which is not reasonable for really small datasets) is shown below. 30

31 Chapter: Backup and restore 8. Backup and restore 8.1. Creating a backup After you uploaded a project, it is recommended to create a backup of the database. Go to the dump window by clicking on Menu and Backup. Click on Dump! to create a backup. 31

32 Chapter: Backup and restore The process is finished when you see the following message. Take a look into your root directory ( Dump ) and find the created dump archive file. 32

33 Chapter: Backup and restore 8.2. Restoring the database It is ensured that all users are disconnected from the database before the process starts. If you want to restore a dump archive, click on Restore and choose the desired dump archive. Then the restore process will start and end with the following dialog. 33

34 Chapter: Backup and restore 8.3. Vacuuming The main reason for having the vacuum button is to remove unused large objects referring to IMS measurement files which became orphan because of doing a restore. If you do not store the IMS measurement files into the database (default case, general.properties: saveimsrawdata=0), then you do not need this method. It is ensured that all users are disconnected from the database before the process starts because it requires table locks. Basically if you use this command the postgres application vacuumlo is called to mark the raw data and vacuum full to rewrite all table contents. Also According to the postgres manual, vacuum full is only recommended in a case where a big amount of tuples are deleted, such as having orphaned large objects. The Analyze flag is set for the vacuum full command. Analyze examines all tables and updates statistics used by the query optimizer. If you just want to update such statistics you do not need the vacuum button here. Open pgadmin3.exe and vacuum analyze by right click on your database and follow the vacuum dialog. According to the PostgreSQL documentation VACUUM FULL rewrites the entire contents of the table into a new disk file with no extra space, allowing unused space to be returned to the operating system. This form is much slower and requires an exclusive lock on each table while it is being processed. ( last accessed ). 34

35 Chapter: Installation Linux (Ubuntu) 9. Installation Linux (Ubuntu) The following installation steps are performed on Ubuntu Installation Java environment In order to run the IMS database application, a Java Runtime Environment is required. Install OpenJDK version > 6. We take version 7.: $ sudo apt-get install openjdk-7-jre Check your version. There should be at least Installation PostgreSQL database server 1. Install the latest PostgreSQL server which is in this case postgresql-9.1: $ sudo apt-get install postgresql Note that if you want to make sure installing version 9.1: $ sudo apt-get install postgresql

36 Chapter: Installation Linux (Ubuntu) 2. The following contrib package is required in order to use the vacuumlo command-line tool: $ sudo apt-get install postgresql-contrib If you want to make sure version 9.1: $ sudo apt-get install postgresql-contrib Install the graphical user interface tool Pgadmin3 for managing the postgres server. $ sudo apt-get install pgadmin3 36

37 Chapter: Installation Linux (Ubuntu) 4. Enter the postgres terminal to set a password for the super user postgres which was created automatically for administration. $ sudo u postgres psql postgres You will enter the postgres terminal, there you set the pw. It will be decrypted and send as an alter role command, no plain text or log etc.: #\password postgres Enter new password: Enter it again: 5. Check the port and if your server is running: $ sudo service postgresql status 37

38 Chapter: Installation Linux (Ubuntu) or $ /etc/init.d/postgresql status You can start/stop the server: $ sudo service postgresql start $ sudo service postgresql stop $ sudo service postgresql restart This also works if you call the following $ sudo /etc/init.d/postgresql start $ sudo /etc/init.d/postgresql stop $ sudo /etc/init.d/postgresql restart 6. Use the convenience command line tool to create a database with name imsdb. (This tool does nothing else than calling CREATE DATABASE in the postgres terminal.). Further details can be found on Alternatively you can use this graphical tool pgadmin3 to create the database imsdb as described in the Windows 7 installation guide or you follow the next steps and create the database by using the command line. 38

39 Chapter: Installation Linux (Ubuntu) $ sudo u postgres createdb imsdb 7. Optionally you can check the details of your database by entering the postgres terminal. $ sudo u postgres psql postgres # \l 39

40 Chapter: Ubuntu (Linux): first IMSDB application run 10. Ubuntu (Linux): first IMSDB application run Compared to the first run in Windows 7 and in the UNIX environment, there are no application specific differences. Therefore the author refers to Installation: first IMSDB application run on page 13 for details. Start the application: $ java jar ImsDbApp.jar The following screenshots illustrate the first steps of starting the application, uploading a project. Please see Upload a project on page 17 for details. 40

41 Chapter: Ubuntu (Linux): first IMSDB application run 41

42 Chapter: Ubuntu (Linux): first IMSDB application run 42

43 Chapter: Ubuntu (Linux): first IMSDB application run 43

44 Chapter: Ubuntu (Linux): first IMSDB application run 44

45 Chapter: Ubuntu (Linux): first IMSDB application run Hint: If you followed some other installation of the postgres server and the vacuum command in the dump window of the IMS database application (menu -> backup -> vacuum) is not working because vacuumlo tool is missing ("Error: pg_wrapper: invalid command name") you need to install the contrib package of postgres I: sudo apt-get install postgresql-contrib-9.1 which was stated above to install. 45