Configuring and Running CD Tools

Transcription

1 Presented to: Advanced IDC Training Course Configuring and Running CD Tools Preparatory Commission for the Comprehensive Nuclear-Test-Ban Treaty Organization Provisional Technical Secretariat Vienna International Centre P.O. Box 1200 A-1400 Vienna AUSTRIA Page 1

2 CD Tools Software Basic Principles of Operation 1. Make appropriate directories 2. Configure the software 3. Start the program with the appropriate command line argument(s) 4. Program reads the configuration file(s) 5. Program reads input 6. Program writes output 7. Program writes log files 8. In on-line mode, some programs (cdrecv, cdsend, cd2w) run forever Software must be properly configured before starting Data will be stored under /home/user1/idcsw/data Configuration file will be under /home/user1/idcsw/config Reports will be written to /home/user1/idcsw/reports Log files will be written to syslog and /home/user1/idcsw/logs Full details concerning CD Tools operation are available in CD Tools Software User s Guide (on the CD) Page 2

3 CD Tools Installation Make the necessary data directories mkdir /home/user1/idcsw/data/frames mkdir /home/user1/idcsw/data/frames/receiver mkdir /home/user1/idcsw/data/frames/sender mkdir /home/user1/idcsw/data/frames/sender/ref mkdir /home/user1/idcsw/data/waveforms mkdir /home/user1/idcsw/data/work mkdir /home/user1/idcsw/param mkdir /home/user1/idcsw/seg Make the configuration, reports, and logs directories mkdir /home/user1/idcsw/config mkdir /home/user1/idcsw/reports mkdir /home/user1/idcsw/logs Page 3

4 CDrecv and CDsend sample parameter files Sample parameter files can be found in the directory /home/user1/src/cdtools-x.x.x/samples The files of interest are: config.par sample configuration file used by CDrecv and CDsend station_file used by CDrecv to limit which stations can connect dataconsumers used by CDsend to specify consumer names, IP addresses and ports productionlines used by CDsend to specify which station(s) should be sent to which dataconsumers In the following steps, we will identify what parameters need to be changed copy these files to our own area configure the files appropriately Page 4

5 CDrecv and CDsend sample parameter files Before configuring the files, we need to determine: what parameters we will change the new values of those parameters Data will come from the IDC, to our receiver The data will come from the machine: The station that will be sent by the IDC: BOSA (your station: ) The address where my CDrecv will listen: 8000 (your port: ) Path where my data will be stored: /home/user1/idcsw/data/frames/receiver/%y/%m/%d Path to where my index file is created: /home/user1/idcsw/data/frames/receiver Path to the station file: /home/user1/idcsw/config/station_file Path to the dataconsumer file: /home/user1/idcsw/config/dataconsumers Path to the production lines file: /home/user1/idcsw/config/productionlines Page 5

6 CDrecv and CDsend sample parameter files Copy the sample files to the correct location cd /home/user1/idcsw/config/ cp /home/user1/src/cdtools /samples/config.par cdtools.par cp /home/user1/src/cdtools /samples/station_file. cp /home/user1/src/cdtools /samples/dataconsumers. cp /home/user1/src/cdtools /samples/productionlines. chmod u+w * Note: The parameter file config.par should be named cdtools.8000.par, where 8000 is your port number! This will make it easier to find your process in the future. Page 6

7 CDrecv parameter file (config.par) Edit the CDrecv/CDsend parameter file correctly The file to edit is /home/user1/idcsw/config/cdtools.par Edit the following lines. The new parts are in bold. port=8000 filepath=/home/user1/idcsw/data/frames/receiver/%y/%m/%d Add the following new lines to the file: cdrecvindexfilepath=/home/user1/idcsw/data/frames/receiver (stationfile=/home/user1/idcsw/config/station_file) dataconsumerfile=/home/user1/idcsw/config/dataconsumers productionlinefile=/home/user1/idcsw/config/productionlines refliststorepath=/home/user1/data/frames/sender/ref enabletextfile=more Page 7

8 CDrecv station file Station File The station file specifies which stations are allowed to connect to the receiver, and the IP address where the station will connect from. For ndc0, we want to allow the station BOSAto connect from Add the following line to the file /home/user1/idcsw/config/station_file BOSA= Page 8

9 CDsend dataconsumer file Dataconsumer File The dataconsumer file specifies where data can be sent by CDsend. The file specifies consumer names, IP addresses and ports For NDC0, we want to send data to NDC1. Add the following line to the file /home/user1/idcsw/config/cdtools/dataconsumers ndc1= :8100 Remove the other sample entries in the file. Data will be sent in the following way: ndc0 ndc1 ndc2 ndc3 ndc4 ndc5 ndc6 ndc7 ETC Note: Entries in this file are case sensitive, so the consumer name give here must be identical with the name in the production line file (next slide) Page 9

10 CDsend production file Production Line File The productionline file specifies which station(s) should be sent to which dataconsumers by CDsend. The file specifies station name(s) and data consumer names For ndc0, we want to send data from the station STKA to NDC1. Add the following line to the file: /home/user1/idcsw/config/productionlines STKA;ndc1;;; Remove the other sample entries in the file. Note: Entries in this file are case sensitive, so the consumer name give here must be identical with the name in the production line file (previous slide) This file can also be used to send only a subset of channels to a consumer This is explained in the User s Guide and by comments in the file Page 10

11 Start CDrecv For now, we will start cdrecv from the command line, with the following type of command: /home/user1/idcsw/bin/ /home/user1/idcsw/config/cdtools.8000.par > \ /home/user1/idcsw/logs/cdrecv.1.log 2>&1 & If all goes well, this will send the process to the background, and data will begin to flow to your ndc Page 11

12 Basic CDrecv Monitoring After cdrecv has started, we can use different commands to check things: Is the process running? Use ps to see that the process is running: $ps ef grep cdrecv grep ndc0 UID PID PPID C STIME TTY TIME CMD ndc Nov25? 00:11:23 /home/user1/idcsw/bin/cdrecv /home/user1/idcsw/config/cdtools.par Was the log file created? Check that the log file was created $ls l /home/user1/idcsw/logs/cdrecv.1.log -rw-r--r-- 1 ndc0 ndc0 0 Nov 17 12:46 cdrecv.1.log By default, all log information is written to syslog, and not the log file, so the log file should be empty Page 12

13 CDrecv Data Monitoring (1/2) If cdrecv is running, then we can check the data related files: Has a station connected successfully? An entry in the index file is created after a station has successfully connected: $ cd /home/user1/idcsw/data/frames/receiver $ ls <the response should show an index file with today s date> _ idx $ more _ idx /home/user1/idcsw/data/frames/receiver/2006/12/02/stka =cd1 This indicates that we should find a group of files in the directory /home/user1/idcsw/data/frames/receiver/2006/12/02/ which begin with STKA Page 13

14 CDrecv Data Monitoring (2/2) $ cd /home/user1/idcsw/data/frames/receiver/2006/12/02/ $ ls STKA * STKA bin <- contains the frames as received from sender STKA clf <- concise list of frames STKA txt <- text representation of frames Examine the txt file (STKA txt). This file can also be monitored with the tail f command. Page 14

15 CDrecv Log Messages Output messages written by CDrecv are written to syslog by default The default location for syslog messages under Linux are in files under /var/log idclx007.idc.ctbto.org:ls /var/log/mess* /var/log/messages /var/log/messages.1 /var/log/messages.2 /var/log/messages.3 /var/log/messages.4 Many applications can write messages to syslog To find the messages written by CDrecv listening on port 8200, use the following command: idclx007.idc.ctbto.org:grep CDrecv_8200 /var/log/messages less Nov 27 04:08:49 idclx007 ctbt.acquisition.cdrecv_8200[27588]: ( ) SUR: Connection info: connected, CD1.0, 8202, unknown Log time machine program PID (process id) message text The format of syslog output can be customized by your system administrator Page 15

16 Allow a new station to be received by CDrecv To allow a new station to be received by CDrecv: 1. Update the station_file used by CDrecv If ndc8 (IP address ) wants to send station XYZ to us (ndc0), we need to add the following line to the file /home/user1/idcsw/config/station_file XYZ= Stop cdrecv Find out the process ID of CDrecv on your port, and send it a kill -2 signal, which allows a safe shutdown $ ps ef grep cdrecv grep ndc0 UID PID PPID C STIME TTY TIME CMD ndc Nov25? 00:11:23/home/user1/IDCSW/bin/cdrecv $ kill Restart cdrecv We could restart CDrecv as we did before. But this time we will use the start.cdtool script instead. This script is a wrapper, which will start a process and restart the process if it stops. Any time the process is restarted, the script will record the event in syslog Page 16

17 start.cdtool script (1/2) Start.cdtool is a wrapper script, which will start a process and restart the process if it stops. Any time the process is restarted, the script will record the event in syslog Example: /local/ndc0/bin/start.cdtool -p /local/ndc0/bin/cdrecv -c /local/ndc0/config/cdtools/cdtools.8000.par -l /local/ndc0/logs \ -o ndctr0 -h idclx007 u ndctr0 # <the script itself contains this text, and a lot more!> # Mandatory arguments # -p PROGRAM - full path to the CD Tool to start. # -c CONFIG_FILE - configuration file to be read by the CD Tool. # -l LOG_DIR - directory that contains the log file written by the CD Tool. # # Optional arguments # -o OPER - when the CD Tool is started or is stopped, mail is sent to the # user specified by oper. If the list is in quotes and comma separated # then mail is sent to all of the users in the list. # -h HOSTNAME - the name of the host where the script should be run. If this # name does not match the output of the 'hostname' command, the script # will exit with a fatal error. If this argument is not given, this # check is skipped. # -u USERNAME - the name of the user which should run this script. If this name # does not match the output of the 'whoami' command, the script will # exit with a fatal error. If this argument is not given, this check # is skipped. # -s LOGSTRING - a string which is added to the name of the logfile, to help # with identification. If used, the log file will be named: # $program_name.$port.$pid.$logstring.log # If LOGSTRING is not given, the log file will be named: # $program_name.$port.$pid.log Page 17

18 start.cdtool script (2/2) After the station file is fixed, restart your CDrecv and start your CDsend. You can check the status of both processes as before Example for CDrecv: /home/user1/idcsw/bin/start.cdtool -p /home/user1/idcsw/bin/cdrecv \ -c /home/user1/idcsw/config/cdtools.8000.par \ -l /home/user1/idcsw/logs -o ndctr0 -h idclx007 u ndctr0 > /dev/null 2>&1 & Example for CDsend: /home/user1/idcsw/bin/start.cdtool -p /home/user1/idcsw/bin/cdsend \ -c /home/user1/idcsw/config/cdtools.8000.par \ -l /home/user1/idcsw/logs -o ndctr0 -h idclx007 u ndctr0 > /dev/null 2>&1 & Page 18

19 Start cdrecv and cdsend from the command line Alternatively start both CDrecv and CDsend from the command line. You can check the status of both processes as before Example for CDrecv: /home/user1/idcsw/bin/cdrecv /home/user1/idcsw/config/cdtools.8000.par > \ /home/user1/idcsw/logs/cdrecv.1.log 2>&1 & Example for CDsend: /home/user1/idcsw/bin/cdsend /home/user1/idcsw/config/cdtools.8000.par > \ /home/user1/idcsw/logs/cdsend.1.log 2>&1 & Page 19

20 Reporting Problems All software has problems, and users always want enhancements Often times these problems and needs are not well communicated Making a good bug report What software was being used? (include version number!) What were you doing when the problem was observed? Is the problem reproducible? (if it is, and you can describe how to reproduce it, the probability of getting this problem fixed increases dramatically) Some problems are data related. This means that the problem can only be reproduced with a specific type of data (e.g., unexpected data). If this is the case, make a note of it, and include the data with the bug submission (if possible). Problems and enhancement requests should be sent to services@ctbto.org Page 20

21 Examining Problems When applications unexpectedly fail under UNIX, they often produce a core file, which is a file named core Examining a core file can provide a valuable insight into the problem Core files are examined using a debugger The debugger must be compatible with the compiler which was used The debugger which comes with gcc is named gdb (there are also GUI front ends to gdb) The CFLAGS= -g option given to the./configure script produces programs which are easier to check with the debugger Starting gdb: gdb /home/user1/idcsw/bin/cd2w core Some useful gdb commands: gdb> where gbd> list gdb> print $variable Page 21

22 Daily Station Routine (1/2) # 6 1,6,18 * * * find /home/user1/idcsw/data/frames type f mtime +2 exec rm f {} \; > /dev/null 2>&1 Cronjob to delete old data Once we begin to receive data, we need a way to delete it before all the disk space is used We can automatically clean the disk of old data using the Unix crontab facility Only need to do this only once, and then it will run everyday (until it is removed) Edit your crontab entry, and add the following type of line: $ crontab -e # remove files after 2 days # 6 1,6,18 * * * find /local/ndc0/data/frames -type f -mtime +2 -exec rm -fr {} + > /dev/null 2>&1 # run command start the find here look for files older run rm command for matched at 1:06, 6:06 and plain files than 2 days files 18:06 everyday Page 22

23 Daily Station Routine (2/2) Visualizing data timeliness The receipt time of each CD frame is recorded in the clf file These times can be visualized using the tool pnt, e.g., /home/user1/idcsw/bin/pnt KMBO clf Abnormal Normal Page 23