AMON Software Hands-on

Transcription

1 AMON AMON Software Hands-on Gordana Tešić (for the AMON team) Under development at The Third AMON Workshop December 11-12, 2014, DESY Zeuthen,Germany The Pennsylvania StateinUniversity AMON 1

2 Installation Installa0on ============ Ignore if you have a developer distribu0on of the code from the AMON SVN repository and/or if you are planning to make changes to the modules To install AMON package run: python setup.py install Code will run without installa0on as well, if run from the source code subdirectories: amonpy/ops and amonpy/ops/network Installa0on 2

3 Installation If installed: simula0on and clustering analysis scripts can be found in build/ scripts- 2.7 Copy dbaccess.txt file from amonpy/ops in your running directory and edit it to put your own informa0on there. Also add these lines to your.bash_profile or create a new script to source before running the installed code: export AMONPY="path- to- your- AmonPy" PYTHONPATH="$AMONPY/amonpy:${PYTHONPATH}" PYTHONPATH="$AMONPY/amonpy/dbase:${PYTHONPATH}" PYTHONPATH="$AMONPY/amonpy:${PYTHONPATH}" PYTHONPATH="$AMONPY/amonpy/ops:${PYTHONPATH}" PYTHONPATH="$AMONPY/amonpy/sim:${PYTHONPATH}" PYTHONPATH="$AMONPY/amonpy/tools:${PYTHONPATH}" PYTHONPATH="$AMONPY/amonpy/anal:${PYTHONPATH}" export PYTHONPATH These lines are not needed to be sourced if code is run from the source code directories Installa0on 3

4 Documentation Documenta0on ============ To browse the AmonPy package documenta0on, open docs/html/index.html file with your browser. Also README files in every subdirectory explain how to run the code Docs 4

5 Create AMON databases (archival data and simula0on) AmonPy/mysql Database from mysql prompt: CREATE DATABASE <DBNAME>; 1) db_build.sql is a script that creates AMON_test1 database tables for archival data 2) db_mc_build.sql is a script that creates AMON_test2 database tables for MC data To run, use: mysql - u username - h hostname - p dbname < scriptname DB 5

6 Database DB 6

7 Test wri0ng to DB, populate AMON_test1 with IceCube- 40 and AMON_test2 with default alert and event configura0on files. Database AmonPy/amonpy/dbase/test Edit and run test_db_write.py This script will populate event and alert configura0on tables (parents of "alert" and "event" tables) both for archival and real- 0me analysis (in AMON_test1 and AMON_test2), and also it will write archival IceCube- 40 data into AMON_test1. DB 7

8 Simulations Top level script to run basic simula0ons for some triggering observatories (IceCube, Auger, HAWC, ANTARES) Run script AmonPy/build/scripts2.7/run_basic_sim.py or AmonPy/amonpy/ops/run_basic_sim.py AMON_test2 will be populated with these MC data Simula0ons 8

9 Archival analysis Coincidence search archival example Run script AmonPy/build/scripts2.7/run_archival.py or AmonPy/amonpy/ops/run_archival.py This is a top level script which imports a large 0meslice of events from the database and executes an analysis on them to generate alerts. python run_archival.py - - help Archival 9

10 Archival analysis op#onal arguments: - h, - - help show this help message and exit - v, - - verbose Be verbose. - p password, - - password password Input password in plain text on command line instead of being asked for it at the start of execu0on requirements: - H address, - - host address Database host address - u username, - - username username Database username - d name, - - database name Name of database - o int, - - output- config int What to do with the alert, 1=Do note write to database, 2=Overwrite alert stream, 3=Make new alert stream Archival 10

11 Archival analysis alert configura#ons (most choose one and only one): - - use- test- config Use the test Alert configura0on [Default] - - use- db- config Get Alert configura0on from database - - use- new- config Create a new Alert configura0on op0ons required if invoking - - use- new- config: - s int, - - stream- config int Stream configura0on, 1=IceCube only,2=antares only,3=hawc only, 4=Swil only, 5=IceCube + ANTARES,6=IceCube + HAWC,7=ANTARES + HAWC, 8=IceCube +Swil, 9=All - P float, - - pvalue float P- Value threshold level, 0.0 means no threshold [default: 0.0] - n [int [int...]], - - num- events- thresh [int [int...]] Number of events threshold level; each detector in stream- config needs 1 int, and in the order listed. e.g. if steam- config=4, to use a num thresh of 2 for IceCube and 7 for ANTARES, - n would take 2 7 as its input Archival 11

12 Archival analysis - c int, - - cluster- method int Analysis cluster method, 1=Fisher (no other methods supported yet) - t float, - - 0me- window float Search 0me window - - cluster- threshold float Cluster threshold (# of sigma) Archival 12

13 RabiqMQ server a broker for transmisng messages about new received events between the AMON server and AMON analysis code sudo rabbitmq- server Real-time or for daemon sudo rabbitmq- server detached to terminate: sudo rabbitmqctl stop To monitor it: hqp://localhost:15672/ Real- 0me 13

14 Run AMON analysis code as a Celery worker Real-time Before running the script, edit file AmonPy/amonpy/ops/ analyser/runuanal.py to enter your creden0als i.e. change this lines with your informa0on: self.hostfancyname='localhost self.userfancyname='yourname self.passwordfancy='yourpass self.dbfancyname='amon_test2 self.alertdir = 'your directory to save each alert in a form of VOEvent Real- 0me 14

15 Run AMON analysis code as a Celery worker: Real-time celery worker - - app=analyser - - concurrency=1 - l info or as a daemon celery worker - - detach - - pidfile="celery.pid - - logfile="celery.log" - - app=analyser - - concurrency=1 - l info To terminate the process: kill `cat celery.pid` Real- 0me 15

16 To launch Celery Flower monitoring tool, type: Real-time celery flower 5672// hqp://localhost:5555 Real- 0me 16

17 Real-time From network directory start the AMON server: twistd - l server.log - - pidfile server.pid serverpost - - port iface localhost Real- 0me 17

18 To simulate a given observatory run the AMON client from / amonpy/ops/network (local op0on without TLS authoriza0on) AMON client twistd- lclient.log pidfileclient.pid clientpost- - hostport hqp:// :8080 epath / path_to_client_ev where path_to_client_ev is the directory name where your test VOEvents live. The client is setup to send a new event to the server every 10 seconds. The 0me can be changed by edi0ng amon_client post_pluggin.py script in ops/network/twisted/pluggin directory and changing number 10.0 in the following line to the desired number of seconds: loop service = TimerService(10.0, check for files, op0ons [ hostport ], op0ons[ epath ]) Twisted client 18

19 Accessing AMON PennState (TLS cer0ficate is needed: client.crt and client.key) Create a cer0ficate request: Real-time yourcomputer> openssl genrsa - out client.key 1024 yourcomputer> openssl req - new - key client.key - out client.csr end client.csr to AMON, get client.crt from AMON and then: twistd - l client.log - - pidfile client.pid clientpostssl - - hostport hqps://amonaddress - - epath /path_to_client_events - - cfile client.crt - - kfile client.key Real- 0me 19