Technical Note: On The Usage and Development of the AWAKE Web Server and Web Applications Dillon Berger August 10, 2017 Introduction The purpose of this technical note is to give a brief explanation of the AWAKE Web Server, the current web applications it serves, and how to edit, maintain, and update the source code. The majority of this paper, however, will be dedicated to the development of the server and its web applications. The AWAKE web server is currently located at http://awake-web-server.cern.ch, and is currently only accessible to those on the CERN network 1. The website is written entirely using Flask, a Python module for creating websites and web applications. Using the AWAKE Web Server Upon navigating to the website, you will see a list of analysis tools (web applications) to choose from. As shown in Figure 1, there are currently two web applications to choose from: (1) Event Viewer, and (2) Event Viewer (raw). Figure 1: Image of the web app selection screen on the home page of the AWAKE Web Server website Thank you to the NSF for funding my research here at CERN. Thank you to my research advisor, Spencer Gessner, for helping me with the Event Builder database, and for his invaluable criticisms in the development of the AWAKE Web Server web applications. Thank you to Alexey Petrenko for his input and collaboration in the development of Event Viewer. 1 A future goal is to make this website accessible to users outside of CERN who are members of the EOS e-group, through CERN s SSO (single sign-on) service. 1
The intention of both of these web applications is to provide a platform for conveniently viewing certain Event Builder data fields which meet search criteria specified by the user. After choosing which web application you want to use, you will arrive at the data selection screen. I will not create a figure of this page, as it is subject to change relatively soon. Though, the function of this page is straight forward. The data selection screen is where choose the specifications on the data you which to view (e.g. start date, end date, Beam on/off, etc.). After specifying which data to view, you will be directed to a webpage with a list of events matching your specifications, as shown in Figure 2. Figure 2: List of events that match the user s data specifications After clicking on a link, you will be directed to one of two pages, depending on which web application was chosen on the home screen (Figure 1). Event Viewer If you chose Event Viewer on the home screen, then you will be directed to the screen shown in Figure 3, upon clicking a link on the previous page (Figure 2). On this screen, there are four images: XMPP Streak Image, BTV Streak Image, and the Upstream and Downstream Halo Cameras. If you hover over these images you will see a home and a pan button appear to the bottom left of the plots. If you click the pan button, you can hover over any of the images and pan in any direction, as well as zoom in and out of the image using the scroll wheel on your mouse. Clicking the home button resets the images to their default ratios and alignmetns. Directly under the images are various text boxes to input vmin and vmax values for each image. These options adjust the color scale of the images. To update the color scale of a desired image, type the desired values of vmin and vmax in the textbox and click Send. If a text box is left blank when Send is pressed, then the values corresponding to that text box will not change. To reset the color scale to the default values click Reset. Above the images, you will see a small section that is labeled Event Data. Directly below, you will find various scalar and boolean data for the event, such 2
as upstream and downstream plasma density, bunch rotation status, and beam charge. Directly above Event Data is text that says Event ID: #, and two links that say Next and Previous. The Event ID is itself a link to the HDF5 file for that event, and clicking on it will download the file 2. The Next and Previous buttons take you to the next and previous events. Lastly, to the left you will see a scroll bar, with a search bar located at the top. The scroll bar is populated with links to all the events for that day that meet the search criteria, and are labeled by their 24-hour time (instead of Unix timestamp). Figure 3: Event Viewer web application example Event Viewer (raw) If you chose Event Viewer (raw) on the home screen, then you will be directed to the screen shown in Figure 4, upon clicking a link on the previous page (Figure 2). All of the links, text boxes, and buttons on this page do exactly the same things as those described in the Event Viewer section. This is just more of a lightweight Event Viewer, if you will, without the scroll and search bars, and the navigation bar. 2 This is the exact file as the one in the Event Builder database. 3
Figure 4: Event Viewer (raw) web application example Development of AWAKE Web Server Web Applications In this section, we will discuss how to edit, update, and develop the code for Event Viewer. 3 Basic Layout of App In order to access the code, you will either need to log on to the AwakeWeb- Master account 4, or download the repository off Gitlab. 5 Once you have downloaded the repository, please put all of the contents it in a directory named AwakeApp. 3 The procedure is identical for Event Viewer (raw) 4 If you desire to access the AwakeWebMaster account directly, please contact Spencer Gessner. 5 Note that if you download the repo off Gitlab and are not on AwakeWebMaster, you will only be able to run the app locally in development mode (see below). That is, deployment and version updating can only be done from AwakeWebMaster. 4
Once you have the contents in AwakeApp, you will notice that it contains many files and subdirectories. Below, is a brief list describing the function of each of them. awake app.py: This is actually what we refer to as the app. It is the python file where the actual instance (named app) of the Flask application exists. In this file, is the entire structure of the website and is where all of its contents are called and rendered. templates: This is a directory where all of the HTML templates are stored. Every HTML file that you want to render on the web site must be located here. The function render template() searches here by default. static: This is the directory where all of the static.js,.css, etc. files are stored. The function app.send static file() searches here by default. awakeappmodules: For development purposes, this directory was created to contain all of the functions called in the Event Viewer and Event Viewer (raw) applications. timestamp lists: This is a directory containing text files that list the timestamps of all of the data points for each day that data exists. 6 wsgi.py: This is the entry point file, that is necessary to run our application using the uwsgi python interpreter (and hence to serve it using nginx). awakenv 7 : This is the directory to the AwakeWebMaster virtual (python) environment, which is activated by the awakeapp.service configuration file 8. To activate this environment from the AwakeApp directory, execute awakenv/bin/activate in the terminal. uwsgi.ini: This is the configuration file for the uwsgi interpreter. One can run the code with uwsgi, but for development purposes there is no obvious advantage. It is needed to tell nginx what to serve. For the app to run locally (see Version Updating The Web Applications section), you only need the first 5 on this list, as well as all of the dependencies given in requirements.txt installed on your virtual environment (or whatever environment you want to run python with). Moreover, one does not even need any contents in the static and timestamp lists directories; they just have to exist at the same level as templates. 6 This populates automatically via the application, when new data is found. 7 If you downloaded the repo off Gitlab and are not on AwakeWebMaster, you will need to delete this virtual environment and create a new one by running virtualenv --python=/path/to/python awakenv in your AwakeApp directory. 8 See Configuring The AwakeWebMaster Server section 5
Version Updating The Web Applications Now that we have a basic understanding of the layout of the AwakeApp, let us go over how to edit and update the web applications (and hence the website). First, before making any changes in the actual source code on the AwakeWeb- Master account, please be sure that the code runs exactly how you want it to in development mode. Briefly, here is how to test your code in development mode. Running In Development Mode First, if you are using a virtual environment, activate the virtual environment that will run flask. So, if you are on AwakeWebMaster, this would be awakenv/bin/activate. If you are not on AwakeWebMaster, this would be a good time to run pip install -r requirements.txt, to be sure that you have all the necessary dependencies in your environment. Next, to start serving your updated app locally, go to the AwakeApp directory (if you re not already there) and and type export FLASK APP=awake app.py export FLASK DEBUG=1 flask run --host 0.0.0.0 where awake app.py is your updated application. Note that once you execute the first two lines you do not need to again (if you are running the same app). Now, open your favorite web browser and enter <your ip>:5000 into the url. You should see your updated app running, which you can now test. Deploying New Versions of The Web Applications 9 So now you ve updated the awake app.py code, it runs flawlessly in development mode, but now you want to be able to access it from awake-web-server.cern.ch. That is, you want to now deploy a new version of the web app. The first thing you ll obviously want to do is replace the old awake app.py code in AwakeWebMaster s AwakeApp directory. If you haven t already, push your updated awake app.py to Gitlab. If you have access to AwakeWebMaster, you can then update the code on AwakeWebMaster via the usual Gitlab methods. Now that you ve got the updated version of the app in the AwakeWebMaster s AwakeApp directory, you are ready to deploy the newest version of the web app. To do this, go to the terminal and run sudo systemctl restart awakeapp sudo systemctl restart nginx Lastly, navigate to awake-web-server.cern.ch in your favorite web browser, and you should see the newest version of your web app running. 9 Everything in the following section can only be done from AwakeWebMaster. 6
Configuring The AwakeWebMaster Server The server is currently configured on the AwakeWebMaster account. Therefore, there is no need for a tutorial for how to configure a machine to serve our web apps. However, if for some reason there is a need to change machines or accounts, please log on to the AwakeWebMaster account and navigate to /AWAKE server config and follow the steps given in the README.md. In this same directory, you will find copies of all of the files needed to configure the AwakeWebMaster machine. Please note, however, that these are just copies of the configuration files, and still need to be placed in the right directories, which can be found in the tutorial given in the README.md. 7