RandoPony Documentation

Transcription

1 RandoPony Documentation Release Doug Latornell Feb 03, 2018

2

3 Contents 1 Data Model Management Database Table Creation Manipulation of Database Model Instances Deployment and Management Webfaction Environment Initial Staging Deployment Working With the Staging Deployment Promoting Staging Deployment to Production Working With the Production Deployment Development Python Versions Development Environment Testing and Coverage Documentation Source Code Reporting Bugs Indices and tables 15 i

4 ii

5 This is the documentation for the RandoPony-tetra project. RandoPony-tetra a re-implementation of RandoPony, the BC Randonneurs Cycling Club online event registration system. It uses: Python 3 Pyramid for the web application framework SQLAlchemy for the database interface layer Bootstrap for responsive layout in the browser The app was initially deployed for the 2013 season. This project is released under a New BSD License. Please see LICENSE.txt. Contents: Contents 1

6 2 Contents

7 CHAPTER 1 Data Model Management The data model objects and underlying persistence in the database of model instances can be managed from the command line via the Pyramid pshell command. This is useful for tasks like: creating database tables for new model objects during development manipulation of model instances for which there is no admin interface; e.g. models. address and models.link bulk deletion of model instances The code examples below assume that the configuration file that pshell is started with includes a section like: [pshell] m = randopony.models session = randopony.models.dbsession t = transaction Launch pshell with: $ cd webapps/randopony_py3/randopony-tetra $../env/bin/pshell production.ini 1.1 Database Table Creation To create tables for new data model objects that don t already exist in the database use: >>> m.meta.base.metadata.create_all() 1.2 Manipulation of Database Model Instances Example: Create a database row for the club webmaster s address. 3

8 >>> = m. address(key='webmaster', ='tom@example.com') >>> with t.manager: >>> session.add( ) Example: Update the club membership sign-up link URL. >>> with t.manager:... link = session.query(m.link).filter_by(key='membership_link').one()... link.url = ' membership' Example: Add a 2nd organizer address to a brevet (pending resolution of randopony-tetra/issues/24). >>> with t.manager:... brevet = session.query(m.brevet).filter_by(region='lm', distance=400).one()... brevet.organizer_ += ', tom@example.com' 4 Chapter 1. Data Model Management

9 CHAPTER 2 Deployment and Management This section describes deployment and management of RandoPony-tetra on the randonneurs.bc.ca domain hosted on Webfaction. Note: As of mid-january 2013 Webfaction does not provide a Pyramid/Python3 environment that is installable from the control panel. While it should be possible to build such an environment, this document describes deployment in a Python 2.7, Pyramid 1.4 environment created from the control panel. The deployment strategy described here is: Deployment of nearly release-ready code from a local development environment to a staging environment Webfaction app that is mounted in a website hosted at a test domain; e.g. randopony.bcrandonneur. webfactional.com. Testing of the staging deployment to the extent necessary to confirm that it is ready for production release. Promotion of the staging environment to production by giving it access to the production database, and mounting it in a website hosted at the production domain; e.g. randopony.randonneurs.bc.ca. Most of the deployment and management activities described in this section have been implemented as Fabric tasks, so, Fabric needs to be installed in the local virtualenv from which you are working, and you need to have ssh key authentication working on Webfaction. 2.1 Webfaction Environment Use the Webfaction control panel to: 1. Create staging and production sub-domains: randopony.bcrandonneur.webfactional.com and randopony.randonneurs.bc.ca 5

10 2.2 Initial Staging Deployment If the app is being deployed to the staging environment for the first time in a newly created Webfaction application directory: 1. Edit fabfile.py to set the module level values appropriately. In particular, the sum of app_name and staging_release should equal the name of the Webfaction application you created in the Webfaction Environment; e.g. env.user = 'bcrandonneur' env.hosts = ['bcrandonneur.webfactional.com'] app_name = 'randopony' staging_release = '2013r1' staging_dir = ( '/home/{0}/webapps/{1}{2}'.format(env.user, app_name, staging_release)) 2. Edit randopony/scripts/initializedb.py to set the addresses and link URLs that will be initialized into the database to values appropriate for staging testing. 3. Prepare the staging environment with: (randopony-tetra)randopony-tetra$ fab init_staging That launches a sequence of Fabric tasks to: Create a staging application, initially just a Pyramid 1.4/Python 2.7 application named to reflect the release that is being prepared; e.g. randopony2013r1. Make note of the port number that is assigned to the application. Create a website with the same name as the application and host it at randopony. bcrandonneur.webfactional.com with the application mounted at / Upload the code via rsync (rsync_code task) Install the RandoPony app and its dependencies in the Webfaction staging_dir and its associated virtualenv directories, and: Delete the template app code directory and config files that were created when the Pyramid app was created from the Webfaction control panel Tighten permissions to owner read-only (400) on the randopony/private_credentials.py file in the install directory Create symlinks to staging.ini and the randopony-tetra/randopony directory in staging_dir so that Webfaction will serve the app, and supervisor and celery will work properly Change the bin/start file to use staging.ini to configure the app (install_app task) Create the RandoPony-staging.sqlite database, and initialize it with link and address records that the app requires (init_staging_db task) 4. Edit staging.ini and production.ini to set the port number in the server:main section to the port that Webfaction provided for the application; e.g. [server:main] use = egg:waitress#main host = port = then upload those changes with the rsync_code task. 6 Chapter 2. Deployment and Management

11 5. Start the app with: (randopony-tetra)randopony-tetra$ fab start_app:release=staging and test things out. Before testing event pre-registrations, start supervisor and thence celery with: (randopony-tetra)randopony-tetra$ fab start_staging_supervisor 2.3 Working With the Staging Deployment Once the Initial Staging Deployment has been completed updates can be pushed from the local development environment to the staging environment with: (randopony-tetra)randopony-tetra$ fab deploy_staging or just: (randopony-tetra)randopony-tetra$ fab since deploy_staging is set as the default task in fabfile.py. deploy_staging launches the following tasks: rsync_code install_app restart_app:release=staging restart_staging_supervisor Other useful Fabric tasks: start_app:release=staging stop_app:release=staging restart_app:release=staging tail_app_log:release=staging start_staging_supervisor stop_staging_supervisor restart_staging_supervisor tail_staging_supervisor_log tail_staging_celery_log See fab -l for the complete list of defined tasks Working With the Staging Deployment 7

12 2.4 Promoting Staging Deployment to Production Warning: When a staging deployment is promoted to production all of the data generated during the staging deployment period is deleted. Specifically, the RandoPony and celery databases, and the supervisord. log, celery.log, pyramid.log files are deleted. 1. Ensure that the module variables staging_release and production_release in fabfile.py are set to appropriate values. If there is no production release, or the data from an existing production release (perhaps from the previous year) is to be preserved, set the value of production_release to None. 2. Promote the staging environment to production with: (randopony-tetra)randopony-tetra$ fab promote_staging_to_production That task does the following: Confirm the release identifiers of the staging deployment that is to be promoted (staging_release), and the production release (if any) that data are to be migrated from (production_release) Stop the staging app (if it is running), via the stop_app:release=staging task Stop the staging instances of supervisor and thence celery, via the stop_staging_supervisor task Ask for the Webfaction account password and use the Webfaction API to get the list of configured websites Delete the RandoPony and celery staging databases Delete the supervisord.log, celery.log, and pyramid.log files Delete the staging.ini symlink Create a symlink to production.ini in the staging_dir Change the bin/start file to use production.ini to configure the app If production_release is None: Create the RandoPony-production.sqlite database, and initialize it with link and address records that the app requires via the initialize_randopony_db srcript, Otherwise: Stop the production app (if it is running), via the stop_app:release=production task Stop the production instances of supervisor and thence celery, via the stop_production_supervisor task Detach the production_domain from the old production app via the Webfaction update_website API call Copy the RandoPony-production.sqlite and celery.sqlite databases from the old production_dir to the staging_dir Attach the production_domain to the staging app via the Webfaction update_website API call Update production_release and production_dir to point to staging_release and staging_dir Start the app, via the start_app:release=production task 8 Chapter 2. Deployment and Management

13 Start the production instances of supervisor and thence celery, via the start_production_supervisor task 2.5 Working With the Production Deployment The Fabric tasks available for working with the production deployment are: restart_app:release=production start_app:release=production stop_app:release=production tail_app_log:release=production restart_production_supervisor start_production_supervisor stop_production_supervisor tail_production_supervisor_log tail_production_celery_log See fab -l for the complete list of defined tasks. To apply code changes, create a new release and deploy it to staging, test the changes, and promote the tested staging deployment to production Working With the Production Deployment 9

14 10 Chapter 2. Deployment and Management

15 CHAPTER 3 Development 3.1 Python Versions RandoPony-tetra is developed and tested with Python Development Environment Development in a conda environment is strongly recommended to isolate randopony-tetra and the packages it depends on from other installed Python packages. Clone the code repository from Bitbucket: $ hg clone Use conda to create a development environment and activate it: $ cd randopony-tetra $ conda env create -f environment.yaml... $ source activate randopony To deactivate the development environment, use: (randopony)$ source deactivate Install RandoPony as a development package: (randopony-tetra)$ pip install --editable. Run the development server: (randopony-tetra)$ pserve --reload development.ini 11

16 3.3 Testing and Coverage The test suite for the randopony-tetra package is in randopony-tetra/tests/. The pytest tool is used for test fixtures and as the test runner for the suite. With your randopony development environment activated, use: (randopony-tetra)$ cd randopony-tetra/ (randopony-tetra)$ py.test to run the test suite. You can monitor what lines of code the test suite exercises using the coverage.py tool with the command: (randopony-tetra)$ cd SalishSeaCmd/ (randopony-tetra)$ coverage run -m py.test and generate a test coverage report with: (randopony-tetra)$ coverage report to produce a plain text report, or (randopony-tetra)$ coverage html to produce an HTML report that you can view in your browser by opening randopony-tetra/htmlcov/ index.html. randopony-tetra/.coveragerc contains settings which enable branch coverage, limit coverage analysis to the randopony package and its sub-packages, and include the line number of code without test coverage in the report. 3.4 Documentation The documentation is written using restructuredtext markup, and built with Sphinx. Build the docs with: (randopony-tetra)$ (cd docs && make html) The results are browsable in randopony-tetra/docs/_build/html/. Online, the docs are hosted at where they are automatically updated whenever changes are pushed to the Bitbucket repository. 3.5 Source Code The source repository is hosted on Bitbucket: 12 Chapter 3. Development

17 3.6 Reporting Bugs Please report bugs via the Bitbucket issue tracker: Reporting Bugs 13

18 14 Chapter 3. Development

19 CHAPTER 4 Indices and tables genindex modindex search 15