Connexion Sqlalchemy Utils Documentation

Transcription

1 Connexion Sqlalchemy Utils Documentation Release Michael Housh Apr 17, 2017

2

3 Contents 1 Connexion Sqlalchemy Utils Features Running example api in Docker Credits Installation Stable release From sources Usage Connexion Sqlalchemy Utils Example API BaseMixinABC BaseMixin Decorators CRUD Contributing Types of Contributions Get Started! Pull Request Guidelines Tips Credits Development Lead Contributors History ( ) Indices and tables 27 i

4 ii

5 Contents: Contents 1

6 2 Contents

7 CHAPTER 1 Connexion Sqlalchemy Utils Sqlalchemy, Postgres, Connexion utility Documentation: Features Helps create REST api s quickly with Connexion, Sqlalchemy, and Postgresql Running example api in Docker By cloning the repo: git clone cd./connexion_sql_utils docker-compose up Without cloning the repo: docker pull mhoush/connexion_sql_utils docker pull postgres/alpine docker run -d --name some_postgres \ -e POSTGRES_PASSWORD=postgres \ postgres:alpine docker run --rm -it --link some_postgres:postgres \ -e DB_HOST=postgres \ -e DB_PASSWORD=postgres \ -p "8080:8080" \ mhoush/connexion_sql_utils Check out the example api at 3

8 Credits This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template. 4 Chapter 1. Connexion Sqlalchemy Utils

9 CHAPTER 2 Installation Stable release To install Connexion Sqlalchemy Utils, run this command in your terminal: $ pip install connexion_sql_utils This is the preferred method to install Connexion Sqlalchemy Utils, as it will always install the most recent stable release. If you don t have pip installed, this Python installation guide can guide you through the process. From sources The sources for Connexion Sqlalchemy Utils can be downloaded from the Github repo. You can either clone the public repository: $ git clone git://github.com/m-housh/connexion_sql_utils Or download the tarball: $ curl -OL Once you have a copy of the source, you can install it with: $ python setup.py install 5

10 6 Chapter 2. Installation

11 CHAPTER 3 Usage To use Connexion Sqlalchemy Utils in a project: import connexion_sql_utils This package is meant to be used in conjunction with Connexion which utilizes an API first approach to building REST API s for micro-services. See also: This package has an Sqlalchemy Mixin used with Postgresql to create a declarative base, which can then be used to declare your database models. This package also has a set of utility functions that when combined with functools.partial can be used to quickly create the routes for the api. Connexion Sqlalchemy Utils Example By cloning the repo: git clone cd./connexion_sql_utils docker-compose up Without cloning the repo: docker pull mhoush/connexion_sql_utils docker pull postgres/alpine docker run -d --name some_postgres \ -e POSTGRES_PASSWORD=postgres \ postgres:alpine docker run --rm -it --link some_postgres:postgres \ -e DB_HOST=postgres \ -e DB_PASSWORD=postgres \ -p "8080:8080" \ mhoush/connexion_sql_utils 7

12 Check out the example api at app.py: #!/usr/bin/env python import os from functools import partial import logging import connexion from sqlalchemy import Column, String, Numeric, create_engine from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import scoped_session, sessionmaker from connexion_sql_utils import BaseMixin, to_json, event_func, dump_method from connexion_sql_utils import crud # Most of this would typically be in a different module, but since # this is just an example, I'm sticking it all into this module. DB_USER = os.environ.get('db_user', 'postgres') DB_PASSWORD = os.environ.get('db_password', 'postgres') DB_HOST = os.environ.get('db_host', 'localhost') DB_PORT = os.environ.get('db_port', 5432) DB_NAME = os.environ.get('db_name', 'postgres') DB_URI = 'postgres+psycopg2://{user}:{password}@{host}:{port}/{db}'.format( user=db_user, password=db_password, host=db_host, port=db_port, db=db_name ) engine = create_engine(db_uri) Session = scoped_session( sessionmaker(autocommit=false, autoflush=false, bind=engine, expire_on_commit=false) ) # The only method required to complete the mixin is to add a staticmethod # that should return a session. This is used in the queries. class MyBase(BaseMixin): # give the models an access to a def session_maker(): return Session() # By attaching the ``session_maker`` method to the class we now create a # ``declarative_base`` to be used. The ``BaseMixin`` class declares an # ``id`` column, that is ``postgresql.uuid``. It also has an declared attr # for the tablename. If you would to override these, they can be declared 8 Chapter 3. Usage

13 # when create your database model DbModel = declarative_base(cls=mybase) class Foo(DbModel): bar = Column(String(40), nullable=false) baz = Column(Numeric, nullable=true) # a method to be called to help in the conversion to def convert_decimal(self, val): if val is not None: logging.debug('converting baz...') return float(val) return val # add a custom value when dumping to def add_bang(self, vals): logging.debug('adding bang') vals['bang'] = 'boom' return vals # attach an event listener to ensure ``bar`` is only saved # as a lower 'before_update') def lower_baz(mapper, connection, target): target.bar = str(target.bar).lower() # CRUD methods used in ``opertionid`` field of ``swagger.yml`` # connexion needs named parameters in it's operitionid field, so you must # declare them in the partial in order to work correctly. get_foo = partial(crud.get, Foo, limit=none, bar=none) post_foo = partial(crud.post, Foo, foo=none) get_foo_id = partial(crud.get_id, Foo, foo_id=none) put_foo = partial(crud.put, Foo, foo_id=none, foo_data=none) delete_foo = partial(crud.delete, Foo, foo_id=none) app = connexion.app( name ) app.add_api('swagger.yml') if name == ' main ': port = os.environ.get('app_port', 8080) DbModel.metadata.create_all(bind=engine) app.run(debug=true, port=int(port)) swagger.yml: swagger: '2.0' info: title: Example Foo api version: '0.1' consumes: 3.1. Connexion Sqlalchemy Utils Example 9

14 - application/json produces: - application/json paths: /foo: get: tags: [Foo] operationid: app.get_foo summary: Get all the foo parameters: - name: limit in: query type: integer minimum: 0 default: name: bar in: query type: string pattern: "^[a-za-z0-9-_]*$" responses: 200: description: Return the foo's schema: type: array items: $ref: '#/definitions/foo' post: tags: [Foo] operationid: app.post_foo summary: Create a new foo. parameters: - name: foo in: body schema: $ref: '#/definitions/foo' responses: 201: description: Foo created schema: $ref: '#/definitions/foo' 400: description: Failed to save foo /foo/{foo_id}: get: tags: [Foo] operationid: app.get_foo_id summary: Get a single foo parameters: - $ref: '#/parameters/foo_id' responses: 200: description: Return the foo schema: $ref: '#/definitions/foo' 404: description: Foo does not exist put: 10 Chapter 3. Usage

15 tags: [Foo] operationid: app.put_foo summary: Update a foo parameters: - $ref: '#/parameters/foo_id' - name: foo_data in: body schema: $ref: '#/definitions/foo' responses: 200: description: Updated foo schema: $ref: '#/definitions/foo' 400: description: Failed due to invalid data 404: description: Foo does not exist delete: tags: [Foo] operationid: app.delete_foo summary: Delete a foo parameters: - $ref: '#/parameters/foo_id' responses: 204: description: Foo was deleted 404: description: Foo does not exist parameters: foo_id: name: foo_id description: Foo's unique identifier type: string in: path required: true pattern: "^[a-za-z0-9-]+$" definitions: Foo: type: object required: - bar properties: id: type: string description: Unique identifier example: 44a288c1-829c-410d-9a6e-1fce1beb62d4 readonly: true bar: type: string description: The bar attribute maxlength: 40 example: 'some bar' baz: type: number description: The number of baz 3.1. Connexion Sqlalchemy Utils Example 11

16 minimum: 1 example: 10 bang: type: string description: A custom read-only variable added to every Foo readonly: true 12 Chapter 3. Usage

17 CHAPTER 4 API The public interface to connexion_sql_utils. BaseMixinABC base_mixin_abc.py This module contains an abc class that can be used to check that the correct interface is defined to use the crud methods. Nothing actually inherits from this class as it breaks when making an sqlalchemy.ext.declarative:declarative_base, but it is helpful for issubclass and isinstance checks. When inheriting from this package s BaseMixin then one only needs to implement a session_maker method. class connexion_sql_utils.basemixinabc Used to test validity of a mixin. The mixin can not directly inherit from this abc class because of inheritance conflicts with sqlalchemy s declarative base class, however if a mixin declares all the required methods, then it will pass an isinstance or an issubclass check. The mixin in this package declares all of the methods except the session_maker method which should be implemented once you create a sqlalchemy.orm.sessionmaker Example: engine = create_engine(db_uri) session = scoped_session(sessionmaker(bind=engine)) class def session_maker(): return session() DbModel = declarative_base(cls=mybase) assert issubclass(dbmodel, BaseMixinABC) # True Then all of your sqlalchemy models could inherit from the DbModel. delete() Delete an instance from the database. 13

18 dump() Return a json representation of the instance. This is also used as the str() representation of an instance. classmethod get_id(id) Query the database for a single item by it s unique id. classmethod query_by(**kwargs) Query the database model with the given criteria. This is used as you would use filter_by on an sqlalchemy query. And should always return a list of items. save() Save an instance to the database. static session_maker() Return an sqlalchemy.orm.session to be used in the other methods. classmethod session_scope() A context manager for a session, should yield a sqlalchemy.orm.session update(**kwargs) Update an instance s attributes and save to the database. BaseMixin The following can be found in the sqlmixins.py module. class connexion_sql_utils.basemixin Base sqlalchemy mixin. Adds id column as a postgresql.uuid column, and will create the uuid before saving to the database. A user must define a session_maker on the mixin, to complete it as a classmethod or a staticmethod. All query methods, automatically create a session from the session_maker method that should be declared on a sub-class. Any method that creates, updates, or deletes automatically adds and commits the changes. static create_id(mapper, connection, target) Automatically creates a UUID before inserting a new item. delete(session=none) Delete an instance from the database. Parameters session An optional sqlalchemy session, if one is not passed a session will be created for the query. dump(_dict=none) str Return a json serialized string or a dict representation of the instance. Any methods that are wrapped with to_json decorator will be called on the values before returning the json string. Parameters _dict If True return a dict instead of a json string, or the class attribute dump_dict is true on a sub-class. classmethod get_id(id, session=none) Get by id. Parameters id The unique identifier for the class. 14 Chapter 4. API

19 session An optional sqlalchemy session, if one is not passed a session will be created for the query. This is would be like: >>> session.query(mydbmodel).filter_by(id=1234).first() classmethod query_by(session=none, **kwargs) Return a query statement for the class. Parameters session An optional sqlalchemy session, if one is not passed a session will be created for the query. kwargs kwargs passed into the query to filter results. This would be simalar to: >>> session.query(mydbmodel).filter_by(id=1234) save(session=none) Save an instance to the database. Parameters session An optional sqlalchemy session, if one is not passed a session will be created for the query. classmethod session_scope() A context manager for a session. Which creates a session from the import session_maker`() method that should be declared by sub-class. And is used in the database methods for a class/instance. The session will automatically try to commit any changes, rolling back on any errors, and finally closing the session. update(session=none, **kwargs) Update attributes on an instance. Parameters kwargs The attributes to update on the instance. Any attribute not declared on the class is ignored. session An optional sqlalchemy session, if one is not passed a session will be created for the query. Decorators decorators.py This module holds decorators either used by this package or for use when creating sqlalchemy database models. connexion_sql_utils.event_func(*event_names) Declare a function to listen/register for an event. The wrapped method should have the correct signature for an sqlalchemy.event. And should be declared as a staticmethod on the class that is registering for the event. See also: connexion_sql_utils.basemixin.create_id() method for an example Decorators 15

20 Parameters event_name The event name to register the function for. Example: before_insert connexion_sql_utils.to_json(*keys) Marks a function to be called on key, when converting an instance to json. This allows you to do work on an item to serialize it to json. A good example use is when you use the Numeric type, that returns a Decimal from the database, which is not json serializable, so you must convert it to a string, a float, or an int, before calling json.dumps. Parameters keys The keys/attributes to call the method on when converting. connexion_sql_utils.dump_method(fn) Allow s the ability to create custom methods that are called during an BaseMixin dump method. The method should recieve on argument which is a dict of all the values so far. The method should then return a dict of the current state of the values passed in. This allows manipulation of existing values, or adding values that are not automatically added during the dump to json. connexion_sql_utils.ensure_asset(fn) Ensure s that an asset passes an isinstance or an issubclass check for BaseMixinABC before calling a function. The asset must be the first arg to the function. CRUD crud.py This module contains methods that can be used to query the database. They are designed to be used in conjunction with the BaseMixin or a class that passes a check for the BaseMixinABC methods. These methods do work stand-alone, however they are designed to be used with functools.partial and connexion. Connexion expects the functions for it s api calls to be module level functions. So this allows one to declare an sqlalchemy model that typically derives from the BaseMixin. Then one is able to just create partial s of these utility functions for the actual operations, providing the correct kwargs needed by connexion. The only caveat is that any of the functions that require an id, must have id in their kwarg key. All of the functions in this module will fail if the asset does not pass isinstance or an issubclass check for the BaseMixinABC. A class does not need to directly inherit from BaseMixinABC, it just must declare all the methods of that interface. connexion_sql_utils.get(asset, limit=1, **kwargs) Retrieves assets from the database. Assets are always returned as a list(array) of size limit. Parameters asset The database class to query. This must inherit from Base limit The limit for the return values kwargs Are query parameters to filter the assets by Raises TypeError if the asset does not inherit from Base connexion_sql_utils.get_id(asset, **kwargs) Get an asset by the unique id. The key for the id must have id in the name in the kwargs. Example: 16 Chapter 4. API

21 get_id(foo, foo_id=1) # works get_id(foo, foo=1) # TypeError connexion_sql_utils.post(asset, **kwargs) Post an asset to the database. Parameters asset The database class to query. This must inherit from Base kwargs This should be of length 1, the key only matters to connexion, the value for the key is used as the kwargs to make an asset instance to save to the database. This allows this to be used with functools.partial. Raises TypeError if the asset does not inherit from Base connexion_sql_utils.put(asset, **kwargs) Update an asset. The kwargs should be of length 2, one of which is an id key (has id in it s name), used to look up the item in the database. The other key should not have id in it s name, and used as the data to update the asset. Parameters asset The database asset. Raises TypeError If could not find a key with id in it s name or could not find a key without id in it s name. connexion_sql_utils.delete(asset, **kwargs) Delete an asset 4.4. CRUD 17

22 18 Chapter 4. API

23 CHAPTER 5 Contributing Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given. You can contribute in many ways: Types of Contributions Report Bugs Report bugs at If you are reporting a bug, please include: Your operating system name and version. Any details about your local setup that might be helpful in troubleshooting. Detailed steps to reproduce the bug. Fix Bugs Look through the GitHub issues for bugs. Anything tagged with bug and help wanted is open to whoever wants to implement it. Implement Features Look through the GitHub issues for features. Anything tagged with enhancement and help wanted is open to whoever wants to implement it. Write Documentation Connexion Sqlalchemy Utils could always use more documentation, whether as part of the official Connexion Sqlalchemy Utils docs, in docstrings, or even on the web in blog posts, articles, and such. 19

24 Submit Feedback The best way to send feedback is to file an issue at If you are proposing a feature: Explain in detail how it would work. Keep the scope as narrow as possible, to make it easier to implement. Remember that this is a volunteer-driven project, and that contributions are welcome :) Get Started! Ready to contribute? Here s how to set up connexion_sql_utils for local development. 1. Fork the connexion_sql_utils repo on GitHub. 2. Clone your fork locally: $ git clone git@github.com:your_name_here/connexion_sql_utils.git 3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development: $ mkvirtualenv connexion_sql_utils $ cd connexion_sql_utils/ $ python setup.py develop 4. Create a branch for local development: $ git checkout -b name-of-your-bugfix-or-feature Now you can make your changes locally. 5. When you re done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox: $ flake8 connexion_sql_utils tests $ python setup.py test or py.test $ tox To get flake8 and tox, just pip install them into your virtualenv. 6. Commit your changes and push your branch to GitHub: $ git add. $ git commit -m "Your detailed description of your changes." $ git push origin name-of-your-bugfix-or-feature 7. Submit a pull request through the GitHub website. Pull Request Guidelines Before you submit a pull request, check that it meets these guidelines: 1. The pull request should include tests. 20 Chapter 5. Contributing

25 2. If the pull request adds functionality, the docs should be updated. Put your new functionality into a function with a docstring, and add the feature to the list in README.rst. 3. The pull request should work for Python 2.6, 2.7, 3.3, 3.4 and 3.5, and for PyPy. Check m-housh/connexion_sql_utils/pull_requests and make sure that the tests pass for all supported Python versions. Tips To run a subset of tests: $ py.test tests.test_connexion_sql_utils 5.4. Tips 21

26 22 Chapter 5. Contributing

27 CHAPTER 6 Credits Development Lead Michael Housh <mhoush@houshhomeenergy.com> Contributors None yet. Why not be the first? 23

28 24 Chapter 6. Credits

29 CHAPTER 7 History ( ) First release on PyPI Added ability to pass a session into BaseMixin database methods. Added dump_method decorator to allow custom methods to be called while creating the json representation Added the ability to dump as dict instead of always being a json string. 25

30 26 Chapter 7. History

31 CHAPTER 8 Indices and tables genindex modindex search 27

32 28 Chapter 8. Indices and tables

33 Index C connexion_sql_utils (module), 13, 15, 16 29