Testing My Patience. Automated Testing for Python Projects. Adam Lowry Portland Python Users Group

Transcription

1 Testing My Patience Automated Testing for Python Projects Adam Lowry Portland Python Users Group

2 Unit Testing Integration/Functional Testing Web Testing

3 Maurice Koop: Avoid This

4 michelle_p: Unit Testing Purpose: ensure correctness, exercise code Test a small area; one class, or ideally one method Cover possible errors here, because that s harder at higher levels TDD worth trying, but unit testing is useful no matter what

5 unittest import unittest class TestMath(unittest.TestCase): def test_addition(self): """Simple addition must be correct.""" self.assertequals(2 + 2, 4) assert == 6 if name == ' main ': unittest.main() Ugh, assertequals. So non-pep8! assertequals -> failure assert -> error (any exception)

6 Running python test-examples.py --verbose Simple addition must be correct.... ok Ran 1 test in 0.000s OK Collection is a pain. What file to run?

7 nose for sanity s sake adam@rye:~/projects/pdxpython$ nosetests -v Simple addition must be correct.... ok Ran 1 test in seconds OK (?:^ [\\b_\\.-])[tt]est Searches filesystem for tests, auto-adds current dir to pythonpath, runs anything that looks like a test (matches that regexp) Includes very handy plugins (like code coverage!)

8 py.test worth considering py.test -v test-examples.py ============================= test session starts ============================== python: platform darwin -- Python /Users/adam/projects/pdxpython/bin/ python pytest test object 1: /Users/adam/projects/pdxpython/test-examples.py test-examples.py:6: TestMath.test_addition PASS =========================== 1 passed in 0.03 seconds =========================== Cool! Lots of features. Totally non intuitive to me

9 import nose def test_add(): assert == 3 def test_add2(): nose.tools.assert_equals(2 + 2, 3) (pdxpython)adam@rye:~/projects/pdxpython$ nosetests -v nose-examples.py nose-examples.test_add... FAIL nose-examples.test_add2... FAIL ====================================================================== FAIL: nose-examples.test_add Traceback (most recent call last): File "/Users/adam/projects/pdxpython/lib/python2.5/site-packages/nose py2.5.egg/nose/case.py", line 183, in runtest self.test(*self.arg) File "/Users/adam/projects/pdxpython/nose-examples.py", line 5, in test_add assert == 3 AssertionError ====================================================================== FAIL: nose-examples.test_add Traceback (most recent call last): File "/Users/adam/projects/pdxpython/lib/python2.5/site-packages/nose py2.5.egg/nose/case.py", line 183, in runtest self.test(*self.arg) File "/Users/adam/projects/pdxpython/nose-examples.py", line 9, in test_add2 nose.tools.assert_equals(2 + 2, 3) AssertionError: 4!= Ran 2 tests in seconds FAILED (failures=2) Plain functions! More Pythonic, but mixing & matching is good.

10 class TestMath(unittest.TestCase): def setup(self): self.target = 4 def test_addition(self): """Simple addition must be correct.""" self.assertequals(2 + 2, self.target) assert == 6 target = None def set_target(): global target target = def test_add(): nose.tools.assert_equals(2 + 2, target) Do you have fixtures to set up before every function? Storing data as member variables? Go ahead and use a class. Kind of a ideological.

11 Pros: Show API usage in test - Executable documentation Cons: No fixtures (setup/teardown) - No editor support - Can be hard to debug Doctest class Outer(object): """Outside worker >>> outer = Outer() >>> outer.outer_work() True """ def init (self): self.inner = Inner() def outer_work(self): try: return self.inner.inner_work() except Exception: return False (pdxpython)adam@rye:~/projects/pdxpython$ nosetests -v --with-doctest worker.py Doctest: worker.outer... ok Ran 1 test in seconds OK

12 Mocks and Stubs Sakurako Kitsa Andrei Z

13 Mock: action assertion Mox: record replay

14 class Inner(object): def inner_work(self): return True class Outer(object): def init (self): self.inner = Inner() def outer_work(self): try: return self.inner.inner_work() except Exception: return False Goal: test Outer isolated from Inner

15 lambda x: True) def test_outer_w_mock(): outer = worker.outer() nose.tools.assert_equal(outer.outer_work(), lambda x: False) def test_outer_inner_fails_w_mock(): outer = worker.outer() nose.tools.assert_equal(outer.outer_work(), False)

16 mox.stuboutwithmock() def test_outer_w_mox(): moxer = mox.mox() moxer.stuboutwithmock(worker.inner, 'inner_work') worker.inner.inner_work().andreturn(true) moxer.replayall() outer = worker.outer() nose.tools.assert_equal(outer.outer_work(), True) moxer.verifyall() moxer.unsetstubs()

17 class TestOuterWMox(mox.MoxTestBase): def test_outer_fails(self): self.mox.stuboutwithmock(worker.inner, 'inner_work') worker.inner.inner_work().andreturn(false) self.mox.replayall() outer = worker.outer() nose.tools.assert_equal(outer.outer_work(), False) self.mox.verifyall()

18 Testing Exceptions def raiser(obj): raise raiser) def test_outer_exc_w_mock(): outer = worker.outer() nose.tools.assert_equal(outer.outer_work(), False) def test_outer_exception(self): self.mox.stuboutwithmock(worker.inner, 'inner_work') worker.inner.inner_work().andraise(exception) self.mox.replayall() outer = worker.outer() nose.tools.assert_equal(outer.outer_work(), False) self.mox.verifyall()

19 This is your application AriCee: It is a unique and beautiful snowflake. It has special needs.

20 class User(object): """User object, connected to the user_table >>> user = User(password='password') >>> db.session.flush() >>> user # doctest: +ELLIPSIS <User...> >>> user.is_anonymous() False >>> User.query.filter_by(id=user.id).one() is user True >>> user.id 1 """ def init (self, **kwargs): Our goal: use doctests like this on SQLAlchemy-backed objects, touching an in-memory SQLite database, automatic rollback after every test SQLAlchemy had auto-add, scoped sessions, so lots automatic

21 Solution: Custom Nose Plugin from nose.plugins import Plugin class RollbackPlugin(Plugin): """Rollback transaction after each test""" name = 'rollback' def beforetest(self, test): from myapp import test_utils test_utils.set_up_app() def aftertest(self, test): from myapp import db db.session.rollback() db.session.close()

22 setup.py try: import ez_setup ez_setup.use_setuptools() except ImportError: pass from setuptools import setup setup( name='rollbackplugin', description='nose plugin to rollback transaction after each test', py_modules=['rollback'], version='0.1', zip_safe=false, entry_points={ 'nose.plugins': [ 'rollback = rollback:rollbackplugin' ] } ) nosetests -v --with-doctest --with-rollback Nose uses setuptools entry points for plugins, so you ll need to make a package Nose config handling isn t the best, so I like to put default plugins in test running script (Paver)

23 Integration/Functional Testing fotoopa: When you put the pieces together, it can get a little complex. Especially setup & teardown - need to clean DB, file system? - Keep it totally separate from normal workspace so you don t kill your personal test data

24 # in tests/ init.py server_proc = None def setup_package(): global server_proc subprocess.call("dropdb myapp-test") subprocess.call("createdb myapp-test") server_proc = subprocess.popen("myapp/manage.py runserver --noreload") def teardown_package(): server_proc.kill() Use nose package fixtures for setup/teardown easy to read, can still use nose to handle most everything

25 Web Testing I mostly work on web apps. Use functional testing, set up like before. But how to Interact?

26 Twill go $home code 200 find "Don't have an account yet?" # login screen fv 1 username $username fv 1 password $password submit code 200 notfind 'Sorry' from twill.commands import * go(home) code(200) find("don't have an account yet?") # login screen fv(1, "username", username) fv(1, "password", password) submit() code(200) notfind('sorry') Simple language or python functions no javascript, but very fast speaks http directly, but has a wrapper for wsgi to speak to an app directly

27 Twill Plugins def enter_password( , password): """Enters and password in login form.""" import twill.commands as cmds cmds.fv(' ', ' ', ) cmds.fv(' ', 'password', password) cmds.fv(' ', 'has_password', 'yes') cmds.submit() cmds.code(200) twill.commands.extend_with('myapp.twill_extensions') Useful way to reduce duplication in tests Remember: tests are code! You have to refactor them, as well

28 Javascript? Windmill Avoid Selenium if you haven t already invested time. Great project, difficult to debug

29 References nose: py.test: Mock: Mox: Twill: Windmill:

30 Thanks!