Commits

Miki Tebeka committed afe9c86

python testing (nose)

Comments (0)

Files changed (34)

python-testing/Makefile

+txt = python-testing.txt
+html = $(subst .txt,.html,$(txt))
+
+
+all: $(html)
+
+$(html): $(txt) tests/*.py ${PWD}/style.css
+	asciidoc -a stylesheet=${PWD}/style.css $<
+
+clean:
+	rm -f $(html)
+
+fresh: clean all
+
+.PHONY: all clean fresh

python-testing/images/icons/callouts/1.png

Added
New image

python-testing/images/icons/callouts/10.png

Added
New image

python-testing/images/icons/callouts/11.png

Added
New image

python-testing/images/icons/callouts/12.png

Added
New image

python-testing/images/icons/callouts/13.png

Added
New image

python-testing/images/icons/callouts/14.png

Added
New image

python-testing/images/icons/callouts/15.png

Added
New image

python-testing/images/icons/callouts/2.png

Added
New image

python-testing/images/icons/callouts/3.png

Added
New image

python-testing/images/icons/callouts/4.png

Added
New image

python-testing/images/icons/callouts/5.png

Added
New image

python-testing/images/icons/callouts/6.png

Added
New image

python-testing/images/icons/callouts/7.png

Added
New image

python-testing/images/icons/callouts/8.png

Added
New image

python-testing/images/icons/callouts/9.png

Added
New image

python-testing/images/icons/caution.png

Added
New image

python-testing/images/icons/example.png

Added
New image

python-testing/images/icons/home.png

Added
New image

python-testing/images/icons/important.png

Added
New image

python-testing/images/icons/next.png

Added
New image

python-testing/images/icons/note.png

Added
New image

python-testing/images/icons/prev.png

Added
New image

python-testing/images/icons/tip.png

Added
New image

python-testing/images/icons/up.png

Added
New image

python-testing/images/icons/warning.png

Added
New image

python-testing/python-testing.txt

+Testing in Python (nose)
+========================
+:author: Miki Tebeka <miki.tebeka@gmail.com>
+:backend: slidy
+:max-width: 45em
+:data-uri:
+:icons:
+
+
+nose
+----
+* There are many other testing frameworks, but we'll use
+  link:https://nose.readthedocs.org/en/latest/testing.html[nose]
+* Nose is a
+  link:https://nose.readthedocs.org/en/latest/finding_tests.html[discovery
+  based] system.
+** Any Python file under root directory that starts with `test` will be included
+** Any function that starts with `test` will be ran
+* Use `assert` to make test fail
+** And write a *good* error message
+** You can use `print` since nose will hide stdout of passed tests
+* Use `setup` and `teardown` for fixtures
+** For functions specific fixture there's the
+    link:https://nose.readthedocs.org/en/latest/writing_tests.html#test-functions[with_setup]
+    decorator
+* There are
+  link:https://nose.readthedocs.org/en/latest/plugins/builtin.html[many plugins], 
+  we'll talk about some of them later
+
+Example
+-------
+[source,python,numbered]
+---------------------------------------------------
+include::tests/test_simple.py[]
+---------------------------------------------------
+
+<1> By default, nose hides `stdout`
+
+Running
+-------
+[source,bash,numbered]
+---------------------------------------------------
+    $ nosetests tests/test_simple.py
+    Let the games begin!
+    .F
+    Are we there yet?
+
+    ======================================================================
+    FAIL: simple.test_bad
+    ----------------------------------------------------------------------
+    Traceback (most recent call last):
+      File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
+        self.test(*self.arg)
+      File "/Users/mikitebeka/work/talks/python-testing/tests/test_simple.py", line 6, in test_bad
+        assert 1 == 2, 'please reinstall universe and reboot'
+    AssertionError: please reinstall universe and reboot
+
+    ----------------------------------------------------------------------
+    Ran 2 tests in 0.001s
+
+    FAILED (failures=1)
+---------------------------------------------------
+
+We're Done
+----------
+That's about all you need to know. Now go and write some tests...
+
+image:question.png[]
+
+
+More Advanced Topics
+--------------------
+OK, since you asked so nicely :)
+
+
+Going Verbose
+-------------
+* Use the `-v` switch for verbose
+** `nosetests` will print the docstring
+* Use `-d` for detailed errors
+
+The Test
+--------
+[source,python,numbered]
+---------------------------------------------------
+include::tests/test_verbose.py[]
+---------------------------------------------------
+
+Regular Run
+-----------
+[source,bash,numbered]
+---------------------------------------------------
+    $ nosetests tests/test_verbose.py
+    F
+    ======================================================================
+    FAIL: Hello, My name is Inigo Montoya...
+    ----------------------------------------------------------------------
+    Traceback (most recent call last):
+      File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
+        self.test(*self.arg)
+      File "/Users/mikitebeka/work/talks/python-testing/tests/test_verbose.py", line 7, in test_verbose
+        assert sword == 'sharp', 'sharpen sword please'
+    AssertionError: sharpen sword please
+    -------------------- >> begin captured stdout << --------------------- <1>
+    How many finger do you have?
+
+    --------------------- >> end captured stdout << ----------------------
+
+    ----------------------------------------------------------------------
+    Ran 1 test in 0.001s
+
+    FAILED (failures=1)
+---------------------------------------------------
+<1> `stdout` of failed test
+
+Adding -v
+---------
+[source,bash,numbered]
+---------------------------------------------------
+$ nosetests -v tests/test_verbose.py
+    Hello, My name is Inigo Montoya... ... FAIL <1>
+
+    ======================================================================
+    FAIL: Hello, My name is Inigo Montoya...
+    ----------------------------------------------------------------------
+    Traceback (most recent call last):
+      File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
+        self.test(*self.arg)
+      File "/Users/mikitebeka/work/talks/python-testing/tests/test_verbose.py", line 7, in test_verbose
+        assert sword == 'sharp', 'sharpen sword please'
+    AssertionError: sharpen sword please
+    -------------------- >> begin captured stdout << ---------------------
+    How many finger do you have?
+
+    --------------------- >> end captured stdout << ----------------------
+
+    ----------------------------------------------------------------------
+    Ran 1 test in 0.001s
+
+    FAILED (failures=1)
+---------------------------------------------------
+<1> Test docstring and full name
+
+Adding -d
+---------
+[source,bash,numbered]
+---------------------------------------------------
+    $ nosetests -vd tests/test_verbose.py
+    Hello, My name is Inigo Montoya... ... FAIL
+
+    ======================================================================
+    FAIL: Hello, My name is Inigo Montoya...
+    ----------------------------------------------------------------------
+    Traceback (most recent call last):
+      File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
+        self.test(*self.arg)
+      File "/Users/mikitebeka/work/talks/python-testing/tests/test_verbose.py", line 7, in test_verbose
+        assert sword == 'sharp', 'sharpen sword please'
+    AssertionError: sharpen sword please
+        '''Hello, My name is Inigo Montoya...'''
+        print('How many finger do you have?')
+    >>  assert 'dull' == 'sharp', 'sharpen sword please' <1>
+    ...
+
+---------------------------------------------------
+<1> See actual assert
+
+Bad `-d`
+--------
+[source,python,numbered]
+---------------------------------------------------
+include::tests/test_verbose2.py[]
+---------------------------------------------------
+
+
+Output
+------
+
+[source,bash,numbered]
+---------------------------------------------------
+    $ nosetests -vd tests/test_verbose2.py
+    test_verbose2.test_range ... FAIL
+
+    ======================================================================
+    FAIL: test_verbose2.test_range
+    ----------------------------------------------------------------------
+    Traceback (most recent call last):
+      File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
+        self.test(*self.arg)
+      File "/Users/mikitebeka/work/talks/python-testing/tests/test_verbose2.py", line 2, in test_range
+        assert len(range(10)) == 11, 'bad range'
+    AssertionError: bad range
+    >>  assert len(range(10)) == 11, 'bad range' <1>
+
+
+    ----------------------------------------------------------------------
+    Ran 1 test in 0.021s
+
+    FAILED (failures=1)
+---------------------------------------------------
+<1> Can't see the actual length
+
+Good `-d`
+---------
+[source,python,numbered]
+---------------------------------------------------
+include::tests/test_verbose3.py[]
+---------------------------------------------------
+
+
+Output
+------
+
+[source,bash,numbered]
+---------------------------------------------------
+    $ nosetests -vd tests/test_verbose3.py
+    test_verbose3.test_range ... FAIL
+
+    ======================================================================
+    FAIL: test_verbose3.test_range
+    ----------------------------------------------------------------------
+    Traceback (most recent call last):
+      File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
+        self.test(*self.arg)
+      File "/Users/mikitebeka/work/talks/python-testing/tests/test_verbose3.py", line 5, in test_range
+        assert size == expected, 'bad range' <1>
+    AssertionError: bad range
+    >>  assert 10 == 11, 'bad range' <2>
+
+
+    ----------------------------------------------------------------------
+    Ran 1 test in 0.021s
+
+    FAILED (failures=1)
+---------------------------------------------------
+<1> Source line
+<2> Values
+
+
+Selecting Tests
+---------------
+* By default nose will run everything it collects
+* If you specify a file, only tests in this file will run
+* You can specify a specific function with `nosetests test_simple.py:test_ok`
+* There's also the
+  link:https://nose.readthedocs.org/en/latest/plugins/attrib.html[attrib] plugin
+  for test groups
+* raise `SkipTest` to skip a test/module "manually"
+** Usually check environment variables, but ...
+
+
+Test Generators
+---------------
+Sometimes we want to run the same test with different input, nose has
+link:https://nose.readthedocs.org/en/latest/writing_tests.html#test-generators[test
+generators] for this.
+
+[source,python,numbered]
+---------------------------------------------------
+include::tests/test_gen.py[]
+---------------------------------------------------
+
+<1> *Don't* start the name with `test`, otherwise nose will try running it
+
+Output
+------
+[source,bash,numbered]
+---------------------------------------------------
+    $ nosetests -v tests/test_gen.py
+    test_gen.test_add(0, 1) ... ok <1>
+    test_gen.test_add(1, 2) ... ok
+    test_gen.test_add(2, 3) ... ok
+
+    ----------------------------------------------------------------------
+    Ran 3 tests in 0.001s
+
+    OK
+---------------------------------------------------
+<1> Name with parameters
+
+
+Jenkins
+-------
+* Use `--with-xunit` to get detailed reports in Jenkins
+** Default output is `nosetests.xml` can be changed with `--xunit-file`
+* Don't forget to enable the `xUnit` plugin in Jenkins
+* Have *one* script/makefile rule to run the tests
+** As opposed to complex Jenkins run script
+** If you change the way you run the tests, you'll do it in the code
+
+Mocking
+-------
+* The First Rule Of Mocking: Don't do it. The Second Rule of Mocking (for
+  experts only!): Don't do it yet!
+** Stolen from the "Rules of Optimization"
+* Every time you mock - you cheat, and you should see it as a "red flag"
+** You also need to "play catch" with the code you're mocking
+* Since Python is dynamic - you can mock with a regular function/class/module
+  that has the minimal signature used by the test
+** Don't forget to update the mock when the object being mocked changes
+* To mock services, span a new process on `setup` and kill it in `teardown`
+** You can have a package level `setup` and `teardown` at package `__init__.py`
+
+Debugging
+---------
+* Most of modern IDE's (like link:http://www.aptana.com/[Aptana]) support
+  breakpoints in tests
+* For dinosaurs like me, you can add a manual breakpoint in the test and re-run
+  it
+** `import pdb; pdb.set_trace()`
+** `from IPython.core.debugger import Pdb; Pdb().set_trace()` (if you have
+   IPython)
+* Run `nosetests -s`, otherwise this won't work
+** `-s, --nocapture       Don't capture stdout`
+
+Misc
+----
+* It's usually a good idea to run pep8 and pylint/pyflakes before the tests and
+  *fail* if they don't pass
+** Nobody reads the report unless they fail the tests
+* `find tests -name \'*.py[co]\' -exec rm {} \;` before testing is a good habit
+** Covers the case you renamed a module, forgot to update the import but the
+    `.pyc` is still there. The test will succeed even though it should fail
+* Cleanup on test start, not end
+** This way when it fails, you'll have all the data there
+* Write code with testing in mind
+** Functional code (no side effects) is the easiest to test
+** link:http://en.wikipedia.org/wiki/Test-driven_development[TDD] anyone?
+* Make sure you're tests don't make it to production
+* Don't name your test directory `test`, there is a builtin module with that
+  name
+
+References
+----------
+* link:https://nose.readthedocs.org/en/latest/writing_tests.html[nose
+  documentation]
+* link:https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin[Jenkins xUnit
+  plugin]
+* link:http://pythonwise.blogspot.com/2012/10/cleanup-after-your-tests-but-be-lazy.html[Cleanup
+  After Your Tests] blog post
+
+
+This presentation was made with
+link:http://www.methods.co.nz/asciidoc/[asciidoc] using the
+link:http://www.w3.org/Talks/Tools/Slidy2/Overview.html[slidy] backend and
+link:http://pygments.org/[Pygments] syntax highlighter.
+
+Thank You
+---------
+image:question.png[]
+
+// vim: ft=asciidoc spell

python-testing/question.png

Added
New image

python-testing/style.css

+/* Vertical lines between line numbers and code */
+div.linenodiv {
+    border-right:1px solid gray;
+    padding-right:2px;
+    margin-right:2px;
+}

python-testing/tests/test_gen.py

+from operator import add
+
+def check_add(x, y):  # <1>
+    assert add(x, y) == x + y
+
+def test_add():
+    for i in range(3):
+        yield check_add, i, i + 1
+

python-testing/tests/test_simple.py

+from sys import stderr
+
+
+def setup():
+    stderr.write('Let the games begin!\n')  # <1>
+
+
+def teardown():
+    stderr.write('\nAre we there yet?\n')
+
+
+def test_ok():
+    assert 1 == 1, 'okey dokey'
+
+
+def test_bad():
+    assert 1 == 2, 'please reinstall universe and reboot'

python-testing/tests/test_verbose.py

+sword = 'dull'
+
+
+def test_verbose():
+    '''Hello, My name is Inigo Montoya...'''
+    print('How many finger do you have?')
+    assert sword == 'sharp', 'sharpen sword please'

python-testing/tests/test_verbose2.py

+def test_range():
+    assert len(range(10)) == 11, 'bad range'

python-testing/tests/test_verbose3.py

+def test_range():
+    items = range(10)
+    size = len(items)
+    expected = 11
+
+    assert size == expected, 'bad range'