DjangoCon 2011 Notes / docs / testing_the_developer_strikes_back.rst

Testing: The Developer Strikes Back

  1. Testing is hard to do right.
    • Delegate test responsibilities correctly.
    • Won't (always) get it right on the first try.
  2. Unit test organization strategies for Django projects

    What is "Unit Testing"? A method by which individual units of source code are tested.

    • Each app needs it's own test module.
      • Each module should have (many) submodules.
        • Each contains classes that test units of code.
    • Organize your test suite however you want
      • Just be consistent.
  3. Dealing with increasingly complex test scenarios
    • Separate code and service/infrastructure testing
    • Don't keep all the tests in one
    • Use sub-module and root files sparingly.
    • Give Mock ( a try.
  4. Beyond the business logic
    • Testing 3rd party APIs or cache

    • These should be tested separately
      • Better speed of tests if you're not constantly making outbound calls.
    • Dealing with cache:
      • Dev env doesn't usually have long-term or complete sets of cached data.
      • Cache should be agnostic to model changes.
      • Do object structure comparison when pulling from cache.
      • Testing how your app deals with 'old' data, not the infrastructure.
  5. Writing tests can improve coding habits
    • Smaller, modular code is testable, 200-line functions aren't.

    • Write more tests
      • Discover easily-testable coding patterns.
      • Functions should perform a single function.
  6. Tests are alive

  7. How do I enforce testing on my team?
    • Most people don't like 2AM fires.
    • Iterate faster with confidence.
    • Git pre-commit hooks, stop checkins without tests.
    • can make it a game.
    • Public shaming?
  8. How much testing is enough testing?
    • No simple answer for that.
  9. Junction between Unit and Integration
    • Difficult areas to test, because behavior is dependant upon environment.
  10. Testing a virgin codebase

    • You will refactor code as you write tests
  11. Successful strategies

    • Require unit tests for all code going forward

    • Reserve time to clear out test debt

      • Good for new team members
      • Everyone takes a turn
    • Establish a good foundation to build on

    • Every bug is two bugs:

      • One in the code that broke
      • One is the test that would/will catch the bug
  12. Test data

    • Fixtures, mocks, ObjectCreator
    • Use SQLite
  13. Should I get test data from cache?

    • No. Adds unnecessary complexity to unit tests.
    • If tests only succeed with certain caches, probably some code that can be refactored.
  14. Graceful code degredation

    • Devs need to think outside their dev instance
    • Service unavailable shouldn't mean your site is unavailable
    • If failure is catastrophic, you should know how and why it happened.
  15. Forcing awareness

    • Code should not be written in such a way that it won't work (at least in some fashion) if certain services or infrastructers are missing/different.
  16. Test infrastructe

    • Staging should be identical to production.
    • Run Solr or RabbitMQ on staging, but not dev.
    • Don't overdo it on logging.
  17. Testing tools

    • Nose
    • Coverage
    • Mock
Code not tested is broken by design