pytest-fix-issue-50 / ISSUES.txt

Full commit
checks / deprecations for next release
tags: bug 2.4 core xdist

* check oejskit plugin compatibility
* move pytest_nose out of pylib because it implicitely extends
  the protocol now - setup/teardown is called at module level.
  consider making calling of setup/teardown configurable

profiling / hook call optimization
tags: enhancement 2.1

bench/ reveals that for very quick running
unit tests the hook architecture is a bit slow.
Profile and improve hook calls.

do early-teardown of test modules
tags: feature 2.1

currently teardowns are called when the next tests is setup
except for the function/method level where interally
"teardown_exact" tears down immediately.  Generalize
this to perform the "neccessary" teardown compared to
the "next" test item during teardown - this should
get rid of some irritations because otherwise e.g.
prints of teardown-code appear in the setup of the next test.

consider and document __init__ file usage in test directories
tags: bug 2.1 core

Currently, a test module is imported with its fully qualified
package path, determined by checking __init__ files upwards.
This has the side effect that a source package at the root
of the test dir could be imported as well.  This is somewhat
convenient but complicates the picture for running tests against
different versions of a package.  Also, implicit sys.path
manipulations are problematic per-se.  Maybe factorting out
a pytest_addsyspath hook which can be disabled from the command line
makes sense.  In any case documentation/recommendations for
certain scenarios makes sense.

relax requirement to have tests/testing contain an __init__
tags: feature 2.1

A local test run of a "tests" directory may work
but a remote one fail because the tests directory
does not contain an "". Either give
an error or make it work without the
i.e. port the nose-logic of unloading a test module.

customize test function collection
tags: feature 2.1

- introduce py.test.mark.nocollect for not considering a function for
  test collection at all.  maybe also introduce a py.test.mark.test to
  explicitely mark a function to become a tested one.  Lookup JUnit ways
  of tagging tests.

- allow an easy way to customize "test_", "Test" prefixes for file paths
  and test function/class names. the current customizable Item requires
  too much code/concepts to influence this collection matching.
  maybe introduce pytest_pycollect_filters = {
    'file': 'test*.py',
    'function': 'test*',
    'class': 'Test*',

introduce py.test.mark.platform
tags: feature 2.1

Introduce nice-to-spell platform-skipping, examples:

    @py.test.mark.platform("not python3")
    @py.test.mark.platform("win32 and not python3")
    @py.test.mark.platform("not (jython and win32)")
    @py.test.mark.platform("not (jython and win32)", xfail=True)

etc. Idea is to allow Python expressions which can operate
on common spellings for operating systems and python
interpreter versions.

pytest.mark.xfail signature change
tags: feature 2.1

change to pytest.mark.xfail(reason, (optional)condition)
to better implement the word meaning.  It also signals
better that we always have some kind of an implementation
reason that can be formualated.
Compatibility? Maybe rename to "pytest.mark.xfail"?

introduce py.test.mark registration
tags: feature 2.1

introduce a hook that allows to register a named mark decorator
with documentation and add "py.test --marks" to get
a list of available marks.  Deprecate "dynamic" mark

allow to non-intrusively apply skipfs/xfail/marks
tags: feature 2.1

use case: mark a module or directory structures
to be skipped on certain platforms (i.e. no import
attempt will be made).

consider introducing a hook/mechanism that allows to apply marks
from conftests or plugins.

explicit referencing of files
tags: feature 2.1

allow to name files (in sub directories) that should
be imported early, as to include command line options.

improve central py.test ini file
tags: feature 2.1

introduce more declarative configuration options:
- (to-be-collected test directories)
- required plugins
- test func/class/file matching patterns
- skip/xfail (non-intrusive)
- pytest.ini and tox.ini and setup.cfg configuration in the same file

new documentation
tags: feature 2.1

- logo py.test
- examples for unittest or functional testing
- resource management for functional testing
- patterns: page object
- parametrized testing
- better / more integrated plugin docs

generalize parametrized testing to generate combinations
tags: feature 2.1

think about extending metafunc.addcall or add a new method to allow to
generate tests with combinations of all generated versions - what to do
about "id" and "param" in such combinations though?

introduce py.test.mark.multi
tags: feature 1.3

introduce py.test.mark.multi to specify a number
of values for a given function argument.

have imported module mismatch honour relative paths
tags: bug 2.1

With 1.1.1 py.test fails at least on windows if an import
is relative and compared against an absolute
path. Normalize.

call termination with small timeout
tags: feature 2.1
test: testing/pytest/dist/ - test_terminate_on_hanging_node

Call gateway group termination with a small timeout if available.
Should make dist-testing less likely to leave lost processes.

consider globals: py.test.ensuretemp and config
tags: experimental-wish 2.1

consider deprecating py.test.ensuretemp and py.test.config
to further reduce py.test globality.  Also consider
having py.test.config and ensuretemp coming from
a plugin rather than being there from the start.

consider allowing funcargs for setup methods
tags: experimental-wish 2.1

Users have expressed the wish to have funcargs available to setup
functions.  Experiment with allowing funcargs there - it might
also help to make the py.test.ensuretemp and config deprecation.
For filling funcargs for setup methods, we could call funcarg
factories with a request object that not have a cls/function
attributes.  However, how to handle parametrized test functions
and funcargs?

setup_function -> request can be like it is now
setup_class -> request has no request.function
setup_module -> request has no request.cls

consider pytest_addsyspath hook
tags: 2.1

py.test could call a new pytest_addsyspath() in order to systematically
allow manipulation of sys.path and to inhibit it via --no-addsyspath
in order to more easily run against installed packages.

Alternatively it could also be done via the config object
and pytest_configure.

show plugin information in test header
tags: feature 2.1

Now that external plugins are becoming more numerous
it would be useful to have external plugins along with
their versions displayed as a header line.

deprecate global py.test.config usage
tags: feature 2.1

py.test.ensuretemp and py.test.config are probably the last
objects containing global state.  Often using them is not
neccessary.  This is about trying to get rid of them, i.e.
deprecating them and checking with PyPy's usages as well
as others.

remove deprecated bits in
tags: feature 2.1

In an effort to further simplify code, review and remove deprecated bits
in  Probably good:
- inline consider_file/dir methods, no need to have them
  subclass-overridable because of hooks