py.test lets you inject objects into test functions and precisely control their life cycle in relation to the test execution. It is also possible to run a test function multiple times with different objects.
The basic mechanism for injecting objects is also called the funcarg mechanism because objects are ultimately injected by calling a test function with it as an argument. Unlike the classical xUnit approach funcargs relate more to Dependency Injection because they help to de-couple test code from objects required for them to execute.
To create a value with which to call a test function a factory function is called which gets full access to the test function context and can register finalizers or invoke lifecycle-caching helpers. The factory can be implemented in same test class or test module, or in a per-directory conftest.py file or even in an external plugin. This allows full de-coupling of test code and objects needed for test execution.
A test function may be invoked multiple times in which case we speak of parametrized testing. This can be very useful if you want to test e.g. against different database backends or with multiple numerical arguments sets and want to reuse the same set of test functions.
py.test comes with Builtin function arguments and there are some refined usages in the examples section.
Let’s look at a simple self-contained test module:
# content of ./test_simplefactory.py
def pytest_funcarg__myfuncarg(request):
return 42
def test_function(myfuncarg):
assert myfuncarg == 17
This test function needs an injected object named myfuncarg. py.test will discover and call the factory named pytest_funcarg__myfuncarg within the same module in this case.
Running the test looks like this:
$ py.test test_simplefactory.py
=========================== test session starts ============================
platform darwin -- Python 2.7.1 -- pytest-2.2.2
collecting ... collected 1 items
test_simplefactory.py F
================================= FAILURES =================================
______________________________ test_function _______________________________
myfuncarg = 42
def test_function(myfuncarg):
> assert myfuncarg == 17
E assert 42 == 17
test_simplefactory.py:5: AssertionError
========================= 1 failed in 0.03 seconds =========================
This means that indeed the test function was called with a myfuncarg argument value of 42 and the assert fails. Here is how py.test comes to call the test function this way:
Note that if you misspell a function argument or want to use one that isn’t available, you’ll see an error with a list of available function arguments.
You can always issue:
py.test --funcargs test_simplefactory.py
to see available function arguments (which you can also think of as “resources”).
Each funcarg factory receives a request object tied to a specific test function call. A request object is passed to a funcarg factory and provides access to test configuration and context:
A request for function arguments from a test function.
Note that there is an optional param attribute in case there was an invocation to metafunc.addcall(param=...). If no such call was done in a pytest_generate_tests hook, the attribute will not be present.
add finalizer function to be called after test function finished execution.
Return a testing resource managed by setup & teardown calls. scope and extrakey determine when the teardown function will be called so that subsequent calls to setup would recreate the resource.
| Parameters: |
|
|---|
Apply a marker to a single test function invocation. This method is useful if you don’t want to have a keyword/marker on all function invocations.
| Parameters: | marker – a _pytest.mark.MarkDecorator object created by a call to py.test.mark.NAME(...). |
|---|
Retrieve a function argument by name for this test function invocation. This allows one function argument factory to call another function argument factory. If there are two funcarg factories for the same test function argument the first factory may use getfuncargvalue to call the second one and do something additional with the resource.
You can parametrize multiple runs of the same test function by adding new test function calls with different function argument values. Let’s look at a simple self-contained example:
Let’s consider a test module which uses the pytest_generate_tests hook to generate several calls to the same test function:
# content of test_example.py
def pytest_generate_tests(metafunc):
if "numiter" in metafunc.funcargnames:
metafunc.parametrize("numiter", range(10))
def test_func(numiter):
assert numiter < 9
Running this will generate ten invocations of test_func passing in each of the items in the list of range(10):
$ py.test test_example.py
=========================== test session starts ============================
platform darwin -- Python 2.7.1 -- pytest-2.2.2
collecting ... collected 10 items
test_example.py .........F
================================= FAILURES =================================
_______________________________ test_func[9] _______________________________
numiter = 9
def test_func(numiter):
> assert numiter < 9
E assert 9 < 9
test_example.py:6: AssertionError
==================== 1 failed, 9 passed in 0.07 seconds ====================
Obviously, only when numiter has the value of 9 does the test fail. Note that the pytest_generate_tests(metafunc) hook is called during the test collection phase which is separate from the actual test running. Let’s just look at what is collected:
$ py.test --collectonly test_example.py
=========================== test session starts ============================
platform darwin -- Python 2.7.1 -- pytest-2.2.2
collecting ... collected 10 items
<Module 'test_example.py'>
<Function 'test_func[0]'>
<Function 'test_func[1]'>
<Function 'test_func[2]'>
<Function 'test_func[3]'>
<Function 'test_func[4]'>
<Function 'test_func[5]'>
<Function 'test_func[6]'>
<Function 'test_func[7]'>
<Function 'test_func[8]'>
<Function 'test_func[9]'>
============================= in 0.01 seconds =============================
If you want to select only the run with the value 7 you could do:
$ py.test -v -k 7 test_example.py # or -k test_func[7]
=========================== test session starts ============================
platform darwin -- Python 2.7.1 -- pytest-2.2.2 -- /Users/hpk/venv/0/bin/python
collecting ... collected 10 items
test_example.py:5: test_func[7] PASSED
======================= 9 tests deselected by '-k7' ========================
================== 1 passed, 9 deselected in 0.01 seconds ==================
You might want to look at more parametrization examples.
metafunc objects are passed to the pytest_generate_tests hook. They help to inspect a testfunction and to generate tests according to test configuration or values specified in the class or module where a test function is defined:
metafunc.funcargnames: set of required function arguments for given function
metafunc.function: underlying python test function
metafunc.cls: class object where the test function is defined in or None.
metafunc.module: the module object where the test function is defined in.
metafunc.config: access to command line opts and general config
Add new invocations to the underlying test function using the list of argvalues for the given argnames. Parametrization is performed during the collection phase. If you need to setup expensive resources you may pass indirect=True and implement a funcarg factory which can perform the expensive setup just before a test is actually run.
| Parameters: |
|
|---|
(deprecated, use parametrize) Add a new call to the underlying test function during the collection phase of a test run. Note that request.addcall() is called during the test collection phase prior and independently to actual test execution. You should only use addcall() if you need to specify multiple arguments of a test function.
| Parameters: |
|
|---|