py.test implements all aspects of configuration, collection, running and reporting by calling well specified hooks. Virtually any Python module can be registered as a plugin. It can implement any number of hook functions (usually two or three) which all have a pytest_ prefix, making hook functions easy to distinguish and find. There are three basic locations types:
local conftest.py plugins contain directory-specific hook implementations. Session and test running activities will invoke all hooks defined in “higher up” conftest.py files. Example: Assume the following layout and content of files:
a/conftest.py:
def pytest_runtest_setup(item):
# called for running each test in 'a' directory
print ("setting up", item)
a/test_in_subdir.py:
def test_sub():
pass
test_flat.py:
def test_flat():
pass
Here is how you might run it:
py.test test_flat.py # will not show "setting up"
py.test a/test_sub.py # will show "setting up"
A note on ordering: py.test loads all conftest.py files upwards from the command line file arguments. It usually performs look up right-to-left, i.e. the hooks in “closer” conftest files will be called earlier than further away ones.
Note
If you have conftest.py files which do not reside in a python package directory (i.e. one containing an __init__.py) then “import conftest” can be ambigous because there might be other conftest.py files as well on your PYTHONPATH or sys.path. It is thus good practise for projects to either put conftest.py under a package scope or to never import anything from a conftest.py file.
Installing a plugin happens through any usual Python installation tool, for example:
pip install pytest-NAME
pip uninstall pytest-NAME
If a plugin is installed, py.test automatically finds and integrates it, there is no need to activate it. If you don’t need a plugin anymore simply de-install it. You can find a list of available plugins through a pytest- pypi.python.org search.
If you want to write a plugin, there are many real-life examples you can copy from:
All of these plugins implement the documented well specified hooks to extend and add functionality.
If you want to make your plugin externally available, you may define a so called entry point for your distribution so that py.test finds your plugin module. Entry points are a feature that is provided by setuptools or Distribute. The concrete entry point is pytest11. To make your plugin available you can insert the following lines in your setuptools/distribute-based setup-invocation:
# sample ./setup.py file
from setuptools import setup
setup(
name="myproject",
packages = ['myproject']
# the following makes a plugin available to py.test
entry_points = {
'pytest11': [
'name_of_plugin = myproject.pluginmodule',
]
},
)
If a package is installed this way, py.test will load myproject.pluginmodule and accordingly call functions if they match the well specified hooks.
py.test loads plugin modules at tool startup in the following way:
You can require plugins in a test module or a conftest file like this:
pytest_plugins = "name1", "name2",
When the test module or conftest plugin is loaded the specified plugins will be loaded as well. You can also use dotted path like this:
pytest_plugins = "myapp.testsupport.myplugin"
which will import the specified module as a py.test plugin.
If a plugin wants to collaborate with code from another plugin it can obtain a reference through the plugin manager like this:
plugin = config.pluginmanager.getplugin("name_of_plugin")
If you want to look at the names of existing plugins, use the --traceconfig option.
_pytest.assertion | support for presented detailed information in failing assertions. |
_pytest.capture | per-test stdout/stderr capturing mechanisms, capsys and capfd function arguments. |
_pytest.config | command line configuration, ini-file and conftest.py processing. |
_pytest.doctest | discover and run doctests in modules and test files. |
_pytest.genscript | generate a single-file self-contained version of py.test |
_pytest.helpconfig | version info, help messages, tracing configuration. |
_pytest.junitxml | report test results in JUnit-XML format, for use with Hudson and build integration servers. |
_pytest.mark | generic mechanism for marking and selecting python functions. |
_pytest.monkeypatch | monkeypatching and mocking functionality. |
_pytest.nose | run test suites written for nose. |
_pytest.pastebin | submit failure or test session information to a pastebin service. |
_pytest.pdb | interactive debugging with PDB, the Python Debugger. |
_pytest.pytester | (disabled by default) support for testing py.test and py.test plugins. |
_pytest.python | Python test discovery, setup and run of test functions. |
_pytest.recwarn | recording warnings during test function execution. |
_pytest.resultlog | (disabled by default) create result information in a plain text file. |
_pytest.runner | basic collect and runtest protocol implementations |
_pytest.session | core implementation of testing process: init, session, runtest loop. |
_pytest.skipping | support for skip/xfail functions and markers. |
_pytest.terminal | terminal reporting of the full testing process. |
_pytest.tmpdir | support for providing temporary directories to test functions. |
_pytest.unittest | discovery and running of std-library “unittest” style tests. |
py.test calls hook functions to implement initialization, running, test execution and reporting. When py.test loads a plugin it validates that all hook functions conform to their respective hook specification. Each hook function name and its argument names need to match a hook specification exactly but it is allowed for a hook function to accept less parameters than specified. If you mistype argument names or the hook name itself you get useful errors.
return initialized config object, parsing the specified args.
return dict of name->object to be made globally available in the py.test/pytest namespace. This hook is called before command line options are parsed.
add optparse-style options and ini-style config values via calls to parser.addoption and parser.addini(...).
called for performing the main command line action. The default implementation will invoke the configure hooks and runtest_mainloop.
All all runtest related hooks receive a pytest.Item object.
implements the standard runtest_setup/call/teardown protocol including capturing exceptions and calling reporting hooks on the results accordingly.
Return boolean: | True if no further hook implementations should be invoked. |
---|
return a _pytest.runner.TestReport object for the given pytest.Item and _pytest.runner.CallInfo.
For deeper understanding you may look at the default implementation of these hooks in _pytest.runner and maybe also in _pytest.pdb which intercepts creation of reports in order to drop to interactive debugging.
The _pytest.terminal reported specifically uses the reporting hook to print information about a test run.
py.test calls the following hooks for collecting files and directories:
return True to prevent considering this path for collection. This hook is consulted for all files and directories prior to calling more specific hooks.
called before traversing a directory for collection files.
return collection Node or None for the given path. Any new node needs to have the specified parent as a parent.
For influencing the collection of objects in Python modules you can use the following hook:
Session related reporting hooks:
And here is the central hook for reporting about test execution:
access to configuration values, pluginmanager and plugin hooks.
return configuration value from an ini file. If the specified name hasn’t been registered through a prior parse.addini call (usually from a plugin), a ValueError is raised.
return name value looked set from command line options.
(deprecated) if we can’t find the option also lookup the name in a matching conftest file.
(deprecated) return getvalue(name) or call py.test.skip if no value exists.
command line option values, usually added via parser.addoption(...) or parser.getgroup(...).addoption(...) calls
a pluginmanager instance
Parser for command line arguments.
base class for all Nodes in the collection tree. Collector subclasses have children, Items are terminal nodes.
deprecated attribute ‘Class’, use pytest.Class
deprecated attribute ‘File’, use pytest.File
deprecated attribute ‘Function’, use pytest.Function
deprecated attribute ‘Item’, use pytest.Item
deprecated attribute ‘Module’, use pytest.Module
the test config object
filesystem path where this node was collected from
return list of all parent collectors up to self, starting from root of collection tree.
a unique name with the scope of the parent
the parent collector node.
the collection this node is part of
Result/Exception info a function invocation.
None or ExceptionInfo object.
context of invocation: one of “setup”, “call”, “teardown”, “memocollect”
Basic test report object (also used for setup and teardown calls if they fail).
a name -> value dictionary containing all keywords and markers associated with a test invocation.
a (filesystempath, lineno, domaininfo) tuple indicating the actual location of a test item - it might be different from the collected one e.g. if a method is inherited from a different module.
None or a failure representation.
normalized collection node id
test outcome, always one of “passed”, “failed”, “skipped”.
one of ‘setup’, ‘call’, ‘teardown’ to indicate runtest phase.