| PYBUILD-AUTOPKGTEST(1) | General Commands Manual | PYBUILD-AUTOPKGTEST(1) |
pybuild-autopkgtest - invokes the test suite against requested Python versions and installed packages
pybuild-autopkgtest takes no options or arguments. All configuration is done via the same mechanisms you use to control pybuild itself, namely by having build dependencies on the right packages, or by setting PYBUILD_* environment variables in debian/rules.
pybuild-autopkgtest is an autopkgtest test runner that reuses all the pybuild infrastructure to run tests against installed packages, as expected by autopkgtest. To enable pybuild-autopkgtest for your package, you need to add Testsuite: autopkgtest-pkg-pybuild to the source stanza in debian/control. This will cause autodep8(1) to produce the correct contents for debian/tests/control.
pybuild-autopkgtest will run the tests in exactly the same way as pybuild will during the build, with the exception that the tests are not run in the build directory. The test files are copied to a temporary directory, so that the tests will run against the installed Python modules, and not against anything in the source tree.
All the pybuild infrastructure is used, so for most packages you don't need to add any extra code configure for pybuild-autopkgtest. For example, just having a python3-pytest as a build dependency is enough to make the test runner use pytest to run the tests.
The tests are executed via a temporary makefile that includes debian/rules from the package, so that any environment variables defined there will also be available during autopkgtest, including but not limited to PYBUILD_* variables for configuring the behavior of pybuild itself.
Since pybuild-autopkgtest reuses environment variables set in debian/rules, migrating packages to use pybuild-autopkgtest should not require much effort.
You might have a debian/tests/control that duplicates what pybuild-autopkgtest already does, e.g. copying the test files to a temporary directory, changing to it, and running the tests from there. Such test entries can usually be removed in favor of adding Testsuite: autopkgtest-pkg-pybuild to debian/control.
In general, you want to move any test-related command line arguments to pybuild into environment variables in debian/rules.
You can also have specialized, manually-written test cases, alongside the ones autogenerated by autodep8. For this, both set Testsuite: autopkgtest-pkg-pybuild in debian/control and keep your custom tests in debian/tests/control.
Ideally, the behavior of the tests should be the same during the build and under autopkgtest, except for the fact that during autopkgtest the tests should load the code from the installed package. pybuild-autopkgtest should do this correctly most of the time.
There are situations, however, in which you need a slightly different behavior during the autopkgtest run. There are a few mechanisms to support that:
The control file produced by autodep8(1) looks like this:
Test-Command: pybuild-autopkgtest Depends: @, @builddeps@, Restrictions: allow-stderr, skippable, Features: test-name=pybuild-autopkgtest
You should never need to hardcode this in debian/tests/control. You can add extra items to Restrictions and Depends by providing a configuration file for autodep8 (debian/tests/autopkgtest-pkg-pybuild.conf) like this:
extra_depends=foo, bar extra_restrictions=isolation-container, breaks-testbed
Antonio Terceiro, 2021