Running a subset of the tests

First, you have to build the test packages with:

 $ rm -rf debian/test-out; private/build-test-packages;

Then, the following options are available:

Running a single test

To run a single test by its name, use:

 $ private/runtests --onlyrun=test:$name
    

Running all tests for a check

To run all tests for a given check, use:

 $ private/runtests --onlyrun=check:$name
    

$check must be the name of a check (it will test for checks/$check.desc) or "legacy". This will run all tests that start with "$check-".

Running all tests designed for a specific tag

To run all tests that have a "Test-For" or a "Test-Against" for a given tag, use:

 $ private/runtests --onlyrun=tag:$name
    

Running tests under coverage

This feature is currently untested.

It is possible to run most of the tests under Devel::Cover. This is done by passing --coverage to private/runtests. Example:

  $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out

Please note that Devel::Cover does not seem to handle multiple threads too well. You may see spurious warnings/errors if you run the tests with 2 or more active worker threads.

Caveat 1: Coverage for collections (i.e. programs in collection/) does not seem to work at the moment. Therefore, they often end up with (next to) zero coverage in the generated reports.

Caveat 2: Devel::Cover sometimes changes the output of Lintian or tools called by Lintian. Obviously, this leads to test failures. Therefore, you may see weird test failures (or warnings) when running under coverage.

Collecting the coverage you want in a reasonable time

Collecting coverage is excruciatingly slow. This is not helped by the fact that it becomes unreliable when run under 2 or more threads.

Fortunately, Devel::Cover "appends" to its cover database. This allows you to "slowly" build up the coverage database over multiple runs. Example:

  $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out suite:scripts
  $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out suite:debs
  $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out suite:source
  ...

Or:

  $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out $check
  $ private/runtests --coverage --dump-logs -j1 -k t debian/test-out legacy