Unit Tests in Twisted¶

Unit Tests in the Twisted Philosophy¶

The Twisted development team adheres to the practice of Extreme Programming (XP), and the usage of unit tests is a cornerstone XP practice. Unit tests are a tool to give you increased confidence. You changed an algorithm – did you break something? Run the unit tests. If a test fails, you know where to look, because each test covers only a small amount of code, and you know it has something to do with the changes you just made. If all the tests pass, you’re good to go, and you don’t need to second-guess yourself or worry that you just accidentally broke someone else’s program.

What to Test, What Not to Test¶

You don’t have to write a test for every single method you write, only production methods that could possibly break.

– Kent Beck, Extreme Programming Explained

Running the Tests¶

$ bin/trial twisted

(defun runtests () (interactive)
  (compile "python /somepath/Twisted/bin/trial /somepath/Twisted"))

(global-set-key [(alt t)] 'runtests)

Adding a Test¶

Please don’t add new modules to Twisted without adding tests for them too. Otherwise we could change something which breaks your module and not find out until later, making it hard to know exactly what the change that broke it was, or until after a release, and nobody wants broken code in a release.

Tests go into dedicated test packages such as twisted/test/ or twisted/conch/test/ , and are named test_foo.py , where foo is the name of the module or package being tested. Extensive documentation on using the PyUnit framework for writing unit tests can be found in the links section below.

One deviation from the standard PyUnit documentation: To ensure that any variations in test results are due to variations in the code or environment and not the test process itself, Twisted ships with its own, compatible, testing framework. That just means that when you import the unittest module, you will from twisted.trial import unittest instead of the standard import unittest .

As long as you have followed the module naming and placement conventions, trial will be smart enough to pick up any new tests you write.

PyUnit provides a large number of assertion methods to be used when writing tests. Many of these are redundant. For consistency, Twisted unit tests should use the assert forms rather than the fail forms. Also, use assertEqual , assertNotEqual , and assertAlmostEqual rather than assertEquals , assertNotEquals , and assertAlmostEquals . assertTrue is also preferred over assert_ . You may notice this convention is not followed everywhere in the Twisted codebase. If you are changing some test code and notice the wrong method being used in nearby code, feel free to adjust it.

When you add a unit test, make sure all methods have docstrings specifying at a high level the intent of the test. That is, a description that users of the method would understand.

Test Implementation Guidelines¶

Here are some guidelines to follow when writing tests for the Twisted test suite. Many tests predate these guidelines and so do not follow them. When in doubt, follow the guidelines given here, not the example of old unit tests.

class SSHClientTests(unittest.TestCase):
    def test_sshClient(self):
        foo() # the actual test

Skipping tests, TODO items¶

Trial, the Twisted unit test framework, has some extensions which are designed to encourage developers to add new tests. One common situation is that a test exercises some optional functionality: maybe it depends upon certain external libraries being available, maybe it only works on certain operating systems. The important common factor is that nobody considers these limitations to be a bug.

To make it easy to test as much as possible, some tests may be skipped in certain situations. Individual test cases can raise the SkipTest exception to indicate that they should be skipped, and the remainder of the test is not run. In the summary (the very last thing printed, at the bottom of the test output) the test is counted as a”skip” instead of a “success” or “fail” . This should be used inside a conditional which looks for the necessary prerequisites:

class SSHClientTests(unittest.TestCase):
    def test_sshClient(self):
        if not ssh_path:
            raise unittest.SkipTest("cannot find ssh, nothing to test")
        foo() # do actual test after the SkipTest

You can also set the .skip attribute on the method, with a string to indicate why the test is being skipped. This is convenient for temporarily turning off a test case, but it can also be set conditionally (by manipulating the class attributes after they’ve been defined):

class SomeThingTests(unittest.TestCase):
    def test_thing(self):
        dotest()
    test_thing.skip = "disabled locally"

class MyTests(unittest.TestCase):
    def test_one(self):
        ...
    def test_thing(self):
        dotest()

if not haveThing:
    MyTests.test_thing.im_func.skip = "cannot test without Thing"
    # but test_one() will still run

Finally, you can turn off an entire TestCase at once by setting the .skip attribute on the class. If you organize your tests by the functionality they depend upon, this is a convenient way to disable just the tests which cannot be run.

class TCPTests(unittest.TestCase):
    ...
class SSLTests(unittest.TestCase):
    if not haveSSL:
        skip = "cannot test without SSL support"
    # but TCPTests will still run
    ...

Associating Test Cases With Source Files¶

Please add a test-case-name tag to the source file that is covered by your new test. This is a comment at the beginning of the file which looks like one of the following:

# -*- test-case-name: twisted.test.test_defer -*-

or

#!/usr/bin/env python
# -*- test-case-name: twisted.test.test_defer -*-

This format is understood by emacs to mark “File Variables” . The intention is to accept test-case-name anywhere emacs would on the first or second line of the file (but not in the File Variables: block that emacs accepts at the end of the file). If you need to define other emacs file variables, you can either put them in the``File Variables:`` block or use a semicolon-separated list of variable definitions:

# -*- test-case-name: twisted.test.test_defer; fill-column: 75; -*-

If the code is exercised by multiple test cases, those may be marked by using a comma-separated list of tests, as follows: (NOTE: not all tools can handle this yet.. trial --testmodule does, though)

# -*- test-case-name: twisted.test.test_defer,twisted.test.test_tcp -*-

The test-case-name tag will allow trial --testmodule twisted/dir/myfile.py to determine which test cases need to be run to exercise the code in myfile.py . Several tools (as well as http://launchpad.net/twisted-emacs’s twisted-dev.el ‘s F9 command) use this to automatically run the right tests.

Links¶

• A chapter on Unit Testing in Mark Pilgrim’s Dive Into Python .

• unittest module documentation, in the Python Library Reference .

• UnitTest on the PortlandPatternRepository Wiki , where all the cool ExtremeProgramming kids hang out.

• Unit Tests in Extreme Programming: A Gentle Introduction .

• Ron Jeffries expounds on the importance of Unit Tests at 100% .

• Ron Jeffries writes about the Unit Test in the Extreme Programming practices of C3 .

• PyUnit’s homepage .

• The top-level tests directory, twisted/test , in Subversion.

See also Tips for writing tests for Twisted code .