Running the tests
If you push a commit to a branch in the Dallinger organization on GitHub, or open a pull request from your own fork, Dallinger’s automated code tests will be run on GitHub.
The tests include:
Making sure that a source distribution of the Python package can be created.
Running flake8 to make sure Python code conforms to the PEP 8 style guide.
Running the tests for the Python code using pytest and making sure they pass on Python 3.9, 3.10, 3.11, and 3.12.
Making sure that code coverage for the Python code is above the desired threshold.
Making sure the docs build without error.
If you see ImportErrors related to demo packages, this most likely means you
have not installed the dlgr.demos
sub-package. See the
Dallinger development installation instructions for details.
Amazon Mechanical Turk Integration Tests
You can also run all these tests locally, with some additional requirements:
The Amazon Web Services credentials set in .dallingerconfig must correspond to a valid MTurk Sandbox Requester account.
Some tests require access to an MTurk Sandbox Worker account, so you should create this account (probably using the same AWS account as above).
The Worker ID from the Worker account (visible on the dashboard) needs to be set in
tests/config.py
, which should be created by making a copy oftests/config.py.in
before setting the value.tests/config.py
is excluded from version control, so your Id will not be pushed to a remote repository.
Commands
Tests
You can run all tests locally, simply by running:
tox
To run just the fastest Python tests (it’s recommended to run these tests first):
pytest
To run include slower Python tests:
pytest --runslow
To run the Python tests excluding those that interact with Amazon Mechanical Turk, run:
pytest -m "not mturk"
To run all tests except those that require a MTurk Worker ID, run:
pytest -m "not mturkworker"
To run the complete, comprehensive suite of tests which interact Mechanical Turk,
add the mturkfull
option when running the tests:
pytest --mturkfull --runslow
To run the JavaScript tests, run:
npm run test --coverage
Linting
To run black:
black dallinger
To run flake8:
flake8