| TOX(1) | tox | TOX(1) |
tox - tox Documentation
tox aims to automate and standardize testing in Python. It is part of a larger vision of easing the packaging, testing and release process of Python software.
tox is a generic virtualenv management and test command line tool you can use for:
First, install tox with pip install tox. Then put basic information about your project and the test environments you want your project to run in into a tox.ini file residing right next to your setup.py file:
# content of: tox.ini , put in same dir as setup.py [tox] envlist = py27,py36 [testenv] # install pytest in the virtualenv where commands will be executed deps = pytest commands =
# NOTE: you can run any command line tool here - not just tests
pytest
You can also try generating a tox.ini file automatically, by running tox-quickstart and then answering a few simple questions.
To sdist-package, install and test your project against Python2.7 and Python3.6, just type:
tox
and watch things happening (you must have python2.7 and python3.6 installed in your environment otherwise you will see errors). When you run tox a second time you'll note that it runs much faster because it keeps track of virtualenv details and will not recreate or re-install dependencies. You also might want to checkout examples to get some more ideas.
tox roughly follows the following phases:
python setup.py sdist
Note that for this operation the same Python environment will be used as the one tox is installed into (therefore you need to make sure that it contains your build dependencies). Skip this step for application projects that don't have a setup.py.
2. install (optional): install the environment dependencies specified inside the deps configuration section, and then the earlier packaged source distribution. By default pip is used to install packages, however one can customise this via install_command. Note pip will not update project dependencies (specified either in the install_requires or the extras section of the setup.py) if any version already exists in the virtual environment; therefore we recommend to recreate your environments whenever your project dependencies change.
3. commands: run the specified commands in the specified order. Whenever the exit code of any of them is not zero stop, and mark the environment failed. Note, starting a command with a single dash character means ignore exit code.
____________________ summary ____________________ py27: commands succeeded ERROR: py36: commands failed
Only if all environments ran successfully tox will return exit code 0 (success). In this case you'll also see the message congratulations :).
tox will take care of environment isolation for you: it will strip away all operating system environment variables not specified via passenv. Furthermore, it will also alter the PATH variable so that your commands resolve first and foremost within the current active tox environment. In general all executables in the path are available in commands, but tox will emit a warning if it was not explicitly allowed via whitelist_externals.
tox has influenced several other projects in the Python test automation space. If tox doesn't quite fit your needs or you want to do more research, we recommend taking a look at these projects:
Pythons: CPython 2.7 and 3.4 or later, Jython-2.5.1, pypy-1.9ff
Operating systems: Linux, Windows, OSX, Unix
Installer Requirements: setuptools
License: MIT license
git repository: https://github.com/tox-dev/tox
Use the following command:
pip install tox
It is fine to install tox itself into a virtualenv environment.
Consult the GitHub page how to clone the git repository:
and then install in your environment with something like:
$ cd <path/to/clone> $ pip install .
or install it editable if you want code changes to propagate automatically:
$ cd <path/to/clone> $ pip install --editable .
so that you can do changes and submit patches.
You can also find tox packaged for many Linux distributions and Homebrew for macOs - usually under the name of python-tox or simply tox. Be aware though that there also other projects under the same name (most prominently a secure chat client with no affiliation to this project), so make sure you install the correct package.
Put basic information about your project and the test environments you want your project to run in into a tox.ini file that should reside next to your setup.py file:
# content of: tox.ini , put in same dir as setup.py [tox] envlist = py27,py36 [testenv] # install testing framework # ... or install anything else you might need here deps = pytest # run the tests # ... or run any other command line tool you need to run here commands = pytest
To sdist-package, install and test your project, you can now type at the command prompt:
tox
This will sdist-package your current project, create two virtualenv Environments, install the sdist-package into the environments and run the specified command in each of them. With:
tox -e py36
you can run restrict the test run to the python3.6 environment.
Available "default" test environments names are:
py py2 py27 py3 py34 py35 py36 py37 py38 jython pypy pypy2 pypy27 pypy3 pypy35
The environment py uses the version of Python used to invoke tox.
However, you can also create your own test environment names, see some of the examples in examples.
The tox configuration can also be in pyproject.toml (if you want to avoid an extra file).
Currently only the old format is supported via legacy_tox_ini, a native implementation is planned though.
[build-system] requires = [ "setuptools >= 35.0.2", "wheel >= 0.29.0"] build-backend = "setuptools.build_meta" [tool.tox] legacy_tox_ini = """ [tox] envlist = py27,py36 [testenv] deps = pytest >= 3.0.0, <4 commands = pytest """
Note that when you define a pyproject.toml you must define the build-requires section per PEP-518.
New in version 2.0.
If you want to specify which platform(s) your test environment runs on you can set a platform regular expression like this:
[testenv] platform = linux2|darwin
If the expression does not match against sys.platform the test environment will be skipped.
New in version 1.5.
Sometimes you may want to use tools not contained in your virtualenv such as make, bash or others. To avoid warnings you can use the whitelist_externals testenv configuration:
# content of tox.ini [testenv] whitelist_externals = make
/bin/bash
New in version 1.6.1.
(experimental) If you have a requirements.txt file or a constraints.txt file you can add it to your deps variable like this:
[testenv] deps = -rrequirements.txt
or
[testenv] deps = -cconstraints.txt
or
[testenv] deps = -rrequirements.txt -cconstraints.txt
All installation commands are executed using {toxinidir} (the directory where tox.ini resides) as the current working directory. Therefore, the underlying pip installation will assume requirements.txt or constraints.txt to exist at {toxinidir}/requirements.txt or {toxinidir}/constraints.txt.
This is actually a side effect that all elements of the dependency list is directly passed to pip.
For more details on requirements.txt files or constraints.txt files please see:
New in version 0.9.
To install dependencies and packages from a different default PyPI server you can type interactively:
tox -i https://pypi.my-alternative-index.org
This causes tox to install dependencies and the sdist install step to use the specified url as the index server.
You can cause the same effect by this tox.ini content:
[tox] indexserver =
default = https://pypi.my-alternative-index.org
New in version 0.9.
You can instrument tox to install dependencies from different PyPI servers, example:
[tox] indexserver =
DEV = https://mypypiserver.org [testenv] deps =
# docutils will be installed directly from PyPI
docutils
# mypackage will be installed from custom "DEV" PyPI url
:DEV:mypackage
This configuration will install docutils from the default Python PYPI server and will install the mypackage from our DEV indexserver, and the respective https://mypypiserver.org url. You can override config file settings from the command line like this:
tox -i DEV=https://pypi.org/simple # changes :DEV: package URLs tox -i https://pypi.org/simple # changes default
New in version 1.6.
By default tox uses pip to install packages, both the package-under-test and any dependencies you specify in tox.ini. You can fully customize tox's install-command through the testenv-specific install_command=ARGV setting. For instance, to use pip's --find-links and --no-index options to specify an alternative source for your dependencies:
[testenv]
install_command = pip install --pre --find-links https://packages.example.com --no-index {opts} {packages}
New in version 0.9.
To force tox to recreate a (particular) virtual environment:
tox --recreate -e py27
would trigger a complete reinstallation of the existing py27 environment (or create it afresh if it doesn't exist).
New in version 2.0.
By default tox will only pass the PATH environment variable (and on windows SYSTEMROOT and PATHEXT) from the tox invocation to the test environments. If you want to pass down additional environment variables you can use the passenv option:
[testenv] passenv = LANG
When your test commands execute they will execute with the same LANG setting as the one with which tox was invoked.
New in version 1.0.
If you need to set an environment variable like PYTHONPATH you can use the setenv directive:
[testenv]
setenv = PYTHONPATH = {toxinidir}/subdir
When your test commands execute they will execute with a PYTHONPATH setting that will lead Python to also import from the subdir below the directory where your tox.ini file resides.
New in version 1.6.2.
By default, tox sets PYTHONHASHSEED for test commands to a random integer generated when tox is invoked. This mimics Python's hash randomization enabled by default starting in Python 3.3. To aid in reproducing test failures, tox displays the value of PYTHONHASHSEED in the test output.
You can tell tox to use an explicit hash seed value via the --hashseed command-line option to tox. You can also override the hash seed value per test environment in tox.ini as follows:
[testenv] setenv = PYTHONHASHSEED = 100
If you wish to disable this feature, you can pass the command line option --hashseed=noset when tox is invoked. You can also disable it from the tox.ini by setting PYTHONHASHSEED = 0 as described above.
WARNING:
In some cases, you may want to ignore a command exit code. For example:
[testenv:py27] commands = coverage erase
{envbindir}/python setup.py develop
coverage run -p setup.py test
coverage combine
- coverage html
{envbindir}/flake8 loads
By using the - prefix, similar to a make recipe line, you can ignore the exit code for that command.
If you have a large matrix of dependencies, python versions and/or environments you can use generative-envlist and conditional settings to express that in a concise form:
[tox]
envlist = py{27,34,36}-django{15,16}-{sqlite,mysql}
[testenv]
deps =
django15: Django>=1.5,<1.6
django16: Django>=1.6,<1.7
# use PyMySQL if factors "py34" and "mysql" are present in env name
py34-mysql: PyMySQL
# use urllib3 if any of "py36" or "py27" are present in env name
py27,py36: urllib3
# mocking sqlite on 2.7 and 3.6 if factor "sqlite" is present
py{27,36}-sqlite: mock
By default virtualenv will use symlinks to point to the system's python files, modules, etc. If you want the files to be copied instead, possibly because your filesystem is not capable of handling symbolic links, you can instruct virtualenv to use the "--always-copy" argument meant exactly for that purpose, by setting the alwayscopy directive in your environment:
[testenv] alwayscopy = True
tox allows running environments in parallel:
WARNING:
Example final output:
$ tox -e py27,py36,coverage -p all ✔ OK py36 in 9.533 seconds ✔ OK py27 in 9.96 seconds ✔ OK coverage in 2.0 seconds ___________________________ summary ______________________________________________________
py27: commands succeeded
py36: commands succeeded
coverage: commands succeeded
congratulations :)
Example progress bar, showing a rotating spinner, the number of environments running and their list (limited up to 120 characters):
⠹ [2] py27 | py36
Although one can use tox to develop and test applications one of its most popular usage is to help library creators. Libraries need first to be packaged, so then they can be installed inside a virtual environment for testing. To help with this tox implements PEP-517 and PEP-518. This means that by default tox will build source distribution out of source trees. Before running test commands pip is used to install the source distribution inside the build environment.
To create a source distribution there are multiple tools out there and with PEP-517 and PEP-518 you can easily use your favorite one with tox. Historically tox only supported setuptools, and always used the tox host environment to build a source distribution from the source tree. This is still the default behavior. To opt out of this behaviour you need to set isolated builds to true.
Using the pyproject.toml file at the root folder (alongside setup.py) one can specify build requirements.
[build-system] requires = [
"setuptools >= 35.0.2",
"setuptools_scm >= 2.0.0, <3" ] build-backend = "setuptools.build_meta"
# tox.ini [tox] isolated_build = True
flit requires Python 3, however the generated source distribution can be installed under python 2. Furthermore it does not require a setup.py file as that information is also added to the pyproject.toml file.
[build-system] requires = ["flit >= 1.1"] build-backend = "flit.buildapi" [tool.flit.metadata] module = "package_toml_flit" author = "Happy Harry" author-email = "happy@harry.com" home-page = "https://github.com/happy-harry/is"
# tox.ini [tox] isolated_build = True
It is easy to integrate pytest runs with tox. If you encounter issues, please check if they are listed as a known issue and/or use the support channels.
Assuming the following layout:
tox.ini # see below for content setup.py # a classic distutils/setuptools setup.py file
and the following tox.ini content:
[tox]
envlist = py35,py36
[testenv]
deps = pytest # PYPI package providing pytest
commands = pytest {posargs} # substitute with tox' positional arguments
you can now invoke tox in the directory where your tox.ini resides. tox will sdist-package your project, create two virtualenv environments with the python3.5 and python3.6 interpreters, respectively, and will then run the specified test command in each of them.
Assuming the following layout:
tox.ini # see below for content setup.py # a classic distutils/setuptools setup.py file tests # the directory containing tests
and the following tox.ini content:
[tox]
envlist = py35,py36
[testenv]
changedir = tests
deps = pytest
# change pytest tempdir and add posargs from command line
commands = pytest --basetemp={envtmpdir} {posargs}
you can invoke tox in the directory where your tox.ini resides. Differently than in the previous example the pytest command will be executed with a current working directory set to tests and the test run will use the per-virtualenv temporary directory.
pytest supports distributing tests to multiple processes and hosts through the pytest-xdist plugin. Here is an example configuration to make tox use this feature:
[testenv]
deps = pytest-xdist
changedir = tests
# use three sub processes
commands = pytest --basetemp={envtmpdir} \
--confcutdir=.. \
-n 3 \
{posargs}
Too long filenames. you may encounter "too long filenames" for temporarily created files in your pytest run. Try to not use the "--basetemp" parameter.
installed-versus-checkout version. pytest collects test modules on the filesystem and then tries to import them under their fully qualified name. This means that if your test files are importable from somewhere then your pytest invocation may end up importing the package from the checkout directory rather than the installed package.
This issue may be characterised by pytest test-collection error messages, in python 3.x environments, that look like:
import file mismatch: imported module 'myproj.foo.tests.test_foo' has this __file__ attribute:
/home/myuser/repos/myproj/build/lib/myproj/foo/tests/test_foo.py which is not the same as the test file we want to collect:
/home/myuser/repos/myproj/myproj/foo/tests/test_foo.py HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
There are a few ways to prevent this.
With installed tests (the tests packages are known to setup.py), a safe and explicit option is to give the explicit path {envsitepackagesdir}/mypkg to pytest. Alternatively, it is possible to use changedir so that checked-out files are outside the import path, then pass --pyargs mypkg to pytest.
With tests that won't be installed, the simplest way to run them against your installed package is to avoid __init__.py files in test directories; pytest will still find and import them by adding their parent directory to sys.path but they won't be copied to other places or be found by Python's import system outside of pytest.
The discover project allows to discover and run unittests and we can easily integrate it in a tox run. As an example, perform a checkout of Pygments:
hg clone https://bitbucket.org/birkenfeld/pygments-main
and add the following tox.ini to it:
[tox] envlist = py27,py35,py36 [testenv] changedir = tests commands = discover deps = discover
If you now invoke tox you will see the creation of three virtual environments and a unittest-run performed in each of them.
Michael Foord has contributed a tox.ini file that allows you to run all tests for his mock project, including some sphinx-based doctests. If you checkout its repository with:
git clone https://github.com/testing-cabal/mock.git
The checkout has a tox.ini file that looks like this:
[tox] envlist = py27,py34,py35,py36 [testenv] deps = unittest2 commands = unit2 discover [] [testenv:py36] commands =
unit2 discover []
sphinx-build -b doctest docs html
sphinx-build docs html deps =
unittest2
sphinx [testenv:py27] commands =
unit2 discover []
sphinx-build -b doctest docs html
sphinx-build docs html deps =
unittest2
sphinx
mock uses unittest2 to run the tests. Invoking tox starts test discovery by executing the unit2 discover commands on Python 2.7, 3.4, 3.5 and 3.6 respectively. Against Python3.6 and Python2.7 it will additionally run sphinx-mediated doctests. If building the docs fails, due to a reST error, or any of the doctests fails, it will be reported by the tox run.
The [] parentheses in the commands provide positional substitution which means you can e.g. type:
tox -- -f -s SOMEPATH
which will ultimately invoke:
unit2 discover -f -s SOMEPATH
in each of the environments. This allows you to customize test discovery in your tox runs.
It is easy to integrate nosetests runs with tox. For starters here is a simple tox.ini config to configure your project for running with nose:
Assuming the following layout:
tox.ini # see below for content setup.py # a classic distutils/setuptools setup.py file
and the following tox.ini content:
[testenv]
deps = nose
# ``{posargs}`` will be substituted with positional arguments from comand line
commands = nosetests {posargs}
you can invoke tox in the directory where your tox.ini resides. tox will sdist-package your project create two virtualenv environments with the python2.7 and python3.6 interpreters, respectively, and will then run the specified test command.
Also you might want to checkout general and documentation.
It's possible to generate the projects documentation with tox itself. The advantage of this path is that now generating the documentation can be part of the CI, and whenever any validations/checks/operations fail while generating the documentation you'll catch it within tox.
No need to use the cryptic make file to generate a sphinx documentation. One can use tox to ensure all right dependencies are available within a virtual environment, and even specify the python version needed to perform the build. For example if the sphinx file structure is under the doc folder the following configuration will generate the documentation under {toxworkdir}/docs_out and print out a link to the generated documentation:
[testenv:docs]
description = invoke sphinx-build to build the HTML docs
basepython = python3.7
deps = sphinx >= 1.7.5, < 2
commands = sphinx-build -d "{toxworkdir}/docs_doctree" doc "{toxworkdir}/docs_out" --color -W -bhtml {posargs}
python -c 'import pathlib; print("documentation available under file://\{0\}".format(pathlib.Path(r"{toxworkdir}") / "docs_out" / "index.html"))'
Note here we say we also require python 3.7, allowing us to use f-strings within the sphinx conf.py. Now one can specify a separate test environment that will validate that the links are correct.
Define one environment to write/generate the documentation, and another to deploy it. Use the config substitution logic to avoid defining dependencies multiple time:
[testenv:docs] description = Run a development server for working on documentation basepython = python3.7 deps = mkdocs >= 1.7.5, < 2
mkdocs-material commands = mkdocs build --clean
python -c 'print("###### Starting local server. Press Control+C to stop server ######")'
mkdocs serve -a localhost:8080 [testenv:docs-deploy] description = built fresh docs and deploy them deps = {[testenv:docs]deps} basepython = {[testenv:docs]basepython} commands = mkdocs gh-deploy --clean
If you invoke tox like this:
tox -- -x tests/test_something.py
the arguments after the -- will be substituted everywhere where you specify {posargs} in your test commands, for example using pytest:
[testenv]
# Could also be in a specific ``[testenv:<NAME>]`` section
commands = pytest {posargs}
or using nosetests:
[testenv]
commands = nosetests {posargs}
the above tox invocation will trigger the test runners to stop after the first failure and to only run a particular test file.
You can specify defaults for the positional arguments using this syntax:
[testenv]
commands = nosetests {posargs:--with-coverage}
Creating virtual environments and installing dependencies is a expensive operation. Therefore tox tries to avoid it whenever possible, meaning it will never perform this unless it detects with absolute certainty that it needs to perform an update. A tox environment creation is made up of:
These three steps are only performed once (given they all succeeded). Subsequent calls that don't detect changes to the traits of that step will not alter the virtual environment in any way. When a change is detected for any of the steps, the entire virtual environment is removed and the operation starts from scratch (this is because it's very hard to determine what would the delta changes would be needed - e.g. a dependency could migrate from one dependency to another, and in this case we would need to install the new while removing the old one).
Here's what traits we track at the moment for each steps:
Whenever you change traits that are not tracked we recommend you to manually trigger a rebuild of the tox environment by passing the -r flag for the tox invocation. For instance, for a setuptools project whenever you modify the install_requires keyword at the next run force the recreation of the tox environment by passing the recreate cli tox flag.
Using the -e ENV[,ENV36,...] option you explicitly list the environments where you want to run tests against. For example, given the previous sphinx example you may call:
tox -e docs
which will make tox only manage the docs environment and call its test commands. You may specify more than one environment like this:
tox -e py27,py36
which would run the commands of the py27 and py36 testenvironments respectively. The special value ALL selects all environments.
You can also specify an environment list in your tox.ini:
[tox] envlist = py27,py36
or override it from the command line or from the environment variable TOXENV:
export TOXENV=py27,py36 # in bash style shells
If you have multiple projects using tox you can make use of a distshare directory where tox will copy in sdist-packages so that another tox run can find the "latest" dependency. This feature allows to test a package against an unreleased development version or even an uncommitted version on your own machine.
By default, {homedir}/.tox/distshare will be used for copying in and copying out artifacts (i.e. Python packages).
For project two to depend on the one package you use the following entry:
# example two/tox.ini
[testenv]
# install latest package from "one" project
deps = {distshare}/one-*.zip
That's all. tox running on project one will copy the sdist-package into the distshare directory after which a tox run on project two will grab it because deps contain an entry with the one-*.zip pattern. If there is more than one matching package the highest version will be taken. tox uses verlib to compare version strings which must be compliant with PEP 386.
If you want to use this with Jenkins, also checkout the jenkins artifact example.
For any pyXY test environment name the underlying pythonX.Y executable will be searched in your system PATH. Similarly, for jython and pypy the respective jython and pypy-c names will be looked for. The executable must exist in order to successfully create virtualenv environments. On Windows a pythonX.Y named executable will be searched in typical default locations using the C:\PythonX.Y\python.exe pattern.
All other targets will use the system python instead. You can override any of the default settings by defining the basepython variable in a specific test environment section, for example:
[testenv:docs] basepython = python2.7
Some projects are large enough that running an sdist, followed by an install every time can be prohibitively costly. To solve this, there are two different options you can add to the tox section. First, you can simply ask tox to please not make an sdist:
[tox] skipsdist=True
If you do this, your local software package will not be installed into the virtualenv. You should probably be okay with that, or take steps to deal with it in your commands section:
[testenv] commands = python setup.py develop
pytest
Running setup.py develop is a common enough model that it has its own option:
[testenv] usedevelop=True
And a corresponding command line option --develop, which will set skipsdist to True and then perform the setup.py develop step at the place where tox normally performs the installation of the sdist. Specifically, it actually runs pip install -e . behind the scenes, which itself calls setup.py develop.
There is an optimization coded in to not bother re-running the command if $projectname.egg-info is newer than setup.py or setup.cfg.
When a command (defined by commands = in tox.ini) fails, it has a non-zero exit code, and an InvocationError exception is raised by tox:
ERROR: InvocationError for command
'<command defined in tox.ini>' (exited with code 1)
If the command starts with pytest or python setup.py test for instance, then the pytest exit codes are relevant.
On unix systems, there are some rather common exit codes. This is why for exit codes larger than 128, if a signal with number equal to <exit code> - 128 is found in the signal module, an additional hint is given:
ERROR: InvocationError for command
'<command>' (exited with code 139) Note: this might indicate a fatal error signal (139 - 128 = 11: SIGSEGV)
where <command> is the command defined in tox.ini, with quotes removed.
The signal numbers (e.g. 11 for a segmentation fault) can be found in the "Standard signals" section of the signal man page. Their meaning is described in POSIX signals.
Beware that programs may issue custom exit codes with any value, so their documentation should be consulted.
Sometimes, no exit code is given at all. An example may be found in pytest-qt issue #170, where Qt was calling abort() instead of exit().
SEE ALSO:
The Jenkins continuous integration server allows to define "jobs" with "build steps" which can be test invocations. If you install tox on your default Python installation on each Jenkins slave, you can easily create a Jenkins multi-configuration job that will drive your tox runs from the CI-server side, using these steps:
import tox
os.chdir(os.getenv("WORKSPACE"))
tox.cmdline() # environment is selected by ``TOXENV`` env variable
The last point requires that your test command creates JunitXML files, for example with pytest it is done like this:
[testenv]
commands = pytest --junitxml=junit-{envname}.xml
NOTE:
If you manage many Jenkins slaves and want to use the latest officially released tox (or latest development version) and want to skip manually installing tox then substitute the above Python build step code with this:
import urllib, os url = "https://bitbucket.org/hpk42/tox/raw/default/toxbootstrap.py" # os.environ['USETOXDEV']="1" # use tox dev version d = dict(__file__="toxbootstrap.py") exec urllib.urlopen(url).read() in d d["cmdline"](["--recreate"])
The downloaded toxbootstrap.py file downloads all necessary files to install tox in a virtual sub environment. Notes:
If you are using a multi-configuration Jenkins job which collects JUnit Test results you will run into problems using the previous method of running the sphinx-build command because it will not generate JUnit results. To accommodate this issue one solution is to have pytest wrap the sphinx-checks and create a JUnit result file which wraps the result of calling sphinx-build. Here is an example:
[testenv:docs] basepython = python # change to ``doc`` dir if that is where your sphinx-docs live changedir = doc deps = sphinx
py commands = pytest --tb=line -v --junitxml=junit-{envname}.xml check_sphinx.py
import py import subprocess def test_linkcheck(tmpdir):
doctrees = tmpdir.join("doctrees")
htmldir = tmpdir.join("html")
subprocess.check_call(
["sphinx-build", "-W", "-blinkcheck", "-d", str(doctrees), ".", str(htmldir)]
) def test_build_docs(tmpdir):
doctrees = tmpdir.join("doctrees")
htmldir = tmpdir.join("html")
subprocess.check_call(
["sphinx-build", "-W", "-bhtml", "-d", str(doctrees), ".", str(htmldir)]
)
Note that pytest is only installed into the docs environment and does not need to be in use or installed with any other environment.
In an extension to artifacts you can also configure Jenkins jobs to access each others artifacts. tox uses the distshare directory to access artifacts and in a Jenkins context (detected via existence of the environment variable HUDSON_URL); it defaults to to {toxworkdir}/distshare.
This means that each workspace will have its own distshare directory and we need to configure Jenkins to perform artifact copying. The recommend way to do this is to install the Jenkins Copy Artifact plugin and for each job which "receives" artifacts you add a Copy artifacts from another project build step using roughly this configuration:
Project-name: name of the other (tox-managed) job you want the artifact from Artifacts to copy: .tox/dist/*.zip # where tox jobs create artifacts Target directory: .tox/distshare # where we want it to appear for us Flatten Directories: CHECK # create no subdir-structure
You also need to configure the "other" job to archive artifacts; This is done by checking Archive the artifacts and entering:
Files to archive: .tox/dist/*.zip
So our "other" job will create an sdist-package artifact and the "copy-artifacts" plugin will copy it to our distshare area. Now everything proceeds as artifacts shows it.
So if you are using defaults you can re-use and debug exactly the same tox.ini file and make use of automatic sharing of your artifacts between runs or Jenkins jobs.
When using tox on a Jenkins instance, there may be a scenario where tox can not invoke pip because the shebang (Unix) line is too long. Some systems only support a limited amount of characters for an interpreter directive (e.x. Linux as a limit of 128). There are two methods to workaround this issue:
Jenkins has parallel stages allowing you to run commands in parallel, however tox package building it is not parallel safe. Use the --parallel--safe-build flag to enable parallel safe builds (this will generate unique folder names for distdir, distshare and log. Here's a generic stage definition demonstrating how to use this inside Jenkins:
stage('run tox envs') {
steps {
script {
def envs = sh(returnStdout: true, script: "tox -l").trim().split('\n')
def cmds = envs.collectEntries({ tox_env ->
[tox_env, {
sh "tox --parallel--safe-build -vve $tox_env"
}]
})
parallel(cmds)
}
}
}
tox can be used for just preparing different virtual environments required by a project.
This feature can be used by deployment tools when preparing deployed project environments. It can also be used for setting up normalized project development environments and thus help reduce the risk of different team members using mismatched development environments.
Here are some examples illustrating how to set up a project's development environment using tox. For illustration purposes, let us call the development environment dev.
First, we prepare the tox configuration for our development environment by defining a [testenv:dev] section in the project's tox.ini configuration file:
[testenv:dev] basepython = python2.7 usedevelop = True
In it we state:
The development environment will reside in toxworkdir (default is .tox) just like the other tox environments.
We can configure a lot more, if we want to. For example, we can add the following to our configuration, telling tox not to reuse commands or deps settings from the base [testenv] configuration:
[testenv:dev] commands = deps =
Once the [testenv:dev] configuration section has been defined, we create the actual development environment by running the following:
tox -e dev
This creates the environment at the path specified by the environment's envdir configuration value.
Let us say we want our project development environment to:
Here is an example configuration for the described scenario:
[testenv:dev] basepython = python2.7 usedevelop = True deps = -rrequirements.txt
Assuming the following layout:
tox.ini # see below for content setup.py # a classic distutils/setuptools setup.py file
and the following tox.ini content:
[tox]
# platform specification support is available since version 2.0
minversion = 2.0
envlist = py{27,36}-{mylinux,mymacos,mywindows}
[testenv]
# environment will be skipped if regular expression does not match against the sys.platform string
platform = mylinux: linux
mymacos: darwin
mywindows: win32
# you can specify dependencies and their versions based on platform filtered environments
deps = mylinux,mymacos: py==1.4.32
mywindows: py==1.4.30
# upon tox invocation you will be greeted according to your platform
commands=
mylinux: python -c 'print("Hello, Linus!")'
mymacos: python -c 'print("Hello, Steve!")'
mywindows: python -c 'print("Hello, Bill!")'
you can invoke tox in the directory where your tox.ini resides. tox creates two virtualenv environments with the python2.7 and python3.6 interpreters, respectively, and will then run the specified command according to platform you invoke tox at.
At the moment tox supports three configuration locations prioritized in the following order:
As far as the configuration format at the moment we only support standard ConfigParser "ini-style" format (there is a plan to add a pure TOML one soon). tox.ini and setup.cfg are such files. Note that setup.cfg requires the content to be under the tox:tox section. pyproject.toml on the other hand is in TOML format. However, one can inline the ini-style format under the tool.tox.legacy_tox_ini key as a multi-line string.
Below you find the specification for the ini-style format, but you might want to skim some examples first and use this page as a reference.
Global settings are defined under the tox section as:
[tox] minversion = 3.4.0
New in version 3.4.0: What tox environments are ran during the tox invocation can be further filtered via the operating system environment variable TOX_SKIP_ENV regular expression (e.g. py27.* means don't evaluate environments that start with the key py27). Skipped environments will be logged at level two verbosity level.
When skip missing interpreters is true will force tox to return success even if some of the specified environments were missing. This is useful for some CI systems or running on a developer box, where you might only have a subset of all your supported interpreters installed but don't want to mark the build as failed because of it. As expected, the command line switch always overrides this setting if passed on the invocation. Setting it to config means that the value is read from the config file.
tox allows setting the python version for an environment via the basepython setting. If that's not set tox can set a default value from the environment name ( e.g. py37 implies Python 3.7). Matching up the python version with the environment name has became expected at this point, leading to surprises when some configs don't do so. To help with sanity of users a warning will be emitted whenever the environment name version does not matches up with this expectation. In a future version of tox, this warning will become an error.
Furthermore, we allow hard enforcing this rule (and bypassing the warning) by setting this flag to true. In such cases we ignore the basepython and instead always use the base python implied from the Python name. This allows you to configure basepython in the global testenv without affecting environments that have implied base python versions.
Specify python packages that need to exist alongside the tox installation for the tox build to be able to start. Use this to specify plugin requirements and build dependencies.
[tox] requires = tox-venv
setuptools >= 30.0.0
NOTE:
Activate isolated build environment. tox will use a virtual environment to build a source distribution from the source tree. For build tools and arguments use the pyproject.toml file as specified in PEP-517 and PEP-518. To specify the virtual environment Python version define use the isolated_build_env config section.
Name of the virtual environment used to create a source distribution from the source tree.
If set to True the content of the output will always be shown when running in parallel mode.
tox environments this depends on. tox will try to run all dependent environments before running this environment. Format is same as envlist (allows factor usage).
WARNING:
It is possible to override global settings inside a Jenkins instance ( detection is by checking for existence of the JENKINS_URL environment variable) by using the tox:jenkins section:
[tox:jenkins] commands = ... # override settings for the jenkins context
Test environments are defined by a:
[testenv:NAME] commands = ...
section. The NAME will be the name of the virtual environment. Defaults for each setting in this section are looked up in the:
[testenv] commands = ...
testenv default section.
Complete list of settings that you can put into testenv* sections:
Changed in version 3.1: After resolving this value if the interpreter reports back a different version number than implied from the name a warning will be printed by default. However, if ignore_basepython_conflict is set, the value is ignored and we force the basepython implied from the factor name.
Each line is interpreted as one command; however a command can be split over multiple lines by ending the line with the \ character.
Commands will execute one by one in sequential fashion until one of them fails (their exit code is non-zero) or all of them succeed. The exit code of a command may be ignored (meaning they are always considered successful) by prefixing the command with a dash (-) - this is similar to how make recipe lines work. The outcome of the environment is considered successful only if all commands (these + setup + teardown) succeeded (exit code ignored via the - or success exit code value of zero).
Commands to run before running the commands. All evaluation and configuration logic applies from commands.
Commands to run after running the commands. Execute regardless of the outcome of both commands and commands_pre. All evaluation and configuration logic applies from commands.
Determines the command used for installing packages into the virtual environment; both the package under test and its dependencies (defined with deps). Must contain the substitution key {packages} which will be replaced by the package(s) to install. You should also accept {opts} if you are using pip -- it will contain index server options such as --pre (configured as pip_pre) and potentially index-options from the deprecated indexserver option.
The list_dependencies_command setting is used for listing the packages installed into the virtual environment.
If true, a non-zero exit code from one command will be ignored and further commands will be executed (which was the default behavior in tox < 2.0). If false, then a non-zero exit code from one command will abort execution of commands for that environment.
It may be helpful to note that this setting is analogous to the -i or ignore-errors option of GNU Make. A similar name was chosen to reflect the similarity in function.
Note that in tox 2.0, the default behavior of tox with respect to treating errors from commands changed. tox < 2.0 would ignore errors by default. tox >= 2.0 will abort on an error by default, which is safer and more typical of CI and command execution tools, as it doesn't make sense to run tests if installing some prerequisite failed and it doesn't make sense to try to deploy if tests failed.
If true, adds --pre to the opts passed to install_command. If install_command uses pip, this will cause it to install the latest available pre-release of any dependencies without a specified version. If false, pip will only install final releases of unpinned dependencies.
Passing the --pre command-line option to tox will force this to true for all testenvs.
Don't set this option if your install_command does not use pip.
[tox] indexserver =
myindexserver = https://myindexserver.example.com/simple [testenv] deps = :myindexserver:pkg
(Experimentally introduced in 1.6.1) all installer commands are executed using the {toxinidir} as the current working directory.
A testenv can define a new platform setting as a regular expression. If a non-empty expression is defined and does not match against the sys.platform string the test environment will be skipped.
Each line contains a NAME=VALUE environment variable setting which will be used for all test command invocations as well as for installing the sdist package into a virtual environment.
Notice that when updating a path variable, you can consider the use of variable substitution for the current value and to handle path separator.
[testenv] setenv =
PYTHONPATH = {env:PYTHONPATH}{:}{toxinidir}
A list of wildcard environment variable names which shall be copied from the tox invocation environment to the test environment when executing test commands. If a specified environment variable doesn't exist in the tox invocation environment it is ignored. You can use * and ? to match multiple environment variables with one name.
Some variables are always passed through to ensure the basic functionality of standard library functions or tooling like pip:
You can override these variables with the setenv option.
If defined the TOX_TESTENV_PASSENV environment variable (in the tox invocation environment) can define additional space-separated variable names that are to be passed down to the test command environment.
Changed in version 2.7: PYTHONPATH will be passed down if explicitly defined. If PYTHONPATH exists in the host environment but is not declared in passenv a warning will be emitted.
WARNING:
If you forget to do that you will get a warning like this:
WARNING: test command found but not installed in testenv
cmd: /path/to/parent/interpreter/bin/<some command>
env: /foo/bar/.tox/python Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
This is useful for situations where hardlinks don't work (e.g. running in VMS with Windows guests).
(DEPRECATED, will be removed in a future version) Multi-line name = URL definitions of python package servers. Dependencies can specify using a specified index server through the :indexservername:depname pattern. The default indexserver definition determines where unscoped dependencies and the sdist install installs from. Example:
[tox] indexserver =
default = https://mypypi.org
will make tox install all dependencies from this PyPI index server (including when installing the project sdist package).
User can set specific path for environment. If path would not be absolute it would be treated as relative to {toxinidir}.
Install the current package in development mode with "setup.py develop" instead of installing from the sdist package. (This uses pip's -e option, so should be avoided if you've specified a custom install_command that does not support -e).
Do not install the current package. This can be used when you need the virtualenv management but do not want to install the current package into that environment.
If set to true a failing result of this testenv will not make tox fail, only a warning will be produced.
A list of "extras" to be installed with the sdist or develop install. For example, extras = testing is equivalent to [testing] in a pip install command. These are not installed if skip_install is true.
Any key=value setting in an ini-file can make use of value substitution through the {...} string-substitution pattern.
You can escape curly braces with the \ character if you need them, for example:
commands = echo "\{posargs\}" = {posargs}
Note some substitutions (e.g. posargs, env) may have addition values attached to it, via the : character (e.g. posargs - default value, env - key). Such substitutions cannot have a space after the : character (e.g. {posargs: magic} while being at the start of a line inside the ini configuration (this would be parsed as factorial {posargs, having value magic).
If you specify a substitution string like this:
{env:KEY}
then the value will be retrieved as os.environ['KEY'] and raise an Error if the environment variable does not exist.
If you specify a substitution string like this:
{env:KEY:DEFAULTVALUE}
then the value will be retrieved as os.environ['KEY'] and replace with DEFAULTVALUE if the environment variable does not exist.
If you specify a substitution string like this:
{env:KEY:}
then the value will be retrieved as os.environ['KEY'] and replace with an empty string if the environment variable does not exist.
Substitutions can also be nested. In that case they are expanded starting from the innermost expression:
{env:KEY:{env:DEFAULT_OF_KEY}}
the above example is roughly equivalent to os.environ.get('KEY', os.environ['DEFAULT_OF_KEY'])
It's possible to inject a config value only when tox is running in interactive shell (standard input):
The first value is the value to inject when the interactive terminal is available, the second value is the value to use when it's not. The later on is optional. A good use case for this is e.g. passing in the --pdb flag for pytest.
New in version 1.0.
If you specify a substitution string like this:
{posargs:DEFAULTS}
then the value will be replaced with positional arguments as provided to the tox command:
tox arg1 arg2
In this instance, the positional argument portion will be replaced with arg1 arg2. If no positional arguments were specified, the value of DEFAULTS will be used instead. If DEFAULTS contains other substitution strings, such as {env:*}, they will be interpreted.,
Use a double -- if you also want to pass options to an underlying test command, for example:
tox -- --opt1 ARG1
will make the --opt1 ARG1 appear in all test commands where [] or {posargs} was specified. By default (see args_are_paths setting), tox rewrites each positional argument if it is a relative path and exists on the filesystem to become a path relative to the changedir setting.
Previous versions of tox supported the [.*] pattern to denote positional arguments with defaults. This format has been deprecated. Use {posargs:DEFAULTS} to specify those.
New in version 1.4.
Values from other sections can be referred to via:
{[sectionname]valuename}
which you can use to avoid repetition of config values. You can put default values in one section and reference them in others to avoid repeating the same values:
[base] deps =
pytest
mock
pytest-xdist [testenv:dulwich] deps =
dulwich
{[base]deps} [testenv:mercurial] deps =
mercurial
{[base]deps}
New in version 1.8.
Suppose you want to test your package against python2.7, python3.6 and against several versions of a dependency, say Django 1.5 and Django 1.6. You can accomplish that by writing down 2*2 = 4 [testenv:*] sections and then listing all of them in envlist.
However, a better approach looks like this:
[tox]
envlist = {py27,py36}-django{15,16}
[testenv]
deps =
pytest
django15: Django>=1.5,<1.6
django16: Django>=1.6,<1.7
py36: unittest2
commands = pytest
This uses two new facilities of tox-1.8:
Let's go through this step by step.
envlist = {py36,py27}-django{15,16}
This is bash-style syntax and will create 2*2=4 environment names like this:
py27-django15 py27-django16 py36-django15 py36-django16
You can still list environments explicitly along with generated ones:
envlist = {py27,py36}-django{15,16}, docs, flake
Keep in mind that whitespace characters (except newline) within {} are stripped, so the following line defines the same environment names:
envlist = {py27,py36}-django{ 15, 16 }, docs, flake
NOTE:
$ tox -l py27-django15 py27-django16 py36-django15 py36-django16 docs flake
Parts of an environment name delimited by hyphens are called factors and can be used to set values conditionally. In list settings such as deps or commands you can freely intermix optional lines with unconditional ones:
[testenv] deps =
pytest
django15: Django>=1.5,<1.6
django16: Django>=1.6,<1.7
py36: unittest2
Reading it line by line:
tox provides a number of default factors corresponding to Python interpreter versions. The conditional setting above will lead to either python3.6 or python2.7 used as base python, e.g. python3.6 is selected if current environment contains py36 factor.
NOTE:
Sometimes you need to specify the same line for several factors or create a special case for a combination of factors. Here is how you do it:
[tox]
envlist = py{27,34,36}-django{15,16}-{sqlite,mysql}
[testenv]
deps =
py34-mysql: PyMySQL ; use if both py34 and mysql are in the env name
py27,py36: urllib3 ; use if either py36 or py27 are in the env name
py{27,36}-sqlite: mock ; mocking sqlite in python 2.x & 3.6
!py34-sqlite: mock ; mocking sqlite, except in python 3.4
sqlite-!py34: mock ; (same as the line above)
Take a look at the first deps line. It shows how you can special case something for a combination of factors, by just hyphenating the combining factors together. This particular line states that PyMySQL will be loaded for python 3.4, mysql environments, e.g. py34-django15-mysql and py34-django16-mysql.
The second line shows how you use the same setting for several factors - by listing them delimited by commas. It's possible to list not only simple factors, but also their combinations like py27-sqlite,py36-sqlite.
The remaining lines all have the same effect and use conditions equivalent to py27-sqlite,py36-sqlite. They have all been added only to help demonstrate the following:
NOTE:
For example, environment py36-mysql-!dev:
It is possible to mix both values substitution and factor expressions. For example:
[tox] envlist = py27,py36,coverage [testenv] deps =
flake8
coverage: coverage [testenv:py27] deps =
{[testenv]deps}
pytest
With the previous configuration, it will install:
For systems supporting executable text files (scripts with a shebang), the system will attempt to parse the interpreter directive to determine the program to execute on the target text file. When tox prepares a virtual environment in a file container which has a large length (e.x. using Jenkins Pipelines), the system might not be able to invoke shebang scripts which define interpreters beyond system limits (e.x. Linux as a limit of 128; BINPRM_BUF_SIZE). To workaround an environment which suffers from an interpreter directive limit, a user can bypass the system's interpreter parser by defining the TOX_LIMITED_SHEBANG environment variable before invoking tox:
export TOX_LIMITED_SHEBANG=1
When the workaround is enabled, all tox-invoked text file executables will have their interpreter directive parsed by and explicitly executed by tox.
tox will inject the following environment variables that you can use to test that your command is running within tox:
New in version 3.4.
tox options
usage: tox [--version] [-h] [--help-ini] [-v] [-q] [--showconfig] [-l] [-a] [-c CONFIGFILE] [-e envlist] [--notest] [--sdistonly] [-p VAL] [-o] [--parallel--safe-build] [--installpkg PATH]
[--develop] [-i URL] [--pre] [-r] [--result-json PATH] [--hashseed SEED] [--force-dep REQ] [--sitepackages] [--alwayscopy] [-s [val]] [--workdir PATH]
[args [args ...]]
Getting in contact:
contact holger at merlinux.eu, an association of experienced well-known Python developers.
Versions follow Semantic Versioning (<major>.<minor>.<patch>). Backward incompatible (breaking) changes will only be introduced in major versions with advance notice in the Deprecations section of releases.
New command line parameter: -a show all defined environments - not just the ones defined in (or generated from) envlist.
New verbosity settings for -l and -a: show user defined descriptions of the environments. This also works for generated environments from factors by concatenating factor descriptions into a complete description.
Note that for backwards compatibility with scripts using the output of -l it's output remains unchanged.
Thanks Bernát Gábor (@gaborbernat).
New in version 2.0.
A growing number of hooks make tox modifiable in different phases of execution by writing plugins.
tox - like pytest and devpi - uses pluggy to provide an extension mechanism for pip-installable internal or devpi/PyPI-published plugins.
To start using a plugin you need to install it in the same environment where the tox host is installed.
e.g.:
$ pip install tox-travis
You can search for available plugins on PyPI by typing pip search tox and filter for packages that are prefixed tox- or contain the "plugin" in the description. You will get some output similar to this:
tox-pipenv (1.4.1) - A pipenv plugin for tox tox-pyenv (1.1.0) - tox plugin that makes tox use ``pyenv which`` to find
python executables tox-globinterpreter (0.3) - tox plugin to allow specification of interpreter
locationspaths to use tox-venv (0.2.0) - Use python3 venvs for python3 tox testenvs tox-cmake (0.1.1) - Build CMake projects using tox tox-travis (0.10) - Seamless integration of tox into Travis CI tox-py-backwards (0.1) - tox plugin for py-backwards tox-pytest-summary (0.1.2) - tox + Py.test summary tox-envreport (0.2.0) - A tox-plugin to document the setup of used virtual
environments. tox-no-internet (0.1.0) - Workarounds for using tox with no internet connection tox-virtualenv-no-download (1.0.2) - Disable virtualenv's download-by-default in tox tox-run-command (0.4) - tox plugin to run arbitrary commands in a virtualenv tox-pip-extensions (1.2.1) - Augment tox with different installation methods via
progressive enhancement. tox-run-before (0.1) - tox plugin to run shell commands before the test
environments are created. tox-docker (1.0.0) - Launch a docker instance around test runs tox-bitbucket-status (1.0) - Update bitbucket status for each env tox-pipenv-install (1.0.3) - Install packages from Pipfile
There might also be some plugins not (yet) available from PyPI that could be installed directly fom source hosters like Github or Bitbucket (or from a local clone). See the
To see what is installed you can call tox --version to get the version of the host and names and locations of all installed plugins:
3.0.0 imported from /home/ob/.virtualenvs/tmp/lib/python3.6/site-packages/tox/__init__.py registered plugins:
tox-travis-0.10 at /home/ob/.virtualenvs/tmp/lib/python3.6/site-packages/tox_travis/hooks.py
Start from a template
You can create a new tox plugin with all the bells and whistles via a Cookiecutter template (see cookiecutter-tox-plugin - this will create a complete PyPI-releasable, documented project with license, documentation and CI.
$ pip install -U cookiecutter $ cookiecutter gh:tox-dev/cookiecutter-tox-plugin
NOTE:
Let us consider you want to extend tox behaviour by displaying fireworks at the end of a successful tox run (we won't go into the details of how to display fireworks though).
To create a working plugin you need at least a python project with a tox entry point and a python module implementing one or more of the pluggy based hooks tox specifies (using the @tox.hookimpl decorator as marker).
minimal structure:
$ mkdir tox-fireworks $ cd tox-fireworks $ touch tox_fireworks.py $ touch setup.py
contents of tox_fireworks.py:
import pluggy
hookimpl = pluggy.HookimplMarker("tox")
@hookimpl
def tox_addoption(parser):
"""Add command line option to display fireworks on request."""
@hookimpl
def tox_configure(config):
"""Post process config after parsing."""
@hookimpl
def tox_runenvreport(config):
"""Display fireworks if all was fine and requested."""
NOTE:
contents of setup.py:
from setuptools import setup setup(
name="tox-fireworks",
py_modules=["tox_fireworks"],
entry_points={"tox": ["fireworks = tox_fireworks"]},
classifiers=["Framework:: tox"], )
Using the tox- prefix in tox-fireworks is an established convention to be able to see from the project name that this is a plugin for tox. It also makes it easier to find with e.g. pip search 'tox-' once it is released on PyPI.
To make your new plugin discoverable by tox, you need to install it. During development you should install it with -e or --editable, so that changes to the code are immediately active:
$ pip install -e </path/to/tox-fireworks>
If you think the rest of the world could profit using your plugin you can publish it to PyPI.
You need to add some more meta data to setup.py (see cookiecutter-tox-plugin for a complete example or consult the setup.py docs).
NOTE:
You can and publish it like:
$ cd </path/to/tox-fireworks> $ python setup.py sdist bdist_wheel upload
NOTE:
For more information about packaging and deploying Python projects see the Python Packaging Guide.
Hook specifications for tox - see https://pluggy.readthedocs.io/
Please be aware that the config object layout may change between major tox versions.
The first plugin/hook which returns an executable path will determine it.
envconfig is the testenv configuration which contains per-testenv configuration, notably the .envname and .basepython setting.
Called once for every environment.
This could be used for alternative (ie non-pip) package managers, this plugin should return a list of type str
NOTE:
This could be used to have per-venv test reporting of pass/fail status.
This could be used to indicate that tests for a given venv have started, for instance.
Some example usage:
NOTE:
NOTE:
Some example usage:
NOTE:
NOTE:
Types are specified as strings like "bool", "line-list", "string", "argv", "path", "argvlist".
The postprocess function will be called for each testenv like postprocess(testenv_config=testenv_config, value=value) where value is the value as read from the ini (or the default value) and testenv_config is a tox.config.TestenvConfig instance which will receive all ini-variables as object attributes.
Any postprocess function must return a value which will then be set as the final value in the testenv section.
This works as the add_testenv_attribute function but expects "name", "type", "help", and "postprocess" attributes on the object.
In addition to some core attributes/properties this config object holds all per-testenv ini attributes as attributes, see "tox --help-ini" for an overview.
NOTE: Only available during execution, not during parsing.
Pre 2.8.1 missing substitutions crashed with a ConfigError although this would not be a problem if the env is not part of the current testrun. So we need to remember this and check later when the testenv is actually run and crash only then.
This section contains information for users who want to extend the tox source code.
In order to run the unit tests locally all Python versions enlisted in tox.ini need to be installed.
NOTE:
One solution for this is to install the latest conda, and then install all Python versions via conda envs. This will create separate folders for each Python version.
conda create -n python2.7 python=2.7 anaconda
For tox to find them you'll need to:
@echo off REM python2.7.bat @D:\Anaconda\pkgs\python-2.7.13-1\python.exe %*
This way you can also directly call from cli the matching Python version if you need to(similarly to UNIX systems), for example:
python2.7 main.py python3.6 main.py
You can instruct tox to write a json-report file via:
tox --result-json=PATH
This will create a json-formatted result file using this schema:
{
"testenvs": {
"py27": {
"python": {
"executable": "/home/hpk/p/tox/.tox/py27/bin/python",
"version": "2.7.3 (default, Aug 1 2012, 05:14:39) \n[GCC 4.6.3]",
"version_info": [ 2, 7, 3, "final", 0 ]
},
"test": [
{
"output": "...",
"command": [
"/home/hpk/p/tox/.tox/py27/bin/pytest",
"--instafail",
"--junitxml=/home/hpk/p/tox/.tox/py27/log/junit-py27.xml",
"tests/test_config.py"
],
"retcode": "0"
}
],
"setup": []
}
},
"platform": "linux2",
"installpkg": {
"basename": "tox-1.6.0.dev1.zip",
"sha256": "b6982dde5789a167c4c35af0d34ef72176d0575955f5331ad04aee9f23af4326",
"md5": "27ead99fd7fa39ee7614cede6bf175a6"
},
"toxversion": "1.6.0.dev1",
"reportversion": "1"
}
With version 2.5.0 we dropped creating special announcement documents and rely on communicating all relevant changes through the CHANGELOG. See at PyPI for a rendered version of the last changes containing links to the important issues and pull requests that were integrated into the release.
The historic release announcements are still online here for various versions:
Happy testing, The tox maintainers
holger krekel and others
2010-2019, holger krekel and others
| February 16, 2019 | 3.7 |