FLIT(1) | Flit | FLIT(1) |
flit - Flit Documentation
Flit is a simple way to put Python packages and modules on PyPI. It tries to require less thought about packaging and help you avoid common mistakes. See Why use Flit? <https://flit.readthedocs.io/en/latest/rationale.html> for more about how it compares to other Python packaging tools.
$ python3 -m pip install flit
Flit requires Python 3 and therefore needs to be installed using the Python 3 version of pip.
Python 2 modules can be distributed using Flit, but need to be importable on Python 3 without errors.
Say you're writing a module foobar — either as a single file foobar.py, or as a directory — and you want to distribute it.
"""An amazing sample package!""" __version__ = "0.1"
python3 -m pip install flit
[build-system] requires = ["flit_core >=3.2,<4"] build-backend = "flit_core.buildapi" [project] name = "foobar" authors = [{name = "Sir Robin", email = "robin@camelot.uk"}] dynamic = ["version", "description"] [project.urls] Home = "https://github.com/sirrobin/foobar"
You can edit this file to add other metadata, for example to set up command line scripts. See the pyproject.toml page <https://flit.readthedocs.io/en/latest/pyproject_toml.html#scripts-section> of the documentation.
If you have already got a flit.ini file to use with older versions of Flit, convert it to pyproject.toml by running python3 -m flit.tomlify.
flit publish
Once your package is published, people can install it using pip just like any other package. In most cases, pip will download a 'wheel' package, a standard format it knows how to install. If you specifically ask pip to install an 'sdist' package, it will install and use Flit in a temporary environment.
To install a package locally for development, run:
flit install [--symlink] [--python path/to/python]
Flit packages a single importable module or package at a time, using the import name as the name on PyPI. All subpackages and data files within a package are included automatically.
This file lives next to the module or package.
NOTE:
Run python3 -m flit.tomlify to convert a flit.ini file to pyproject.toml.
This tells tools like pip to build your project with flit. It's a standard defined by PEP 517. For any new project using Flit, it will look like this:
[build-system] requires = ["flit_core >=3.2,<4"] build-backend = "flit_core.buildapi"
Version constraints:
New in version 3.2.
The new standard way to specify project metadata is in a [project] table, as defined by PEP 621 <https://peps.python.org/pep-0621/>. Flit works for now with either this or the older [tool.flit.metadata] table (described below), but it won't allow you to mix them.
A simple [project] table might look like this:
[project] name = "astcheck" authors = [
{name = "Thomas Kluyver", email = "thomas@kluyver.me.uk"}, ] readme = "README.rst" classifiers = [
"License :: OSI Approved :: MIT License", ] requires-python = ">=3.5" dynamic = ["version", "description"]
The allowed fields are:
Changed in version 3.8: Hyphens in the project name are now translated to underscores for the import name.
The dependencies field is a list of other packages from PyPI that this package needs. Each package may be followed by a version specifier like >=4.1, and/or an environment marker <https://www.python.org/dev/peps/pep-0508/#environment-markers> after a semicolon. For example:
dependencies = [
"requests >=2.6",
"configparser; python_version == '2.7'", ]
The [project.optional-dependencies] table contains lists of packages needed for every optional feature. The requirements are specified in the same format as for dependencies. For example:
[project.optional-dependencies] test = [
"pytest >=2.7.3",
"pytest-cov", ] doc = ["sphinx"]
You can call these optional features anything you want, although test and doc are common ones. You specify them for installation in square brackets after the package name or directory, e.g. pip install '.[test]'.
Your project's page on pypi.org <https://pypi.org/> can show a number of links. You can point people to documentation or a bug tracker, for example.
This section is called [project.urls] in the file. You can use any names inside it. Here it is for flit:
[project.urls] Documentation = "https://flit.pypa.io" Source = "https://github.com/pypa/flit"
This section is called [project.scripts] in the file. Each key and value describes a shell command to be installed along with your package. These work like setuptools 'entry points'. Here's the section for flit:
[project.scripts] flit = "flit:main"
This will create a flit command, which will call the function main() imported from flit.
A similar table called [project.gui-scripts] defines commands which launch a GUI. This only makes a difference on Windows, where GUI scripts are run without a console.
You can declare entry points <http://entrypoints.readthedocs.io/en/latest/> using sections named [project.entry-points.groupname]. E.g. to provide a pygments lexer from your package:
[project.entry-points."pygments.lexers"] dogelang = "dogelang.lexer:DogeLexer"
In each package:name value, the part before the colon should be an importable module name, and the latter part should be the name of an object accessible within that module. The details of what object to expose depend on the application you're extending.
If the group name contains a dot, it must be quoted ("pygments.lexers" above). Script entry points are defined in scripts tables, so you can't use the group names console_scripts or gui_scripts here.
If your package will have different names for installation and import, you should specify the install (PyPI) name in the [project] table (see above), and the import name in a [tool.flit.module] table:
[project] name = "pynsist" # ... [tool.flit.module] name = "nsist"
Flit looks for the source of the package by its import name. The source may be located either in the directory that holds the pyproject.toml file, or in a src/ subdirectory.
Flit's older way to specify metadata is in a [tool.flit.metadata] table, along with [tool.flit.scripts] and [tool.flit.entrypoints], described below. This is still recognised for now, but you can't mix it with New style metadata.
There are three required fields:
e.g. for flit itself
[tool.flit.metadata] module = "flit" author = "Thomas Kluyver" author-email = "thomas@kluyver.me.uk"
Changed in version 1.1: home-page was previously required.
The remaining fields are optional:
requires = [
"requests >=2.6",
"configparser; python_version == '2.7'", ]
[tool.flit.metadata.requires-extra] test = [
"pytest >=2.7.3",
"pytest-cov", ] doc = ["sphinx"]
New in version 1.1.
Here was the metadata section from flit using the older style:
[tool.flit.metadata] module="flit" author="Thomas Kluyver" author-email="thomas@kluyver.me.uk" home-page="https://github.com/pypa/flit" requires=[
"flit_core >=2.2.0",
"requests",
"docutils",
"tomli",
"tomli-w", ] requires-python=">=3.6" description-file="README.rst" classifiers=[
"Intended Audience :: Developers",
"License :: OSI Approved :: BSD License",
"Programming Language :: Python :: 3",
"Topic :: Software Development :: Libraries :: Python Modules", ]
Your project's page on pypi.org <https://pypi.org/> can show a number of links, in addition to the home-page URL described above. You can point people to documentation or a bug tracker, for example.
This section is called [tool.flit.metadata.urls] in the file. You can use any names inside it. Here it is for flit:
[tool.flit.metadata.urls] Documentation = "https://flit.pypa.io"
New in version 1.0.
A [tool.flit.scripts] table can be used along with [tool.flit.metadata]. It is in the same format as the newer [project.scripts] table described above.
[tool.flit.entrypoints] tables can be used along with [tool.flit.metadata]. They are in the same format as the newer [project.entry-points] tables described above.
New in version 2.0.
When you use flit build or flit publish, Flit builds an sdist (source distribution) tarball containing the files that are checked into version control (git or mercurial). If you want more control, or it doesn't recognise your version control system, you can give lists of paths or glob patterns as include and exclude in this section. For example:
[tool.flit.sdist] include = ["doc/"] exclude = ["doc/*.html"]
These paths:
Changed in version 3.8: Include and exclude patterns can now use recursive glob patterns (**).
Exclusions have priority over inclusions. Bytecode is excluded by default and cannot be included.
NOTE:
New in version 3.7.
Data files which your code will use should go inside the Python package folder. Flit will package these with no special configuration.
However, sometimes it's useful to package external files for system integration, such as man pages or files defining a Jupyter extension. To do this, arrange the files within a directory such as data, next to your pyproject.toml file, and add a section like this:
[tool.flit.external-data] directory = "data"
Paths within this directory are typically installed to corresponding paths under a prefix (such as a virtualenv directory). E.g. you might save a man page for a script as (data)/share/man/man1/foo.1.
Whether these files are detected by the systems they're meant to integrate with depends on how your package is installed and how those systems are configured. For instance, installing in a virtualenv usually doesn't affect anything outside that environment. Don't rely on these files being picked up unless you have close control of how the package will be installed.
If you install a package with flit install --symlink, a symlink is made for each file in the external data directory. Otherwise (including development installs with pip install -e), these files are copied to their destination, so changes here won't take effect until you reinstall the package.
NOTE:
All operations use the flit command, followed by one of a number of subcommands.
Build a wheel and an sdist (tarball) from the package.
Changed in version 3.5: Generating setup.py disabled by default.
Build a wheel and an sdist (tarball) from the package, and upload them to PyPI or another repository.
Changed in version 3.5: Generating setup.py disabled by default.
SEE ALSO:
Install the package on your system.
By default, the package is installed to the same Python environment that Flit itself is installed in; use --python or FLIT_INSTALL_PYTHON to override this.
If you don't have permission to modify the environment (e.g. the system Python on Linux), Flit may do a user install instead. Use the --user or --env flags to force this one way or the other, rather than letting Flit guess.
This can be useful for e.g. building a container image, where your own code is copied or mounted into the container at a later stage.
New in version 3.8.
Changed in version 2.1: Added FLIT_INSTALL_PYTHON and use its value over the Python running Flit when an explicit --python option is not given.
NOTE:
When you use the --symlink or --pth-file options, pip is used to install dependencies. Otherwise, Flit builds a wheel and then calls pip to install that.
Create a new pyproject.toml config file by prompting for information about the module in the current directory.
Setting this to any non-empty value will stop flit from making network connections (unless you explicitly ask to upload a package). This is intended for downstream packagers, so if you use this, it's up to you to ensure any necessary dependencies are installed.
Set a username, password, and index URL for uploading packages. See uploading packages with environment variables for more information.
Token-based upload to PyPI is supported. To upload using a PyPI token, set FLIT_USERNAME to __token__, and FLIT_PASSWORD to the token value.
Setting this to any non-empty value tells Flit to continue if it detects invalid metadata, instead of failing with an error. Problems will still be reported in the logs, but won't cause Flit to stop.
If the metadata is invalid, uploading the package to PyPI may fail. This environment variable provides an escape hatch in case Flit incorrectly rejects your valid metadata. If you need to use it and you believe your metadata is valid, please open an issue <https://github.com/pypa/flit/issues>.
Set a default Python interpreter for flit install to use when --python is not specified. The value can be either an absolute path, or a command name (which will be found in PATH). If this is unset or empty, the module is installed for the copy of Python that is running Flit.
date +%s
SEE ALSO:
The command flit publish will upload your package to a package index server. The default settings let you upload to PyPI <https://pypi.org/>, the default Python Package Index, with a single user account.
If you want to upload to other servers, or with more than one user account, or upload packages from a continuous integration job, you can configure Flit in two main ways:
You can create or edit a config file in your home directory, ~/.pypirc that will be used by default or you can specify a custom location. This is also used by other Python tools such as twine <https://pypi.python.org/pypi/twine>.
For instance, to upload a package to the Test PyPI server <https://test.pypi.org/> instead of the normal PyPI, use a config file looking like this:
[distutils] index-servers =
pypi
testpypi [pypi] repository = https://upload.pypi.org/legacy/ username = sirrobin # Replace with your PyPI username [testpypi] repository = https://test.pypi.org/legacy/ username = sirrobin # Replace with your TestPyPI username
You can select an index server from this config file with the --repository option:
flit publish --repository testpypi
If you don't use this option, Flit will use the server called pypi in the config file. If that doesn't exist, it uploads to PyPI at https://upload.pypi.org/legacy/ by default.
If you publish a package and you don't have a .pypirc file, Flit will create it to store your username.
Flit tries to store your password securely using the keyring <https://pypi.python.org/pypi/keyring> library. If keyring is not installed, Flit will ask for your password for each upload. Alternatively, you can also manually add your password to the .pypirc file (password = ...)
You can specify a server to upload to with FLIT_INDEX_URL, and pass credentials with FLIT_USERNAME and FLIT_PASSWORD. Environment variables take precedence over the config file, except if you use the --repository option to explicitly pick a server from the config file.
This can make it easier to automate uploads, for example to release packages from a continuous integration job.
WARNING:
New in version 0.8.
Wheels built by flit are reproducible: if you build from the same source code, you should be able to make wheels that are exactly identical, byte for byte. This is useful for verifying software. For more details, see reproducible-builds.org <https://reproducible-builds.org/>.
There is a caveat, however: wheels (which are zip files) include the modification timestamp from each file. This will probably be different on each computer, because it indicates when your local copy of the file was written, not when it was changed in version control. These timestamps can be overridden by the environment variable SOURCE_DATE_EPOCH.
SOURCE_DATE_EPOCH=$(date +%s) flit publish # Record the value of SOURCE_DATE_EPOCH in release notes for reproduction
Changed in version 0.12: Normalising permission bits
Flit normalises the permission bits of files copied into a wheel to either 755 (executable) or 644. This means that a file is readable by all users and writable only by the user who owns it.
The most popular version control systems only track the executable bit, so checking out the same repository on systems with different umasks (e.g. Debian and Fedora) produces files with different permissions. With Flit 0.11 and earlier, this difference would produce non-identical wheels.
Make the easy things easy and the hard things possible is an old motto from the Perl community. Flit is entirely focused on the easy things part of that, and leaves the hard things up to other tools.
Specifically, the easy things are pure Python packages with no build steps (neither compiling C code, nor bundling Javascript, etc.). The vast majority of packages on PyPI are like this: plain Python code, with maybe some static data files like icons included.
It's easy to underestimate the challenges involved in distributing and installing code, because it seems like you just need to copy some files into the right place. There's a whole lot of metadata and tooling that has to work together around that fundamental step. But with the right tooling, a developer who wants to release their code doesn't need to know about most of that.
What, specifically, does Flit make easy?
Setuptools, the most common tool for Python packaging, now has shortcuts for many of the same things. But it has to stay compatible with projects published many years ago, which limits what it can do by default.
Flit also has some support for reproducible builds, a feature which some people care about.
There have been many other efforts to improve the user experience of Python packaging, such as pbr <https://pypi.org/project/pbr/>, but before Flit, these tended to build on setuptools and distutils. That was a pragmatic decision, but it's hard to build something radically different on top of those libraries. The existence of Flit spurred the development of new standards, like PEP 518 <https://peps.python.org/pep-0518/> and PEP 517 <https://peps.python.org/pep-0517/>, which are now used by other packaging tools such as Poetry <https://python-poetry.org/> and Enscons <https://pypi.org/project/enscons/>.
If your package needs a build step, you won't be able to use Flit. Setuptools <https://setuptools.readthedocs.io/en/latest/> is the de-facto standard, but newer tools such as Enscons <https://pypi.org/project/enscons/> also cover this case.
Flit also doesn't help you manage dependencies: you have to add them to pyproject.toml by hand. Tools like Poetry <https://python-poetry.org/> and Pipenv <https://pypi.org/project/pipenv/> have features which help add and update dependencies on other packages.
Flit is itself packaged using Flit, as are some foundational packaging tools such as pep517. So where can you start if you need to install everything from source?
NOTE:
The key piece is flit_core. This is a package which can build itself using nothing except Python and the standard library. From an unpacked source archive, you can make a wheel by running:
python -m flit_core.wheel
And then you can install this wheel with the bootstrap_install.py script included in the sdist (or by unzipping it to the correct directory):
# Install to site-packages for this Python: python bootstrap_install.py dist/flit_core-*.whl # Install somewhere else: python bootstrap_install.py --installdir /path/to/site-packages dist/flit_core-*.whl
As of version 3.6, flit_core bundles the tomli TOML parser, to avoid a dependency cycle. If you need to unbundle it, you will need to special-case installing flit_core and/or tomli to get around that cycle.
After flit_core, I recommend that you get installer <https://pypi.org/project/installer/> set up. You can use python -m flit_core.wheel again to make a wheel, and then use installer itself (from the source directory) to install it.
After that, you probably want to get build <https://pypi.org/project/build/> and its dependencies installed as the goal of the bootstrapping phase. You can then use build to create wheels of any other Python packages, and installer to install them.
To get a development installation of Flit itself:
git clone https://github.com/pypa/flit.git cd flit python3 -m pip install docutils requests python3 bootstrap_dev.py
This links Flit into the current Python environment, so you can make changes and try them without having to reinstall each time.
To run the tests in separate environments for each available Python version:
tox
tox <https://tox.readthedocs.io/en/latest/> has many options.
To run the tests in your current environment, run:
pytest
:ghpull:`566`).
:ghpull:`550`).
:ghpull:`581`).
:ghpull:`573`,
:ghpull:`604`).
:ghpull:`567`).
:ghpull:`541`).
:ghpull:`593`).
:ghpull:`474`).
:ghpull:`602`).
:ghpull:`576`).
:ghpull:`589`).
:ghpull:`601`).
:ghpull:`603`).
:ghpull:`531`).
:ghpull:`510`).
:ghpull:`498`).
:ghpull:`511`). flit_core.wheel is usable with python -m to create wheels before the build <https://pypi.org/project/build/> tool is available, and flit_core sdists also include a script to install itself from a wheel before installer <https://pypi.org/project/installer/> is available.
:ghpull:`499`).
:ghpull:`492`). This means flit_core now has no dependencies except Python itself, both at build time and at runtime, simplifying bootstrapping.
:ghpull:`472`).
:ghpull:`468`).
:ghpull:`462`). Modern packaging tools don't need this. You can use the --setup-py flag to keep adding it for now, but this will probably be removed at some point in the future.
:ghpull:`460`).
:ghpull:`467`).
:ghpull:`431`). A future version will stop generating setup.py files in sdists by default.
:ghpull:`400`).
:ghpull:`434`).
:ghpull:`433`).
:ghpull:`448`).
:ghpull:`438`). This supports TOML version 1.0.
:ghpull:`441`).
:ghpull:`410`).
:ghpull:`416`).
:ghpull:`393`). If you try using this, please specify requires = ["flit_core >=3.2.0,<3.3"] in the [build-system] table for now, in case it needs to change for the next release.
:ghpull:`402`).
:ghpull:`401`).
:ghpull:`395`).
:ghpull:`378`).
Breaking changes:
:ghpull:`338`).
:ghpull:`342`).
:ghpull:`334`). You can use pip install git+https://github.com/... instead.
Features and fixes:
:ghpull:`346`).
:ghpull:`348`).
:ghpull:`359`).
:ghpull:`256`).
:ghpull:`368`).
:ghpull:`335`).
:ghpull:`337`).
:ghpull:`357`).
:ghpull:`361`).
:ghpull:`366`).
:ghpull:`328`). Any projects using pyproject.toml (not flit.ini) should be compatible with flit 3.x.
:ghpull:`324`).
:ghpull:`319`).
:ghpull:`331`).
:ghpull:`322`). Passing the option like this is deprecated, and you should now pass it after upload.
:ghpull:`327`,
:ghpull:`318`,
:ghpull:`314`)
:ghpull:`305`).
:ghpull:`311`).
:ghpull:`303`).
:ghpull:`307`).
:ghpull:`300`).
:ghpull:`310`).
:ghpull:`301`,
:ghpull:`306`).
:ghpull:`295`), to configure flit to always install into a Python other than the one it's running on.
Flit 2 is a major architecture change. The flit_core package now provides a PEP 517 <https://peps.python.org/pep-0517/> backend for building packages, while flit is a command line interface extending that.
The build backend works on Python 2, so tools like pip should be able to install packages built with flit from source on Python 2. The flit command requires Python 3.5 or above. You will need to change the build-system table in your pyproject.toml file to look like this:
[build-system] requires = ["flit_core >=2,<4"] build-backend = "flit_core.buildapi"
Other changes include:
:ghpull:`260`). You don't need to change any configuration if you do this.
:ghpull:`286`).
:ghpull:`289`).
:ghpull:`254`)
:ghpull:`233`).
:ghpull:`239`).
:ghpull:`221`).
:ghissue:`228`).
:ghpull:`232`).
:ghpull:`230`).
:ghpull:`234`).
:ghpull:`240`).
:ghpull:`229`).
Although version 1.0 sounds like a milestone, there's nothing that makes this release especially significant. It doesn't represent a step change in stability or completeness. Flit has been gradually maturing for some time, and I chose this point to end the series of 0.x version numbers.
python3 -m flit.tomlify
Thomas Kluyver
2023, Thomas Kluyver
January 4, 2023 | 3.8.0 |