| PIPENV(1) | pipenv | PIPENV(1) |
pipenv - pipenv Documentation
----
Pipenv — the officially recommended Python packaging tool from Python.org, free (as in freedom).
Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world. Windows is a first–class citizen, in our world.
It automatically creates and manages a virtualenv for your projects, as well as adds/removes packages from your Pipfile as you install/uninstall packages. It also generates the ever–important Pipfile.lock, which is used to produce deterministic builds.
The problems that Pipenv seeks to solve are multi-faceted:
$ pip install pipenv ✨🍰✨
If you have excellent taste, there's also a fancy installation method.
[image]
This tutorial walks you through installing and using Python packages.
It will show you how to install and use the necessary tools and make strong recommendations on best practices. Keep in mind that Python is used for a great many different purposes, and precisely how you want to manage your dependencies may change based on how you decide to publish your software. The guidance presented here is most directly applicable to the development and deployment of network services (including web applications), but is also very well suited to managing development and testing environments for any kind of project.
NOTE:
Before you go any further, make sure you have Python and that it's available from your command line. You can check this by simply running:
$ python --version
You should get some output like 3.6.2. If you do not have Python, please install the latest 3.x version from python.org or refer to the Installing Python section of The Hitchhiker's Guide to Python.
NOTE:
>>> python Traceback (most recent call last):
File "<stdin>", line 1, in <module> NameError: name 'python' is not defined
It's because this command is intended to be run in a shell (also called a terminal or console). See the Python for Beginners getting started tutorial for an introduction to using your operating system's shell and interacting with Python.
Additionally, you'll need to make sure you have pip available. You can check this by running:
$ pip --version pip 9.0.1
If you installed Python from source, with an installer from python.org, or via Homebrew you should already have pip9. If you're on Linux and installed using your OS package manager, you may have to install pip separately.
Pipenv is a dependency manager for Python projects. If you're familiar with Node.js' npm or Ruby's bundler, it is similar in spirit to those tools. While pip can install Python packages, Pipenv is recommended as it's a higher-level tool that simplifies dependency management for common use cases.
Use pip to install Pipenv:
$ pip install --user pipenv
NOTE:
On Linux and macOS you can find the user base binary directory by running python -m site --user-base and adding bin to the end. For example, this will typically print ~/.local (with ~ expanded to the absolute path to your home directory) so you'll need to add ~/.local/bin to your PATH. You can set your PATH permanently by modifying ~/.profile.
On Windows you can find the user base binary directory by running py -m site --user-site and replacing site-packages with Scripts. For example, this could return C:\Users\Username\AppData\Roaming\Python36\site-packages so you would need to set your PATH to include C:\Users\Username\AppData\Roaming\Python36\Scripts. You can set your user PATH permanently in the Control Panel. You may need to log out for the PATH changes to take effect.
Pipenv manages dependencies on a per-project basis. To install packages, change into your project's directory (or just an empty directory for this tutorial) and run:
$ cd myproject $ pipenv install requests
Pipenv will install the excellent Requests library and create a Pipfile for you in your project's directory. The Pipfile is used to track which dependencies your project needs in case you need to re-install them, such as when you share your project with others. You should get output similar to this (although the exact paths shown will vary):
Creating a Pipfile for this project... Creating a virtualenv for this project... Using base prefix '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6' New python executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python3.6 Also creating executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python Installing setuptools, pip, wheel...done. Virtualenv location: ~/.local/share/virtualenvs/tmp-agwWamBd Installing requests... Collecting requests
Using cached requests-2.18.4-py2.py3-none-any.whl Collecting idna<2.7,>=2.5 (from requests)
Using cached idna-2.6-py2.py3-none-any.whl Collecting urllib3<1.23,>=1.21.1 (from requests)
Using cached urllib3-1.22-py2.py3-none-any.whl Collecting chardet<3.1.0,>=3.0.2 (from requests)
Using cached chardet-3.0.4-py2.py3-none-any.whl Collecting certifi>=2017.4.17 (from requests)
Using cached certifi-2017.7.27.1-py2.py3-none-any.whl Installing collected packages: idna, urllib3, chardet, certifi, requests Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.6 requests-2.18.4 urllib3-1.22 Adding requests to Pipfile's [packages]... P.S. You have excellent taste! ✨ 🍰 ✨
Now that Requests is installed you can create a simple main.py file to use it:
import requests
response = requests.get('https://httpbin.org/ip')
print('Your IP is {0}'.format(response.json()['origin']))
Then you can run this script using pipenv run:
$ pipenv run python main.py
You should get output similar to this:
Your IP is 8.8.8.8
Using $ pipenv run ensures that your installed packages are available to your script. It's also possible to spawn a new shell that ensures all commands have access to your installed packages with $ pipenv shell.
Congratulations, you now know how to install and use Python packages! ✨ 🍰 ✨
To install pipenv in a fancy way, we recommend using pipsi.
Pipsi is a powerful tool which allows you to install Python scripts into isolated virtual environments.
To install pipsi, first run this:
$ curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python
Follow the instructions, you'll have to update your PATH.
Then, simply run:
$ pipsi install pew $ pipsi install pipenv
To upgrade pipenv at any time:
$ pipsi upgrade pipenv
This will install both pipenv and pew (one of our dependencies) in an isolated virtualenv, so it doesn't interfere with the rest of your Python installation!
If you have a working installation of pip, and maintain certain "toolchain" type Python modules as global utilities in your user environment, pip user installs allow for installation into your home directory. Note that due to interaction between dependencies, you should limit tools installed in this way to basic building blocks for a Python workflow like virtualenv, pipenv, tox, and similar software.
To install:
$ pip install --user pipenv
For more information see the user installs documentation, but to add the installed cli tools from a pip user install to your path, add the output of:
$ python -c "import site; import os; print(os.path.join(site.USER_BASE, 'bin'))"
To upgrade pipenv at any time:
$ pip install --user --upgrade pipenv
If you don't even have pip installed, you can use this crude installation method, which will bootstrap your whole system:
$ curl https://raw.githubusercontent.com/kennethreitz/pipenv/master/get-pipenv.py | python
Congratulations, you now have pip and Pipenv installed!
The main commands are install, uninstall, and lock, which generates a Pipfile.lock. These are intended to replace $ pip install usage, as well as manual virtualenv management (to activate a virtualenv, run $ pipenv shell).
[image]
This document covers some of Pipenv's more basic features.
Here is a simple example of a Pipfile and the resulting Pipfile.lock.
[[source]] url = "https://pypi.python.org/simple" verify_ssl = true name = "pypi" [packages] requests = "*" [dev-packages] pytest = "*"
{
"_meta": {
"hash": {
"sha256": "8d14434df45e0ef884d6c3f6e8048ba72335637a8631cc44792f52fd20b6f97a"
},
"host-environment-markers": {
"implementation_name": "cpython",
"implementation_version": "3.6.1",
"os_name": "posix",
"platform_machine": "x86_64",
"platform_python_implementation": "CPython",
"platform_release": "16.7.0",
"platform_system": "Darwin",
"platform_version": "Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64",
"python_full_version": "3.6.1",
"python_version": "3.6",
"sys_platform": "darwin"
},
"pipfile-spec": 5,
"requires": {},
"sources": [
{
"name": "pypi",
"url": "https://pypi.python.org/simple",
"verify_ssl": true
}
]
},
"default": {
"certifi": {
"hashes": [
"sha256:54a07c09c586b0e4c619f02a5e94e36619da8e2b053e20f594348c0611803704",
"sha256:40523d2efb60523e113b44602298f0960e900388cf3bb6043f645cf57ea9e3f5"
],
"version": "==2017.7.27.1"
},
"chardet": {
"hashes": [
"sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691",
"sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae"
],
"version": "==3.0.4"
},
"idna": {
"hashes": [
"sha256:8c7309c718f94b3a625cb648ace320157ad16ff131ae0af362c9f21b80ef6ec4",
"sha256:2c6a5de3089009e3da7c5dde64a141dbc8551d5b7f6cf4ed7c2568d0cc520a8f"
],
"version": "==2.6"
},
"requests": {
"hashes": [
"sha256:6a1b267aa90cac58ac3a765d067950e7dbbf75b1da07e895d1f594193a40a38b",
"sha256:9c443e7324ba5b85070c4a818ade28bfabedf16ea10206da1132edaa6dda237e"
],
"version": "==2.18.4"
},
"urllib3": {
"hashes": [
"sha256:06330f386d6e4b195fbfc736b297f58c5a892e4440e54d294d7004e3a9bbea1b",
"sha256:cc44da8e1145637334317feebd728bd869a35285b93cbb4cca2577da7e62db4f"
],
"version": "==1.22"
}
},
"develop": {
"py": {
"hashes": [
"sha256:2ccb79b01769d99115aa600d7eed99f524bf752bba8f041dc1c184853514655a",
"sha256:0f2d585d22050e90c7d293b6451c83db097df77871974d90efd5a30dc12fcde3"
],
"version": "==1.4.34"
},
"pytest": {
"hashes": [
"sha256:b84f554f8ddc23add65c411bf112b2d88e2489fd45f753b1cae5936358bdf314",
"sha256:f46e49e0340a532764991c498244a60e3a37d7424a532b3ff1a6a7653f1a403a"
],
"version": "==3.2.2"
}
}
}
If you only have a requirements.txt file available when running pipenv install, pipenv will automatically import the contents of this file and create a Pipfile for you.
You can also specify $ pipenv install -r path/to/requirements.txt to import a requirements file.
Note, that when importing a requirements file, they often have version numbers pinned, which you likely won't want in your Pipfile, so you'll have to manually update your Pipfile afterwards to reflect this.
To tell pipenv to install a specific version of a library, the usage is simple:
$ pipenv install requests==2.13.0
This will update your Pipfile to reflect this requirement, automatically.
To create a new virtualenv, using a specific version of Python you have installed (and on your PATH), use the --python VERSION flag, like so:
Use Python 3:
$ pipenv --python 3
Use Python3.6:
$ pipenv --python 3.6
Use Python 2.7.14:
$ pipenv --python 2.7.14
When given a Python version, like this, Pipenv will automatically scan your system for a Python that matches that given version.
If a Pipfile hasn't been created yet, one will be created for you, that looks like this:
[[source]] url = "https://pypi.python.org/simple" verify_ssl = true [dev-packages] [packages] [requires] python_version = "3.6"
Note the inclusion of [requires] python_version = "3.6". This specifies that your application requires this version of Python, and will be used automatically when running pipenv install against this Pipfile in the future (e.g. on other machines). If this is not true, feel free to simply remove this section.
If you don't specify a Python version on the command–line, either the [requires] python_full_version or python_version will be selected automatically, falling back to whatever your system's default python installation is, at time of execution.
You can tell Pipenv to install a path as editable — often this is useful for the current working directory when working on packages:
$ pipenv install '-e .' --dev
$ cat Pipfile
[dev-packages]
"e1839a8" = {path = ".", editable = true}
Note that all sub-dependencies will get added to the Pipfile.lock as well.
The three primary commands you'll use in managing your pipenv environment are $ pipenv install, $ pipenv uninstall, and $ pipenv lock.
$ pipenv install is used for installing packages into the pipenv virtual environment and updating your Pipfile.
Along with the basic install command, which takes the form:
$ pipenv install [package names]
The user can provide these additional parameters:
WARNING:
NOTE:
$ pipenv uninstall supports all of the parameters in pipenv install, as well as one additional, --all.
$ pipenv lock is used to create a Pipfile.lock, which declares all dependencies (and sub-dependencies) of your project, their latest available versions, and the current hashes for the downloaded files. This ensures repeatable, and most importantly deterministic, builds.
Shells are typically misconfigured for subshell use, so $ pipenv shell --fancy may produce unexpected results. If this is the case, try $ pipenv shell, which uses "compatibility mode", and will attempt to spawn a subshell despite misconfiguration.
A proper shell configuration only sets environment variables like PATH during a login session, not during every subshell spawn (as they are typically configured to do). In fish, this looks like this:
if status --is-login
set -gx PATH /usr/local/bin $PATH end
You should do this for your shell too, in your ~/.profile or ~/.bashrc or wherever appropriate.
NOTE:
Pipenv will resolve the sub–dependencies of VCS dependencies, but only if they are editable, like so:
[packages]
requests = {git = "https://github.com/requests/requests.git", editable=true}
If editable is not true, sub–dependencies will not get resolved.
Pipfile.lock takes advantage of some great new security improvements in pip. By default, the Pipfile.lock will be generated with the sha256 hashes of each downloaded package. This will allow pip to guarantee you're installing what you intend to when on a compromised network, or downloading dependencies from an untrusted PyPI endpoint.
We highly recommend approaching deployments with promoting projects from a development environment into production. You can use pipenv lock to compile your dependencies on your development environment and deploy the compiled Pipfile.lock to all of your production environments for reproducible builds.
[image]
This document covers some of Pipenv's more glorious and advanced features.
If you'd like a specific package to be installed with a specific package index, you can do the following:
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"
[[source]]
url = "http://pypi.home.kennethreitz.org/simple"
verify_ssl = false
name = "home"
[dev-packages]
[packages]
requests = {version="*", index="home"}
maya = {version="*", index="pypi"}
records = "*"
Very fancy.
If you'd like to specify that a specific package only be installed on certain systems, you can use PEP 508 specifiers to accomplish this.
Here's an example Pipfile, which will only install pywinusb on Windows systems:
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"
[packages]
requests = "*"
pywinusb = {version = "*", os_name = "== 'windows'"}
Voilà!
Here's a more complex example:
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
[packages]
unittest2 = {version = ">=1.0,<3.0", markers="python_version < '2.7.9' or (python_version >= '3.0' and python_version < '3.4')"}
Magic. Pure, unadulterated magic.
You can tell Pipenv to install things into its parent system with the --system flag:
$ pipenv install --system
This is useful for Docker containers, and deployment infrastructure (e.g. Heroku does this).
Also useful for deployment is the --deploy flag:
$ pipenv install --system --deploy
This will fail a build if the Pipfile.lock is out–of–date, instead of generating a new one.
You can convert a Pipfile and Pipenv.lock into a requirements.txt file very easily, and get all the benefits of hashes, extras, and other goodies we have included.
Let's take this Pipfile:
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
[packages]
requests = {version="*"}
And generate a requirements.txt out of it:
$ pipenv lock -r chardet==3.0.4 --hash=sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691 --hash=sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae requests==2.18.4 --hash=sha256:6a1b267aa90cac58ac3a765d067950e7dbbf75b1da07e895d1f594193a40a38b --hash=sha256:9c443e7324ba5b85070c4a818ade28bfabedf16ea10206da1132edaa6dda237e certifi==2017.7.27.1 --hash=sha256:54a07c09c586b0e4c619f02a5e94e36619da8e2b053e20f594348c0611803704 --hash=sha256:40523d2efb60523e113b44602298f0960e900388cf3bb6043f645cf57ea9e3f5 idna==2.6 --hash=sha256:8c7309c718f94b3a625cb648ace320157ad16ff131ae0af362c9f21b80ef6ec4 --hash=sha256:2c6a5de3089009e3da7c5dde64a141dbc8551d5b7f6cf4ed7c2568d0cc520a8f urllib3==1.22 --hash=sha256:06330f386d6e4b195fbfc736b297f58c5a892e4440e54d294d7004e3a9bbea1b --hash=sha256:cc44da8e1145637334317feebd728bd869a35285b93cbb4cca2577da7e62db4f
Very fancy.
Pipenv includes the safety package, and will use it to scan your dependency graph for known security vulnerabilities!
Example:
$ cat Pipfile [packages] django = "==1.10.1" $ pipenv check Checking PEP 508 requirements… Passed! Checking installed package safety… 33075: django >=1.10,<1.10.3 resolved (1.10.1 installed)! Django before 1.8.x before 1.8.16, 1.9.x before 1.9.11, and 1.10.x before 1.10.3, when settings.DEBUG is True, allow remote attackers to conduct DNS rebinding attacks by leveraging failure to validate the HTTP Host header against settings.ALLOWED_HOSTS. 33076: django >=1.10,<1.10.3 resolved (1.10.1 installed)! Django 1.8.x before 1.8.16, 1.9.x before 1.9.11, and 1.10.x before 1.10.3 use a hardcoded password for a temporary database user created when running tests with an Oracle database, which makes it easier for remote attackers to obtain access to the database server by leveraging failure to manually specify a password in the database settings TEST dictionary. 33300: django >=1.10,<1.10.7 resolved (1.10.1 installed)! CVE-2017-7233: Open redirect and possible XSS attack via user-supplied numeric redirect URLs ============================================================================================ Django relies on user input in some cases (e.g. :func:`django.contrib.auth.views.login` and :doc:`i18n </topics/i18n/index>`) to redirect the user to an "on success" URL. The security check for these redirects (namely ``django.utils.http.is_safe_url()``) considered some numeric URLs (e.g. ``http:999999999``) "safe" when they shouldn't be. Also, if a developer relies on ``is_safe_url()`` to provide safe redirect targets and puts such a URL into a link, they could suffer from an XSS attack. CVE-2017-7234: Open redirect vulnerability in ``django.views.static.serve()`` ============================================================================= A maliciously crafted URL to a Django site using the :func:`~django.views.static.serve` view could redirect to any other domain. The view no longer does any redirects as they don't provide any known, useful functionality. Note, however, that this view has always carried a warning that it is not hardened for production use and should be used only as a development aid.
✨🍰✨
Pipenv has Flake 8 built into it. You can check the style of your code like so, without installing anything:
$ cat t.py import requests $ pipenv check --style t.py t.py:1:1: F401 'requests' imported but unused t.py:1:16: W292 no newline at end of file
Super useful :)
Pipenv allows you to open any Python module that is installed (including ones in your codebase), with the $ pipenv open command:
$ pipenv install -e git+https://github.com/kennethreitz/background.git#egg=background Installing -e git+https://github.com/kennethreitz/background.git#egg=background… ... Updated Pipfile.lock! $ pipenv open background Opening '/Users/kennethreitz/.local/share/virtualenvs/hmm-mGOawwm_/src/background/background.py' in your EDITOR.
This allows you to easily read the code you're consuming, instead of looking it up on GitHub.
NOTE:
If you have pyenv installed and configured, Pipenv will automatically ask you if you want to install a required version of Python if you don't already have it available.
This is a very fancy feature, and we're very proud of it:
$ cat Pipfile [[source]] url = "https://pypi.python.org/simple" verify_ssl = true [dev-packages] [packages] requests = "*" [requires] python_version = "3.6" $ pipenv install Warning: Python 3.6 was not found on your system… Would you like us to install latest CPython 3.6 with pyenv? [Y/n]: y Installing CPython 3.6.2 with pyenv (this may take a few minutes)… ... Making Python installation global… Creating a virtualenv for this project… Using /Users/kennethreitz/.pyenv/shims/python3 to create virtualenv… ... No package provided, installing all dependencies. ... Installing dependencies from Pipfile.lock… 🐍 ❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒ 5/5 — 00:00:03 To activate this project's virtualenv, run the following:
$ pipenv shell
Pipenv automatically honors both the python_full_version and python_version PEP 508 specifiers.
💫✨🍰✨💫
If a .env file is present in your project, $ pipenv shell and $ pipenv run will automatically load it, for you:
$ cat .env HELLO=WORLD⏎ $ pipenv run python Loading .env environment variables… Python 2.7.13 (default, Jul 18 2017, 09:17:00) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.environ['HELLO'] 'WORLD'
This is very useful for keeping production credentials out of your codebase. We do not recommend committing .env files into source control!
If your .env file is located in a different path or has a different name you may set the PIPENV_DOTENV_LOCATION environment variable:
$ PIPENV_DOTENV_LOCATION=/path/to/.env pipenv shell
pipenv comes with a handful of options that can be enabled via shell environment variables. To activate them, simply create the variable in your shell and pipenv will detect it.
Also note that pip itself supports environment variables, if you need additional customization.
For example:
$ PIP_INSTALL_OPTION="-- -DCMAKE_BUILD_TYPE=Release" pipenv install -e .
Pipenv's underlying pew dependency will automatically honor the WORKON_HOME environment variable, if you have it set — so you can tell pipenv to store your virtual environments wherever you want, e.g.:
export WORKON_HOME=~/.venvs
In addition, you can also have Pipenv stick the virtualenv in project/.venv by setting the PIPENV_VENV_IN_PROJECT environment variable.
Pipenv is being used in projects like Requests for declaring development dependencies and running the test suite.
We've currently tested deployments with both Travis-CI and tox with success.
An example Travis CI setup can be found in Requests. The project uses a Makefile to define common functions such as its init and tests commands. Here is a stripped down example .travis.yml:
language: python python:
- "2.6"
- "2.7"
- "3.3"
- "3.4"
- "3.5"
- "3.6"
- "3.7dev" # command to install dependencies install: "make" # command to run tests script:
- make test
and the corresponding Makefile:
init:
pip install pipenv
pipenv install --dev test:
pipenv run py.test tests
Alternatively, you can configure a tox.ini like the one below for both local and external testing:
[tox] envlist = flake8-py3, py26, py27, py33, py34, py35, py36, pypy [testenv] passenv=HOME deps = pipenv commands=
pipenv install --dev
pipenv run py.test tests [testenv:flake8-py3] passenv=HOME basepython = python3.4 commands=
{[testenv]deps}
pipenv install --dev
pipenv run flake8 --version
pipenv run flake8 setup.py docs project test
NOTE:
To enable completion in fish, add this to your config:
eval (pipenv --completion)
Magic shell completions are now enabled!
✨🍰✨
It's reasonably common for platform specific Python bindings for operating system interfaces to only be available through the system package manager, and hence unavailable for installation into virtual environments with pip. In these cases, the virtual environment can be created with access to the system site-packages directory:
$ pipenv --three --site-packages
To ensure that all pip-installable components actually are installed into the virtual environment and system packages are only used for interfaces that don't participate in Python-level dependency resolution at all, use the PIP_IGNORE_INSTALLED setting:
$ PIP_IGNORE_INSTALLED=1 pipenv install --dev
Kenneth Reitz
2017. A <a href="http://kennethreitz.com/pages/open-projects.html">Kenneth Reitz</a> Project
| October 7, 2017 | 8.2.7 |