gevent.subprocess – Cooperative subprocess module#

Cooperative subprocess module.

Caution

On POSIX platforms, this module is not usable from native threads other than the main thread; attempting to do so will raise a TypeError. This module depends on libev’s fork watchers. On POSIX systems, fork watchers are implemented using signals, and the thread to which process-directed signals are delivered is not defined. Because each native thread has its own gevent/libev loop, this means that a fork watcher registered with one loop (thread) may never see the signal about a child it spawned if the signal is sent to a different thread.

Note

The interface of this module is intended to match that of the standard library subprocess module (with many backwards compatible extensions from Python 3 backported to Python 2). There are some small differences between the Python 2 and Python 3 versions of that module (the Python 2 TimeoutExpired exception, notably, extends Timeout and there is no SubprocessError) and between the POSIX and Windows versions. The HTML documentation here can only describe one version; for definitive documentation, see the standard library or the source code.

exception CalledProcessError(returncode, cmd, output=None, stderr=None)[source]#

Bases: SubprocessError

Raised when run() is called with check=True and the process returns a non-zero exit status.

Attributes:

cmd, returncode, stdout, stderr, output

property stdout#

Alias for output attribute, to match stderr

exception SubprocessError[source]#

Bases: Exception

exception TimeoutExpired(cmd, timeout, output=None, stderr=None)[source]#

Bases: SubprocessError

This exception is raised when the timeout expires while waiting for a child process.

Attributes:

cmd, output, stdout, stderr, timeout

class CompletedProcess(args, returncode, stdout=None, stderr=None)[source]#

Bases: object

A process that has finished running.

This is returned by run().

Attributes:

  • args: The list or str args passed to run().

  • returncode: The exit code of the process, negative for signals.

  • stdout: The standard output (None if not captured).

  • stderr: The standard error (None if not captured).

New in version 1.2a1: This first appeared in Python 3.5 and is available to all Python versions in gevent.

check_returncode()[source]#

Raise CalledProcessError if the exit code is non-zero.

class Popen(args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=<object object>, shell=False, cwd=None, env=None, universal_newlines=None, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), encoding=None, errors=None, text=None, group=None, extra_groups=None, user=None, umask=-1, pipesize=-1, process_group=None, threadpool=None)[source]#

Bases: object

The underlying process creation and management in this module is handled by the Popen class. It offers a lot of flexibility so that developers are able to handle the less common cases not covered by the convenience functions.

See also

subprocess.Popen This class should have the same interface as the standard library class.

Caution

The default values of some arguments, notably buffering, differ between Python 2 and Python 3. For the most consistent behaviour across versions, it’s best to explicitly pass the desired values.

Caution

On Python 2, the read method of the stdout and stderr attributes will not be buffered unless buffering is explicitly requested (e.g., bufsize=-1). This is different than the read method of the standard library attributes, which will buffer internally even if no buffering has been requested. This matches the Python 3 behaviour. For portability, please explicitly request buffering if you want read(n) to return all n bytes, making more than one system call if needed. See issue 1701 for more context.

Changed in version 1.2a1: Instances can now be used as context managers under Python 2.7. Previously this was restricted to Python 3.

Changed in version 1.2a1: Instances now save the args attribute under Python 2.7. Previously this was restricted to Python 3.

Changed in version 1.2b1: Add the encoding and errors parameters for Python 3.

Changed in version 1.3a1: Accept “path-like” objects for the cwd parameter on all platforms. This was added to Python 3.6. Previously with gevent, it only worked on POSIX platforms on 3.6.

Changed in version 1.3a1: Add the text argument as a synonym for universal_newlines, as added on Python 3.7.

Changed in version 1.3a2: Allow the same keyword arguments under Python 2 as Python 3: pass_fds, start_new_session, restore_signals, encoding and errors. Under Python 2, encoding and errors are ignored because native handling of universal newlines is used.

Changed in version 1.3a2: Under Python 2, restore_signals defaults to False. Previously it defaulted to True, the same as it did in Python 3.

Changed in version 20.6.0: Add the group, extra_groups, user, and umask arguments. These were added to Python 3.9, but are available in any gevent version, provided the underlying platform support is present.

Changed in version 20.12.0: On Python 2 only, if unbuffered binary communication is requested, the stdin attribute of this object will have a write method that actually performs internal buffering and looping, similar to the standard library. It guarantees to write all the data given to it in a single call (but internally it may make many system calls and/or trips around the event loop to accomplish this). See issue #1711.

Changed in version 21.12.0: Added the pipesize argument for compatibility with Python 3.10. This is ignored on all platforms.

Changed in version 22.08.0: Added the process_group and check arguments for compatibility with Python 3.11.

communicate(input=None, timeout=None)[source]#

Interact with process and return its output and error.

  • Send input data to stdin.

  • Read data from stdout and stderr, until end-of-file is reached.

  • Wait for process to terminate.

The optional input argument should be a string to be sent to the child process, or None, if no data should be sent to the child.

communicate() returns a tuple (stdout, stderr).

Parameters:

timeout – Under Python 2, this is a gevent extension; if given and it expires, we will raise TimeoutExpired, which extends gevent.timeout.Timeout (note that this only extends BaseException, not Exception) Under Python 3, this raises the standard TimeoutExpired exception.

Changed in version 1.1a2: Under Python 2, if the timeout elapses, raise the gevent.timeout.Timeout exception. Previously, we silently returned.

Changed in version 1.1b5: Honor a timeout even if there’s no way to communicate with the child (stdin, stdout, and stderr are not pipes).

kill()[source]#

Kill the process with SIGKILL

pipe_cloexec()[source]#

Create a pipe with FDs set CLOEXEC.

poll()[source]#

Check if child process has terminated. Set and return returncode attribute.

send_signal(sig)[source]#

Send a signal to the process

terminate()[source]#

Terminate the process with SIGTERM

wait(timeout=None, _raise_exc=True)[source]#

Wait for child process to terminate. Returns returncode attribute.

Parameters:

timeout – The floating point number of seconds to wait. Under Python 2, this is a gevent extension, and we simply return if it expires. Under Python 3, if this time elapses without finishing the process, TimeoutExpired is raised.

call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None) → returncode[source]#

Run command with arguments. Wait for command to complete or timeout, then return the returncode attribute.

The arguments are the same as for the Popen constructor. Example:

retcode = call(["ls", "-l"])

Changed in version 1.2a1: The timeout keyword argument is now accepted on all supported versions of Python (not just Python 3) and if it expires will raise a TimeoutExpired exception (under Python 2 this is a subclass of Timeout).

check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None) → 0[source]#

Run command with arguments. Wait for command to complete. If the exit code was zero then return, otherwise raise CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute.

The arguments are the same as for the Popen constructor. Example:

retcode = check_call(["ls", "-l"])
check_output(args, *, input=None, stdin=None, stderr=None, shell=False, universal_newlines=False, timeout=None) → output[source]#

Run command with arguments and return its output.

If the exit code was non-zero it raises a CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute and output in the output attribute.

The arguments are the same as for the Popen constructor. Example:

>>> check_output(["ls", "-1", "/dev/null"])
'/dev/null\n'

The stdout argument is not allowed as it is used internally.

To capture standard error in the result, use stderr=STDOUT:

>>> output = check_output(["/bin/sh", "-c",
...               "ls -l non_existent_file ; exit 0"],
...              stderr=STDOUT).decode('ascii').strip()
>>> print(output.rsplit(':', 1)[1].strip())
No such file or directory

There is an additional optional argument, “input”, allowing you to pass a string to the subprocess’s stdin. If you use this argument you may not also use the Popen constructor’s “stdin” argument, as it too will be used internally. Example:

>>> check_output(["sed", "-e", "s/foo/bar/"],
...              input=b"when in the course of fooman events\n")
'when in the course of barman events\n'

If universal_newlines=True is passed, the return value will be a string rather than bytes.

Changed in version 1.2a1: The timeout keyword argument is now accepted on all supported versions of Python (not just Python 3) and if it expires will raise a TimeoutExpired exception (under Python 2 this is a subclass of Timeout).

Changed in version 1.2a1: The input keyword argument is now accepted on all supported versions of Python, not just Python 3

Changed in version 22.08.0: Passing the check keyword argument is forbidden, just as in Python 3.11.

getoutput(cmd, *, encoding=None, errors=None)[source]#

Return output (stdout or stderr) of executing cmd in a shell.

Like getstatusoutput(), except the exit status is ignored and the return value is a string containing the command’s output. Example:

>>> import subprocess
>>> subprocess.getoutput('ls /bin/ls')
'/bin/ls'
getstatusoutput(cmd, *, encoding=None, errors=None)[source]#

Return (exitcode, output) of executing cmd in a shell.

Execute the string ‘cmd’ in a shell with ‘check_output’ and return a 2-tuple (status, output). The locale encoding is used to decode the output and process newlines.

A trailing newline is stripped from the output. The exit status for the command can be interpreted according to the rules for the function ‘wait’. Example:

>>> import subprocess
>>> subprocess.getstatusoutput('ls /bin/ls')
(0, '/bin/ls')
>>> subprocess.getstatusoutput('cat /bin/junk')
(1, 'cat: /bin/junk: No such file or directory')
>>> subprocess.getstatusoutput('/bin/junk')
(127, 'sh: /bin/junk: not found')
>>> subprocess.getstatusoutput('/bin/kill $$')
(-15, '')
run(args, *, stdin=None, input=None, stdout=None, stderr=None, shell=False, timeout=None, check=False) → CompletedProcess[source]#

Run command with arguments and return a CompletedProcess instance.

The returned instance will have attributes args, returncode, stdout and stderr. By default, stdout and stderr are not captured, and those attributes will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them. If check is True and the exit code was non-zero, it raises a CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute, and output & stderr attributes if those streams were captured.

If timeout is given, and the process takes too long, a TimeoutExpired exception will be raised.

There is an optional argument “input”, allowing you to pass a string to the subprocess’s stdin. If you use this argument you may not also use the Popen constructor’s “stdin” argument, as it will be used internally. The other arguments are the same as for the Popen constructor. If universal_newlines=True is passed, the “input” argument must be a string and stdout/stderr in the returned object will be strings rather than bytes.

New in version 1.2a1: This function first appeared in Python 3.5. It is available on all Python versions gevent supports.

Changed in version 1.3a2: Add the capture_output argument from Python 3.7. It automatically sets stdout and stderr to PIPE. It is an error to pass either of those arguments along with capture_output.