gevent.subprocess – Cooperative subprocess module¶Cooperative subprocess module.
Caution
On POSIX platforms, this module is not usable from native
threads other than the main thread; attempting to do so will raise
a TypeError. This module depends on libev’s fork watchers.
On POSIX systems, fork watchers are implemented using signals, and
the thread to which process-directed signals are delivered is not
defined. Because each native thread has its own gevent/libev
loop, this means that a fork watcher registered with one loop
(thread) may never see the signal about a child it spawned if the
signal is sent to a different thread.
Note
The interface of this module is intended to match that of
the standard library subprocess module (with many backwards
compatible extensions from Python 3 backported to Python 2). There
are some small differences between the Python 2 and Python 3
versions of that module (the Python 2 TimeoutExpired exception,
notably, extends Timeout and there is no SubprocessError) and between the
POSIX and Windows versions. The HTML documentation here can only
describe one version; for definitive documentation, see the
standard library or the source code.
Bases: SubprocessError
Raised when run() is called with check=True and the process returns a non-zero exit status.
cmd, returncode, stdout, stderr, output
Alias for output attribute, to match stderr
Bases: SubprocessError
This exception is raised when the timeout expires while waiting for a child process.
cmd, output, stdout, stderr, timeout
Bases: object
A process that has finished running.
This is returned by run().
args: The list or str args passed to run().
returncode: The exit code of the process, negative for signals.
stdout: The standard output (None if not captured).
stderr: The standard error (None if not captured).
New in version 1.2a1: This first appeared in Python 3.5 and is available to all Python versions in gevent.
Bases: object
The underlying process creation and management in this module is handled by the Popen class. It offers a lot of flexibility so that developers are able to handle the less common cases not covered by the convenience functions.
See also
subprocess.Popen
This class should have the same interface as the standard library class.
Caution
The default values of some arguments, notably buffering, differ
between Python 2 and Python 3. For the most consistent behaviour across
versions, it’s best to explicitly pass the desired values.
Caution
On Python 2, the read method of the stdout and stderr attributes
will not be buffered unless buffering is explicitly requested (e.g., bufsize=-1).
This is different than the read method of the standard library attributes,
which will buffer internally even if no buffering has been requested. This
matches the Python 3 behaviour. For portability, please explicitly request
buffering if you want read(n) to return all n bytes, making more than
one system call if needed. See issue 1701
for more context.
Changed in version 1.2a1: Instances can now be used as context managers under Python 2.7. Previously this was restricted to Python 3.
Changed in version 1.2a1: Instances now save the args attribute under Python 2.7. Previously this was
restricted to Python 3.
Changed in version 1.2b1: Add the encoding and errors parameters for Python 3.
Changed in version 1.3a1: Accept “path-like” objects for the cwd parameter on all platforms. This was added to Python 3.6. Previously with gevent, it only worked on POSIX platforms on 3.6.
Changed in version 1.3a1: Add the text argument as a synonym for universal_newlines,
as added on Python 3.7.
Changed in version 1.3a2: Allow the same keyword arguments under Python 2 as Python 3:
pass_fds, start_new_session, restore_signals, encoding
and errors. Under Python 2, encoding and errors are ignored
because native handling of universal newlines is used.
Changed in version 1.3a2: Under Python 2, restore_signals defaults to False. Previously it
defaulted to True, the same as it did in Python 3.
Changed in version 20.6.0: Add the group, extra_groups, user, and umask arguments. These were added to Python 3.9, but are available in any gevent version, provided the underlying platform support is present.
Changed in version 20.12.0: On Python 2 only, if unbuffered binary communication is requested,
the stdin attribute of this object will have a write method that
actually performs internal buffering and looping, similar to the standard library.
It guarantees to write all the data given to it in a single call (but internally
it may make many system calls and/or trips around the event loop to accomplish this).
See issue #1711.
Changed in version 21.12.0: Added the pipesize argument for compatibility with Python 3.10.
This is ignored on all platforms.
Changed in version 22.08.0: Added the process_group and check arguments for compatibility with
Python 3.11.
Interact with process and return its output and error.
Send input data to stdin.
Read data from stdout and stderr, until end-of-file is reached.
Wait for process to terminate.
The optional input argument should be a string to be sent to the child process, or None, if no data should be sent to the child.
communicate() returns a tuple (stdout, stderr).
timeout – Under Python 2, this is a gevent extension; if
given and it expires, we will raise TimeoutExpired, which
extends gevent.timeout.Timeout (note that this only extends BaseException,
not Exception)
Under Python 3, this raises the standard TimeoutExpired exception.
Changed in version 1.1a2: Under Python 2, if the timeout elapses, raise the gevent.timeout.Timeout
exception. Previously, we silently returned.
Changed in version 1.1b5: Honor a timeout even if there’s no way to communicate with the child (stdin, stdout, and stderr are not pipes).
Wait for child process to terminate. Returns returncode
attribute.
timeout – The floating point number of seconds to
wait. Under Python 2, this is a gevent extension, and
we simply return if it expires. Under Python 3, if
this time elapses without finishing the process,
TimeoutExpired is raised.
Run command with arguments. Wait for command to complete or timeout, then return the returncode attribute.
The arguments are the same as for the Popen constructor. Example:
retcode = call(["ls", "-l"])
Changed in version 1.2a1: The timeout keyword argument is now accepted on all supported
versions of Python (not just Python 3) and if it expires will raise a
TimeoutExpired exception (under Python 2 this is a subclass of Timeout).
Run command with arguments. Wait for command to complete. If
the exit code was zero then return, otherwise raise
CalledProcessError. The CalledProcessError object will have the
return code in the returncode attribute.
The arguments are the same as for the Popen constructor. Example:
retcode = check_call(["ls", "-l"])
Run command with arguments and return its output.
If the exit code was non-zero it raises a CalledProcessError. The
CalledProcessError object will have the return code in the returncode
attribute and output in the output attribute.
The arguments are the same as for the Popen constructor. Example:
>>> check_output(["ls", "-1", "/dev/null"])
'/dev/null\n'
The stdout argument is not allowed as it is used internally.
To capture standard error in the result, use stderr=STDOUT:
>>> print(check_output(["/bin/sh", "-c",
... "ls -l non_existent_file ; exit 0"],
... stderr=STDOUT).decode('ascii').strip())
ls: non_existent_file: No such file or directory
There is an additional optional argument, “input”, allowing you to pass a string to the subprocess’s stdin. If you use this argument you may not also use the Popen constructor’s “stdin” argument, as it too will be used internally. Example:
>>> check_output(["sed", "-e", "s/foo/bar/"],
... input=b"when in the course of fooman events\n")
'when in the course of barman events\n'
If universal_newlines=True is passed, the return value will be a
string rather than bytes.
Changed in version 1.2a1: The timeout keyword argument is now accepted on all supported
versions of Python (not just Python 3) and if it expires will raise a
TimeoutExpired exception (under Python 2 this is a subclass of Timeout).
Changed in version 1.2a1: The input keyword argument is now accepted on all supported
versions of Python, not just Python 3
Changed in version 22.08.0: Passing the check keyword argument is forbidden, just as in Python 3.11.
Return output (stdout or stderr) of executing cmd in a shell.
Like getstatusoutput(), except the exit status is ignored and the return value is a string containing the command’s output. Example:
>>> import subprocess
>>> subprocess.getoutput('ls /bin/ls')
'/bin/ls'
Return (exitcode, output) of executing cmd in a shell.
Execute the string ‘cmd’ in a shell with ‘check_output’ and return a 2-tuple (status, output). The locale encoding is used to decode the output and process newlines.
A trailing newline is stripped from the output. The exit status for the command can be interpreted according to the rules for the function ‘wait’. Example:
>>> import subprocess
>>> subprocess.getstatusoutput('ls /bin/ls')
(0, '/bin/ls')
>>> subprocess.getstatusoutput('cat /bin/junk')
(1, 'cat: /bin/junk: No such file or directory')
>>> subprocess.getstatusoutput('/bin/junk')
(127, 'sh: /bin/junk: not found')
>>> subprocess.getstatusoutput('/bin/kill $$')
(-15, '')
Run command with arguments and return a CompletedProcess instance.
The returned instance will have attributes args, returncode, stdout and stderr. By default, stdout and stderr are not captured, and those attributes will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them. If check is True and the exit code was non-zero, it raises a CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute, and output & stderr attributes if those streams were captured.
If timeout is given, and the process takes too long, a TimeoutExpired exception will be raised.
There is an optional argument “input”, allowing you to pass a string to the subprocess’s stdin. If you use this argument you may not also use the Popen constructor’s “stdin” argument, as it will be used internally. The other arguments are the same as for the Popen constructor. If universal_newlines=True is passed, the “input” argument must be a string and stdout/stderr in the returned object will be strings rather than bytes.
New in version 1.2a1: This function first appeared in Python 3.5. It is available on all Python versions gevent supports.
Changed in version 1.3a2: Add the capture_output argument from Python 3.7. It automatically sets
stdout and stderr to PIPE. It is an error to pass either
of those arguments along with capture_output.
Next page: gevent.thread – Implementation of the standard thread module that spawns greenlets