Useful methods for working with httplib, completely decoupled from
code specific to urllib3.
Our embarrassingly-simple replacement for mimetools.choose_boundary.
Encode a dictionary of fields using the multipart/form-data MIME format.
fields – Dictionary of fields or list of (key, RequestField).
Values are processed by urllib3.fields.RequestField.from_tuples().
boundary – If not specified, then a random boundary will be generated using
urllib3.filepost.choose_boundary().
Iterate over fields.
Supports list of (k, v) tuples and dicts, and lists of
RequestField.
A convenience, top-level request method. It uses a module-global PoolManager instance.
Therefore, its side effects could be shared across dependencies relying on it.
To avoid side effects create a new PoolManager instance and use it instead.
The method does not accept low-level **urlopen_kw keyword arguments.
Flush and close the IO object.
This method has no effect if the file is already closed.
Should we redirect and where to?
Truthy redirect location string if we got a redirect status
code and valid location. None if redirect status and no
location. False if not a redirect status code.
Parses the body of the HTTP response as JSON.
To use a custom JSON decoder pass the result of HTTPResponse.data to the decoder.
This method can raise either UnicodeDecodeError or json.JSONDecodeError.
Read more here.
Memory-efficient bytes buffer
To return decoded data in read() and still follow the BufferedIOBase API, we need a buffer to always return the correct amount of bytes.
This buffer should be filled using calls to put()
Our maximum memory usage is determined by the sum of the size of:
self.buffer, which contains the full data
the largest chunk that we will copy in get()
The worst case scenario is a single chunk, in which case we’ll make a full copy of the data inside get().
HTTP Response container.
Backwards-compatible with http.client.HTTPResponse but the response body is
loaded and decoded on-demand when the data property is accessed. This
class is also compatible with the Python standard library’s io
module, and can hence be treated as a readable object in the context of that
framework.
Extra parameters for behaviour not present in http.client.HTTPResponse:
preload_content – If True, the response’s body will be preloaded during construction.
decode_content – If True, will attempt to decode the body based on the ‘content-encoding’ header.
original_response – When this HTTPResponse wrapper is generated from an http.client.HTTPResponse
object, it’s convenient to include the original for debug purposes. It’s
otherwise unused.
retries – The retries contains the last Retry that
was used during the request.
enforce_content_length – Enforce content length checking. Body returned by server must match value of Content-Length header, if present. Otherwise, raise error.
Flush and close the IO object.
This method has no effect if the file is already closed.
Read and discard any remaining HTTP response data in the response connection.
Unread data in the HTTPResponse connection blocks the connection from being released back to the pool.
Returns underlying file descriptor if one exists.
OSError is raised if the IO object does not use a file descriptor.
Flush write buffers, if applicable.
This is not implemented for read-only and non-blocking streams.
Similar to http.client.HTTPResponse.read(), but with two additional
parameters: decode_content and cache_content.
amt – How much of the content to read. If specified, caching is skipped because it doesn’t make sense to cache partial content as the full response.
decode_content – If True, will attempt to decode the body based on the ‘content-encoding’ header.
cache_content – If True, will save the returned data such that the same result is
returned despite of the state of the underlying file object. This
is useful if you want the .data property to continue working
after having .read() the file object. (Overridden if amt is
set.)
Similar to HTTPResponse.read(), but with an additional
parameter: decode_content.
amt – How much of the content to read. If specified, caching is skipped because it doesn’t make sense to cache partial content as the full response.
decode_content – If True, will attempt to decode the body based on the ‘content-encoding’ header.
Return whether object was opened for reading.
If False, read() will raise OSError.
A generator wrapper for the read() method. A call will block until
amt bytes have been read from the connection or until the
connection is closed.
amt – How much of the content to read. The generator will return up to much data per iteration, but may return less. This is particularly likely when using compressed data. However, the empty string will never be returned.
decode_content – If True, will attempt to decode the body based on the ‘content-encoding’ header.
Checks if the underlying file-like object looks like a
http.client.HTTPResponse object. We do this by testing for
the fp attribute. If it is present we assume it returns raw chunks as
processed by read_chunked().
Obtain the number of bytes pulled over the wire so far. May differ from
the amount of content returned by :meth:urllib3.response.HTTPResponse.read
if bytes are encoded on the wire (e.g, compressed).
Returns the URL that was the source of this response. If the request that generated this response redirected, this method will return the final redirect location.