Utilities for Downloading Streaming Responses¶

requests_toolbelt.downloadutils.stream.stream_response_to_file(response, path=None, chunksize=512)¶

Stream a response body to the specified file.

Either use the path provided or use the name provided in the Content-Disposition header.

Warning

If you pass this function an open file-like object as the path parameter, the function will not close that file for you.

Warning

This function will not automatically close the response object passed in as the response parameter.

If a path parameter is a directory, this function will parse the Content-Disposition header on the response to determine the name of the file as reported by the server, and return a file path in the specified directory. If no path parameter is supplied, this function will default to the process’ current working directory.

import requests
from requests_toolbelt import exceptions
from requests_toolbelt.downloadutils import stream

r = requests.get(url, stream=True)
try:
    filename = stream.stream_response_to_file(r)
except exceptions.StreamingError as e:
    # The toolbelt could not find the filename in the
    # Content-Disposition
    print(e.message)

You can also specify the filename as a string. This will be passed to the built-in open() and we will read the content into the file.

import requests
from requests_toolbelt.downloadutils import stream

r = requests.get(url, stream=True)
filename = stream.stream_response_to_file(r, path='myfile')

If the calculated download file path already exists, this function will raise a StreamingError.

Instead, if you want to manage the file object yourself, you need to provide either a io.BytesIO object or a file opened with the ‘b’ flag. See the two examples below for more details.

import requests
from requests_toolbelt.downloadutils import stream

with open('myfile', 'wb') as fd:
    r = requests.get(url, stream=True)
    filename = stream.stream_response_to_file(r, path=fd)

print('{0} saved to {1}'.format(url, filename))

import io
import requests
from requests_toolbelt.downloadutils import stream

b = io.BytesIO()
r = requests.get(url, stream=True)
filename = stream.stream_response_to_file(r, path=b)
assert filename is None

Parameters

response (requests.models.Response) – A Response object from requests
path (str, or object with a write()) – (optional), Either a string with the path to the location to save the response content, or a file-like object expecting bytes.
chunksize (int) – (optional), Size of chunk to attempt to stream (default 512B).

Returns

The name of the file, if one can be determined, else None

Return type

str

Raises

requests_toolbelt.exceptions.StreamingError

requests_toolbelt.downloadutils.tee.tee(response, fileobject, chunksize=65536, decode_content=None)¶

Stream the response both to the generator and a file.

This will stream the response body while writing the bytes to fileobject.

Example usage:

resp = requests.get(url, stream=True)
with open('save_file', 'wb') as save_file:
    for chunk in tee(resp, save_file):
        # do stuff with chunk

import io

resp = requests.get(url, stream=True)
fileobject = io.BytesIO()

for chunk in tee(resp, fileobject):
    # do stuff with chunk

Parameters

response (requests.Response) – Response from requests.
fileobject (file, io.BytesIO) – Writable file-like object.
chunksize (int) – (optional), Size of chunk to attempt to stream.
decode_content (bool) – (optional), If True, this will decode the compressed content of the response.

Raises

TypeError if the fileobject wasn’t opened with the right mode or isn’t a BytesIO object.

requests_toolbelt.downloadutils.tee.tee_to_bytearray(response, bytearr, chunksize=65536, decode_content=None)¶

Stream the response both to the generator and a bytearray.

This will stream the response provided to the function, add them to the provided bytearray and yield them to the user.

Note

This uses the bytearray.extend() by default instead of passing the bytearray into the readinto method.

Example usage:

b = bytearray()
resp = requests.get(url, stream=True)
for chunk in tee_to_bytearray(resp, b):
    # do stuff with chunk

Parameters

response (requests.Response) – Response from requests.
bytearr (bytearray) – Array to add the streamed bytes to.
chunksize (int) – (optional), Size of chunk to attempt to stream.
decode_content (bool) – (optional), If True, this will decode the compressed content of the response.

requests_toolbelt.downloadutils.tee.tee_to_file(response, filename, chunksize=65536, decode_content=None)¶

Stream the response both to the generator and a file.

This will open a file named filename and stream the response body while writing the bytes to the opened file object.

Example usage:

resp = requests.get(url, stream=True)
for chunk in tee_to_file(resp, 'save_file'):
    # do stuff with chunk

Parameters

response (requests.Response) – Response from requests.
filename (str) – Name of file in which we write the response content.
chunksize (int) – (optional), Size of chunk to attempt to stream.
decode_content (bool) – (optional), If True, this will decode the compressed content of the response.