Python streaming download file

Please indicate the source for reprint: http://blog.csdn.net/jinixin/article/details/79053741


I wrote an article on using WebUploader to upload large files before . Since there is a need to upload files, downloading files is inevitable. Flask is used as an example here. For ease of porting, the send_file method integrated by the Flask framework is not used.



normal download


Go directly to the code:

@app.route('/file/download/<filename>', methods=['GET'])
def file_download(filename):
    with open('./upload/%s' % filename, 'rb') as target_file: # read file content
        data = target_file.read()
    response = Response(data, content_type='application/octet-stream') # The response specifies the type and writes the content
    return response                                                   

In the response, the content type is specified as byte stream, and the browser will download the content directly after receiving it.



streaming


Although the above code can be downloaded smoothly, if the file is large, directly opening the file to read the content will take up a lot of memory and affect the server performance. So it is best to stream back and read the contents of the file bit by bit. When it comes to streaming, in Python I think of iterators and the yield keyword.


Modified code:

@app.route('/file/download/<filename>', methods=['GET'])
def file_download(filename):
    def send_chunk(): # streaming read
        store_path = './upload/%s' % filename
        with open(store_path, 'rb') as target_file:
            while True:
                chunk = target_file.read(20 * 1024 * 1024) # read 20M each time
                if not chunk:
                    break
                yield chunk

    return Response(send_chunk(), content_type='application/octet-stream')

When running, open Activity Monitor, there is no large memory usage.


Finally, I uploaded the demo about streaming download to the upload_demo project. If you want the full code, click here .



The understanding of streaming is not very clear. If there is anything inappropriate in the text, I hope everyone will tolerate and support it. Thank you.



Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=325563164&siteId=291194637