Transfer Large Files Asynchronously In Flask - asynchronous

Question

What is the best approach for transfer large files Asynchronously in Flask? I have read this article. But I want to know if there is a way to do this without using celery?

Answer 1

Flask is a synchronous framework, you can try flask+gevent and streaming responses like explained here: http://flask.pocoo.org/docs/0.12/patterns/streaming/.
Anyway, if you want to upload properly very large files, I suggest you to use a different approach. Instead of trying to do asynchronous networking with a synchronous framework try delegating the transfer with Nginx upload_module, like explained here: http://blog.thisisfeifan.com/2013/03/nginx-upload-module-vs-flask.html
Nginx is faster and won't load up the files in memory, a thing that regular frameworks like Flask or Django even in Asynchronous mode will do. Remember to configure flask to receive after upload POST with directive upload_pass. The only caveat is that you'll have to learn how to compile a full fledge Nginx from source, here an example of working Dockerfile:
FROM buildpack-deps:jessie
##### NGINX #####
# Base Stuff
RUN apt-get update && apt-get install -y -qq \
libssl-dev
# Nginx with upload_module and upload_progress_module
# "Stable version".
ENV ZLIB_VERSION 1.2.11
ENV PCRE_VERSION 8.39
ENV NGX_UPLOAD_MODULE_VERSION 2.2
ENV NGX_UPLOAD_PROGRESS_VERSION 0.9.1
ENV NGX_HEADERS_MORE_VERSION 0.32
ENV NGX_SPPEDPAGE_VERSION 1.11.33.4
ENV NGINX_VERSION 1.11.8
RUN cd /tmp \
&& wget http://nginx.org/download/nginx-${NGINX_VERSION}.tar.gz \
&& tar xvf nginx-${NGINX_VERSION}.tar.gz \
&& wget https://github.com/openresty/headers-more-nginx-module/archive/v${NGX_HEADERS_MORE_VERSION}.tar.gz \
&& tar -xzvf v${NGX_HEADERS_MORE_VERSION}.tar.gz \
&& wget https://github.com/pagespeed/ngx_pagespeed/archive/latest-stable.tar.gz \
&& tar -xzvf latest-stable.tar.gz \
&& wget https://dl.google.com/dl/page-speed/psol/${NGX_SPPEDPAGE_VERSION}.tar.gz \
&& tar -xzvf ${NGX_SPPEDPAGE_VERSION}.tar.gz \
&& mv psol ngx_pagespeed-latest-stable/ \
&& git clone -b ${NGX_UPLOAD_MODULE_VERSION} https://github.com/Austinb/nginx-upload-module \
&& wget http://zlib.net/zlib-${ZLIB_VERSION}.tar.gz \
&& tar xvf zlib-${ZLIB_VERSION}.tar.gz \
&& wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-${PCRE_VERSION}.tar.bz2 \
&& tar -xjf pcre-${PCRE_VERSION}.tar.bz2 \
&& wget https://github.com/masterzen/nginx-upload-progress-module/archive/v${NGX_UPLOAD_PROGRESS_VERSION}.tar.gz \
&& tar xvf v${NGX_UPLOAD_PROGRESS_VERSION}.tar.gz \
&& cd nginx-${NGINX_VERSION} \
&& ./configure \
--with-pcre=../pcre-${PCRE_VERSION}/ \
--with-zlib=../zlib-${ZLIB_VERSION}/ \
--add-module=../nginx-upload-module \
--add-module=../nginx-upload-progress-module-${NGX_UPLOAD_PROGRESS_VERSION} \
--add-module=../ngx_pagespeed-latest-stable \
--add-module=../headers-more-nginx-module-${NGX_HEADERS_MORE_VERSION} \
--with-select_module \
--with-poll_module \
--with-file-aio \
--with-http_ssl_module \
--with-ipv6 \
--with-pcre-jit \
--with-http_gzip_static_module \
--with-http_ssl_module \
--with-http_v2_module \
--with-http_realip_module \
--user=nginx --group=nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --with-cpu-opt=CPU -- with-ld-opt="-Wl,-E" \
&& make \
&& make install
EXPOSE 80 443
CMD ["nginx", "-g", "daemon off;"]
NOTE: Please in this image is lacking the provision of nginx.conf and default.conf.