gp.fileupload (original) (raw)
The Sphinx version of this documentation can be found here.
A demo page is also available.
If you found a bug submit an issue http://www.bitbucket.org/gawel/gpfileupload/issues/?status=new.
Description
gp.fileupload is a set of wsgi middleware to deal with large file upload.
- gp.fileupload.FileUpload is a wsgi middleware to get the stat of large files uploaded to the server. Optional javascript code is provided to animate a progress bar.
- gp.fileupload.Storage is a wsgi middleware to store files on file system and avoid big transaction in your application.
Upload middleware
The principle is to count uploaded chunks in the wsgi loop, transmit them to the application, and provide a link with up-to-date json information. The optional javascript code is able to use an existing form and replace it with a progress bar during upload. It can also generate its own upload form, then upload several files sequentially with multiple progress bars. gp.fileupload can be used without modifying your application, just by adding it in the wsgi stack.
It has currently been tested with Pylons and Zope 3.
Middleware
Wrap your wsgi application with the middleware:
from gp.fileupload import FileUpload
def my_application(environ, start_response): ... start_response('200 OK', [('Content-Type', 'txt/html')]) ... return ['My app']
app = FileUpload(my_application, tempdir=TEMP_DIR, ... max_size=None)
def application(environ, start_response): ... return app(environ, start_response)
The FileUpload middleware has the following options:
- tempdir: A path to a temporary folder
- max_size: Max allowed size. If the file size is larger than max_size aRuntimeError is raised.
- include_files: A list of static files that the middleware should include in your html body (see below). You might want to include the files by yourself in the relevant pages, or they will be included in all of your pages.
- require_session: a boolean flag that disable the middleware for requests that don’t provide the SESSION_name in the query.
Application code
Write an html form like this:
Where 1 is the session id. The session id must be a digit.
When the form is submitted, you can use some ajax stuff to get the stats of the upload with the url:
http://yourhost/gp.fileupload.stat/1
This will return some JSON data like:
{'state': 1, 'percent': 69}
state can have the following values:
- 0: nothing done yet.
- 1: upload is active
- -1: file is larger than max_size.
You can use this to display the upload progress.
Storage middleware
The storage middleware provide a way to avoid long transaction in your application.
POST content is written to a temporary directory. If the POST contains some files then the files are moved to the storage directory. The files content is replaced by the path of the real file relative to the storage root.
This way, your application receive always a few Ko of data.
Usage
Wrap your wsgi application with the middleware:
from gp.fileupload import Storage from gp.fileupload import purge_files import cgi
def my_application(environ, start_response): ... """simple app to read the file path from the request and remove it ... from the storage directory ... """ ... if environ['REQUEST_METHOD'] == 'POST': ... fields = cgi.FieldStorage(fp=environ['wsgi.input'], ... environ=environ, ... keep_blank_values=1) ... relative_path = fields['file'].read() ... # remove file from storage ... purge_files(environ, relative_path) ... start_response('200 OK', [('Content-Type', 'txt/html')]) ... return ['My app']
app = Storage(my_application, ... upload_to='/tmp/share/files', ... tempdir='/tmp/upload_tmp', ... )
def application(environ, start_response): ... return app(environ, start_response)
The Storage middleware has the following options:
- upload_to: the root directory of the storage tree.
- tempdir: A path to a temporary folder
- exclude_paths: a list of regular expressions. All matched PATH_INFO will be ignored. If this parameters is not None, then the middleware will also catch non-HTML content send by your application. WARNING: This options is experimental and not well tested.
- require_session: a boolean flag that disable the middleware for requests that don’t provide the SESSION_name in the query.
Paste factories
The package provides a filter factory usable in PasteDeploy configuration files.
The factory provides the middleware itself:
[pipeline:main] pipeline = fileupload egg:myapp
[filter:fileupload] use = egg:gp.fileupload
temporary directory to write streams to
tempdir = %(here)s/data/fileupload
file to inject in the html code
include_files = fileupload.css jquery.*
if you already have jquery in your application, use this line
#include_files = fileupload.css jquery.fileupload.*
max upload size is 50Mo
max_size = 50
use this options to also wrap your application with a Storage middleware
#upload_to = %(here)s/storage #exclude_paths = /@@
Then you can access the javascript stuff at /gp.fileupload.static/.
The include_files parameters will inject these tags in your application: