Sunday, November 27, 2011

Need to process uploaded files into database entries(takes hours), how do I return a progress bar?

I'm using Django 1.3 and am processing 3 files into genes, proteins,
and genomes for a tool I built... this processing usually takes
between 5 minutes to a few hours, depending on the genome size.

After uploading the files (10-100 MB), the upload view begins
processing the files, without returning for a long time (causing my
browser to ask me if I want to kill the plupload script).

For the user of this app, they don't know if their upload failed or is
processing, etc... so I'd think forking the processing function (or
making it a new thread) is what needs to happen, then the view can
return to the user, and the upload page can start doing AJAX requests
to check to processing progress.

I feel like I want to do this:
http://www.artfulcode.net/articles/threading-django/
but am scared of crashing because I've heard Django isn't threadsafe
with transactions. (and my processing function is making DB
transactions)

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

No comments:

Post a Comment