Re: problem with django, nginx and gunicorn

On Thu, May 1, 2014 at 9:06 AM, 'ReneMarxis' via Django users
<django-users@googlegroups.com> wrote:
> I started reading a little bit related to GIL. I think this is the root of
> my problem. I'm using xhtml2pdf to generate some large pdf's (up to 200
> pages).


no, the problem isn't the GIL. no matter the framework, language or
architecture, if you take more than a few seconds to generate an HTTP
response, users are going to complain. the appropriate approach for
anything that takes more time is to dispatch to other process.

that's why there are so many queue managers, they make it easier to
allocate a task to be performed by other process.

the high level process should be something like this:

- user click on some "do it" button or link. it generates an HTTP
request to do some work

- the web app (a Django View in this case) gets the request, picks any
necessary parameter and initiates a task. the task manager responds
immediately with some ID.

- the web app immediately generates a response for the user with some
message like "process initiated, please wait" probably shows the ID,
or maybe a link to the "is it ready" with this ID as a parameter.

- meanwhile, the task/queue manager delivers the task parameters to a
worker process.

- if the user (or some Javascript) clicks the "is it ready" link,
another Django view gets the task ID, checks the status and if it's
not ready, shows the "please wait" message.

- eventually, the task is finished, and the result (or some URL to a
produced file) is stored in the task, under the same ID

- when the "is it ready" view is hit again with this ID, it sees the
task has finished, so it redirects to the "result" URL.


about the task/queue manager, there are several available, from a
simple register in a "tasks" table and a cron task to check any
records there in the "unfinished" status (called a Ghetto queue), to
the full-featured Celery.

most of them allow you to define the tasks as simple Python functions
with a @task decorator, and simply calling it results in the queuing
of the arguments. something like this (semi-pseudocode):


---- tasks.py

from queue import task
from models import MyModelA, MyModelB

@task
def dolotsofwork(userid, modelaid, someotherarg):
user = User.objects.get(pk=userid)
modA = MyModelA.get(pk=modelaid)
...... do much more heavy processes....
return "result-%d.pdf" % resultId

------ views.py

from tasks import dolotsofwork

def starttask(request, arg):
user = request.user
modela = get_object_or_404(MyModelA, pk=arg)
taskid = dolotsofwork(user.id, modela.id, ......)
return rendertemplate('pleasewait.html', {'taskid': taskid})

def isitready(request, taskid)
status = queue.taskstatus(taskid)
if status.isready():
return redirect(resulturl, taskid)
return rendertemplate('pleasewait.html', {'taskid': taskid})

def resulturl(request, taskid):
resultfilename = queue.taskresult(taskid)
return rendertemplate('processready.html', {'resultfilename':
resultfilename})





--
Javier

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFkDaoTn0ZotVdzn5d%3DaxwCTZmTs8Fraki05yY2aev3b0eQM3Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.