Re: Suggestions on "Loading" for possible long load-times?

Thanks a lot Russel! It sounds like a pretty reasonable approach to me.

On Tue, Nov 1, 2011 at 7:54 PM, Russell Keith-Magee <russell@keith-magee.com> wrote:

On Wed, Nov 2, 2011 at 7:37 AM, Kurtis <kurtis.mullins@gmail.com> wrote:
> Hey,
>
> I've got a part of my site that is just prototyped at the moment. It's
> not particularly optimized since it's still in the works.
>
> I don't think it's going to take a considerable amount of time to load
> while we have few users. But, when we start migrating our old users to
> the new system, I can imagine there will be a delay before the
> calculations are finished and the data is ready to display.
>
> Unfortunately, this is real-time calculations so it's not a simple
> matter of caching it to death. I'm not ready to do any major
> optimization at this point anyways since it's still a work-in-
> progress.
>
> Anyways, I want to display a "loading" page while the data is
> prepared. I don't mind if it's a matter of a redirect, done using
> AJAX, or just using some sort of a Javascript technique that will
> display the "Loading" dialog while the rest of the data is sent to the
> browser. What are some suggestions on doing this with Django? I prefer
> a simple, straight-forward method that won't cause a lot of
> complication with the existing code base if at all possible.

There's a very common pattern for this kind of activity:

 1) User requests /start page to get some data that is expensive to calculate

 2) View for /start creates a Task object that represents the work to
be executed in the background.

 3) An external process is running, waiting for new Task objects to be
created. Exactly how this is serviced is up to you, but the important
part is that the expensive calculation isn't performed in the view
code -- it's handled by a process external to the web server. Celery
is a common way to handle this sort of task; you can also knock
something together quickly with a script running under a cron (run
once per minute, find all new tasks, process each, then quit. Not
ideal for production, but it works as a proof of concept).

 4) Once the task has been *created* (Not finished -- just created),
/start redirects to a /loading page.

 5) /loading contains an ajax lookup that polls the completion status
of the task

 6) The task executes and completes; on completion, the task is marked
as "complete".

 7) Once the task is complete, /loading reports the completion, and
redirects to /finished to display the results of the calculation.

At every step, the web server's interaction is short lived -- create a
task; check if the task is complete; display the result of the task --
so it scales well. If you're careful about the design of your task
processor, you can make sure that this processor scales too; e.g., add
a second Celery processor for the task, and you've just doubled your
capacity.

Hope this helps!

Yours,
Russ Magee %-)

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.