Monday, November 28, 2011

Re: Simple task dispatching (How heavy is celery + RabbitMQ)

On Mon, Nov 28, 2011 at 11:14 AM, Gelonida N <gelonida@gmail.com> wrote:
> Hi,
>
> I'd like to use a light weight dispatching system for a web server.
>
> Basically some django post requests might require processing, which
> should be done in the back ground due to its run times.
>
> The results would be added to the django data base.
>
> The browser could verify via AJAX requests whether the task is finished.
>
>
> The server would be running on a rather weak virtual machine with rather
> low memory (nginx / uwsgi / django )
>
> ( For testing  I run the server on windows with one of the following
> setups (depending on what I'd like to test)
> - django runserver
> - twisted - django
> - cygwin/nginx - fastcgi - django
>
> Most people seem to recommend celery with RabbitMQ.
> If I understood well, rabbit MQ requires Erlang to be installed and I
> found some posts indicating that RabbitMQ requires quite some memory
>
> So I wondered whether celery / RabbitMq wouldn't be a little on the
> heavy side and eat away a little too much from my meory.
>
>
> Is there any good light weight dispatching alternative to celery or
> would this be one of these  'roll your own dispatcher' tasks?
>
>

On my personal, doesn't-really-do-much, 5k messages a day, home
server, rabbitmq uses a grand total of 19 MB RAM.

On one of our production servers handling millions of messages a day,
rabbitmq uses a total of 27 MB of RAM.

I guess it all depends on your definition of "too much". I doubt a
home-brew python process would be as slender.

Cheers

Tom

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

No comments:

Post a Comment