Re: Django with rabbit (but without async celery) - can be done?

On Tuesday, February 25, 2014 10:58:47 AM UTC-4:30, Alon Nisser wrote:

I need to implement a quite simple Django server that server some http requests and listens to a rabbitmq message queue that streams information into the Django app (that should be written to the db). the data must be written to the db in a synchronized order , So I can't use the obvious celery/rabbit configuration.  I was told that there is no way to do this in the same Django project. since Django would listen to http requests on It's process. and It can't handle another process to listen for Rabbit - forcing me to to add Another python/django project for the rabbit/db writes part - working with the same models The http bound django project works with.. You can smell the trouble with this config from here. .. Any Ideas how to solve this?

Hello,

What you has been told seems to be the right way of solving this. You should have another python (not Django) application listening to the queue to process incoming messages. You can use Django models to insert the data into the database. That queue does not really streams data to _the application_ but to _the database_ via a third running application. I don't have any smell here, only curious of what 'written to the db in a _synchronized order_' means ;)

If you are going to receive a stream of data, split in messages in a queue, then I assume you may need to store the data in the DB in an ordered fashion, in a way that can then easily be retrieved in the right order or sequence to play or use the data. That is a different problem with many solutions with the tools you have, and has nothing to do with Django, but programming in general :) You can have an order field in your models. You can cache the data in your queue listener application and write to the DB only portions that make sense according to order and keep the data that is out of order in memory until it can be written.

Regards,

Camilo

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/3f50643c-fc7a-45f6-9ef8-a51ad74c3452%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.