Re: Celery/Redis Django 2.0 Design Question

I think you are on the right track. I am doing something similar with photos and gps tracks uploaded by users and the celery/redis combo makes it easy to run a compute intensive task without blocking the site.

I'm not sure if the celery-signals approach is strictly necessary, however.  I am following the architecture laid out in cookiecutter-django (using docker) where the celery container is a copy of the django container.

That way, when the transformations are complete, the celery task can just issue a django style create or update command to access the database.

Good luck!

dave

On Saturday, October 27, 2018 at 6:13:19 PM UTC-6, mark wrote:

I am building a django 2.x site to:

* upload documents (images, pdfs, and videos)

* apply metadata to the documents (JSON metadata field)

* transform the documents (thumbnails, OCR, language translations, image conversion, facial recognition, image blurring, etc.) based on some of the metadata fields

* display the documents

Since the document transformations are fairly resource intensive and time consuming, I am thinking of using celery to build a state machine for the transformations, using celery tasks to transform the images, and celery signals to update the state of the document as the transformations complete successfully. I looked at django-fsm for this, but I think it will be better to run the transformations as celery tasks than to block the site. 

Does this plan make sense, or am I missing something regarding django and celery/redis.

Thanks!

Mark

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/70a447cf-312a-42dd-9eaf-24db3ef238b1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.