Re: Django Channels - polling in consumer

Thanks for the reply. I hadn't considered a management command, but that's a great idea. I'll definitely consider that.

On Sunday, July 23, 2017 at 5:22:58 PM UTC-4, Andrew Godwin wrote:

Hi John,

You are right that trying to do polling inside a consumer will tie up a worker thread, and so it's not the right way to write this. Even more, we deliberately have Channels buffer outgoing messages until the end of the consumer, so you wouldn't see anything as it was polling.

There's a multitude of options you could use for this kind of thing, but I think my recommended approach from the little I know is to write a management command to do it. That way it's easy to launch, and you can still push into the channel layer.

Andrew

On Sun, Jul 23, 2017 at 9:52 AM, John Byrne <jhn...@gmail.com> wrote:

Hi, this is a question about whether Channels is suitable for what I'm trying to do. I suppose the tl;dr of my question is: when you need to poll some system and generate push notifications to the browser app, where should you handle the polling - inside a Channels consumer, or in a completely separate process?

I'm working on a problem that's kind of analogous to the one described here. In my case, it's not systemd log files, but the state of various Celery tasks that I want to monitor. However, my problem is similar to that one in that these tasks have no knowledge of my web application, and I can't really have them push notifications to it. I want to display real time state information (probably a progress bar) in the browser based on the state of a task - but something needs to poll Celery to get the latest state.

I'm looking at using Channels to help me solve this overall problem, but I'm not sure where to handle this polling process. At first, I thought Channels was just about providing the Websockets link, and that any long-lived process that produces push notifications should be separate. I would write a process that runs in the background, polling Celery for task states, and pushing notifications into one or more channels, which the JS app can consume.

But as I read more about channels, it seems that it's also intended for managing background tasks. That makes me wonder if I should use a Channels consumer to implement this polling. It would not necessarily have to run in an infinite loop - the polling only needs to happen when when at least one person wants to monitor a task - but that would probably end up being most of the time (there's a many to many relationship between webapp users and Celery tasks, and there will probably several of each on the go at all times).

Is it a bad idea to do something like this in a Channels consumer? I'm not sure how I would prevent all my workers from getting tied up in these polling operations, to the detriment of the REST of the app (pun intended ;) )

Thanks!
John

PS. If it matters, this is not an internet-scale app - it's an intranet app with users numbering in the dozens.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/0295e24f-afda-4cae-a6db-729fa7fafc8a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/dbc6b612-92e5-439a-9742-a455502253b9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.