Re: Architecting a Crypto Market Data Feed using Django Channels

I can't help you with real-time streaming architecture overall - that's a much bigger scope of thing - but I can say that you shouldn't be keeping a synchronous consumer open like that (you're using a whole thread). You should either rewrite it to be async-native, so it doesn't use up a thread and potentially block the server, or rework it to put the feed events onto the channel layer from an external process.

Andrew

On Sat, Apr 28, 2018 at 1:12 AM, Michael <writemichaelmartinez@gmail.com> wrote:

Hi,

What is the best way to architect a Django Channels app that provides a very fast infinite stream of market data? This is what I have so far, but I think it's not the best solution.

This data is updated every millisecond so I would prefer to not persist it (unless there is a way of using redis pub/sub without actually saving the data, only for messaging)

class ChatConsumer(WebsocketConsumer):
    def connect(self):
        self.room_name = 'foo'
        self.room_group_name = 'foo'
        async_to_sync(self.channel_layer.group_add)(
            self.room_group_name,
            self.channel_name
        )


        self.accept()
        while True:
          # Imagine this is another WS feed or Zero MQ Feed.
          feed = Feed(....)
          for event in feed:
              if event.name == "text":
                  data = event.json
                  self.send(str(data)

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/01bf458c-ff1a-4cf6-bd58-da9b2f43123c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFwN1uqD0_qaNksKFo-OZAG3Lmbg023fWr2eEhct%2BGW-JWwoCw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.