Re: Django Channels App Design

Thanks Andrew! I'm glad I'm not the only guy out there with a similar problem. I'll have to look into the feasibility of the database approach, but I'm not sure if it will be efficient enough - I have some data points that require faster resolution than every second.

On Friday, April 28, 2017 at 6:23:41 PM UTC-6, Andrew Godwin wrote:

Hi Ethan,

Distributed collaboration is kind of a generally unsolved problem, so there's no real best approach for what you have here. Having each environment assigned to a single process (worker) definitely helps keep things in shape, and if it's working for you, then I'd keep going with it.

My personal approach would be to instead use a database and log every change to an environment in an append-only format to that database, sending signals on each change that prompts updates to be sent down to clients, which can then be batched by querying the database based on revision number (if the environments are small enough they can be stored on a single server). This has the advantage that you can just "queue up" update transmissions and send them less often (maybe once a second) rather than every change, as you can just keep track of what the last revision each client saw was and send them anything newer. I'm not quite sure if I'd use a sorted set in Redis for this or just a postgres table (probably the latter, with good indexes?)

You might want to do some research into the mechanisms that power things like Google Docs collaborative editing; this is a similar type of problem. Certainly, using sessions for this kind of heavy lifting will be expensive to run.

Andrew

On Fri, Apr 28, 2017 at 10:23 AM, Ethan Knapp <ethan...@gmail.com> wrote:

As background, I am developing a web app that implements Channels in what seems like an unusual way, so I want to do some validation and ask some opinions and suggestions. The application allows multiple users to work together in the same 3D environment (using Three.js on the client), being able to see each other and work together on the model.

I have established that keeping some data regarding the group of users stored in active memory on the app server is necessary. My initial thought was to create a Session object that tied into a Channels Group, but even with a Redis session backend, I can see that becoming very inefficient to request and hydrate the session out of Redis for every message received from a client WebSocket (Is this assertion correct?). My current implementation uses a custom Worker that can be assigned multiple groups of users (each group hereafter called an Environment). An instance of an Environment is only ever handled by one Worker, while a Worker can handle multiple Environments. A consumer within a normal Django-Channels app accepts messages on websocket.connect, websocket.receive, websocket.disconnect, and forwards them to the appropriate worker (each worker is assigned a unique channel), where the worker decides which Environment the message was intended for, and passes it. The Environment may then do whatever it needs to, including return a reply message to send back to the client via WebSocket (directly to the interface server via reply_channel).

My question is: "Is this the best design, or should I consider something else?" I am unsure of whether having one Channel per Worker is ideal, or if I should just give each Environment a unique channel (my idea here is that I only need to receive once per worker instead of n times). I also may need to give each Environment a tick loop beyond just responding to Channels events (likely, but undetermined) (honestly it's starting to feel more like a game/game server pair than a web app! [Considerations of a Java backend here?]). Also, does this fall within the use of Channels, or should I seriously consider using something like RabbitMQ directly, instead of possibly just using it as a Channels backend?

Thanks in advance for any advice or thoughts!
Ethan

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/35e1bed1-c2ce-456f-8770-f9eca719dd50%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/1d626676-cbad-4ae4-b2c3-fc85277c7f18%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.