Re: Django channels with python background tasks

Jaiswal,

Either my solution 1 or 2 will solve your problem, or rewriting your scripts as management commands (which is probably the easiest route).

If you want to keep them separate :-

For solution 1; I would look at using the requests library (see https://pypi.org/project/requests/ ) in your backend scripts and adding django-rest-framework to your django project ( http://www.django-rest-framework.org ) to handle to API views.

For solution 2: it is more roll your own; but googling 'zmq' and 'redis' would be a start to get you thinking about how these things can hang together. This is probably the most advanced solution though.

I'm sorry this is not a lot more help, it is all up to understanding what your comfortable with *exactly* what you want to achieve from here.

On Sun, 2018-05-27 at 02:22 -0700, Sourabh Jaiswal wrote:

Hi Roger,

Thanks for your reply.

Yes you are correct. There are actually 2 modules of this project:-

1. Django front end which is totally isolated in itself if I talk about it with back-end reference. And currently it has no mechanism whatsoever to communicate with back-end.

2. python based back-end scripts. which itself is a program. It can be started with some management IPs of Network elements and that program will keep getting the data from those NE synchronously.

    This back-end can not communicate with any external program yet. And uses a global variable file for starting the threads.

Now to link these 2 modules I understand I need to to introduce APIs in module 2. and call those APIs from django.

Which is exactly what you are suggesting in your suggestion 1. Correct me if I am wrong.

Just a little bit more help. Can you please guide me to some tutorial to get this thing started.

Thanks in advance.

Regards,

Jaiswal

On Sunday, May 27, 2018 at 2:34:05 PM UTC+5:30, Roger Gammans wrote:

I think there is a little bit of confusion in this thread,a s we keep talk about django and the deamon process and ignoring the frontend.

So to clarify; there we are talking about a system with at least 3 execution environments:

1 The Browser

2 Django backend processes

3 The backend daemon/cli -process.

django-channel provides a mechanism which (amongst other uses) can get near-realtime updates from a django environment to the web browser, but if you don't need this you can just store all your status data in the data base . Then when the browser fetches a views it will have the most recent data in the database.

You still however need to communicate between the backend process and the rest of the django environment, the simplest way as Ryan suggesting is to

bring this inside the django environment by implementing it as a management command, from there you have access to the python end of the channels api to send messages to the browsers, or the model objects to save to the database. Management commands can be arbitrarily complex, so that not a problem in itself.

I have a similar system in development and there we do run the backend daemon outside of the dajngo environment, but that makes database access hard as you need a separate copy of the database models, or to use raw SQL. In our system we still have a django management command though which has the responsibility of pumping/handling messages from the deamon to the intended destination.

If you want to keep your backend separate l I can see two reasonable paths:

1. Use a custom set of django views to provide a API your backend connects to over localhost/http to send the it message updates.

2. A django management command as a API interface message handler, which connects to the backend over one of many message channels out there. (ie. zmq, redis, unix-sockets, etc)

Hope this helps a little bit.

On Sun, 2018-05-27 at 01:01 -0700, Sourabh Jaiswal wrote:

HI Ryan,

The back-end script in this case is not only doing telnet. Its getting data from multiple network elements using telnet and multi-threading. There are multiple threads started when I start the backend script which takes monitoring data from diff-diff network elements at the same time. The communication between the threads is happening using a python Sockets. A TCP server is responsible to pass the messages between threads. So I have to have this background program run to collect data.

The control from django I want is, to be able to start multiple instance of this whole back-end package when user clicks on such cluster in django front end. Passing management IP and other relevant information from django DB. and after starting particular instance django should be able to control it.

What I found out from recent googling and stuff is. I need to write some APIs in back-end program and as Andrew suggested it needs to be written in such a way that it can be polled via django.

So where to start in this regard? can you please guide me for that.

Thanks in advance!

Regards,

Jaiswal.

On Saturday, May 26, 2018 at 8:15:28 PM UTC+5:30, Ryan Nowakowski wrote:

Another idea: create a Django management command that does the telnet stuff and just run that periodically via Cron. You can always move to channels or celery later.

On May 26, 2018 9:33:16 AM CDT, Ryan Nowakowski <tub...@fattuba.com> wrote:

In addition to websockets, channels can be used to run background tasks that could take a long time for example a telnet connection. You can Google Django channels background tasks.

An alternative to channels for background tasks is celery.

On May 26, 2018 5:03:51 AM CDT, Andrew Godwin <and...@aeracode.org> wrote:

Hi Jaiswal,

I'm afraid that I can't give detailed help about what your best options are or walk you through how to do it - that's something you'll have to research and decide on yourself. Channels allows you to do low-latency communictation between Django back-ends and JavaScript, but anything you can do with it can also be done slower using a polling API connection.

My recommendation would be to start simple - doing it using an API that you poll every few seconds - and then once you have that figured out, look at how you could improve it using WebSockets and channels.

Andrew

On Sat, May 26, 2018 at 8:38 AM Sourabh Jaiswal <sourabhj...@gmail.com> wrote:

Hi,

I am writing a python based application(CLI Back End) which does telnet to some network components and gets some data. It saves the data in sqlite db.

For this application I am writing Django based frond end. Which will start the CLI app and monitor it. For communication between the CLI App and django I am not able to decide what to use. I read somewhere on net that django channels can be used in this problem.

I have no idea about what django channel is and what it does. Can some one please help me in this I need to come up with a tool for this CLI and django communication ASAP.

Thanks in advance!

Regards, Jaiswal.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/59afd3e9-3552-4572-b6a0-0292ebab3a49%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.