Re: Django and asynchronous tasks

Hi Alain,

I'm not quite sure what you're trying to accomplish by mixing rq and channels; Channels just allows you to either write things against WebSocket connections, or to build your own eventing system from scratch on the same base code.

If you're trying to return results to a webpage, you'll need the reply_channel name that points to an open socket connected to that page, or have connected the socket to a group when it connected and send to that group, and write your own javascript on the client end to handle it.

Andrew

On Tue, Nov 29, 2016 at 11:44 PM, Alain Muls <alain.muls@gmail.com> wrote:

Hi All

I spent several days reading and searching examples of how to implement channels. The current status of my project is as follows:

• I have a page which lets the user enter the data used for the offline calculations, a POST request is send
• the post request is handled and I can forward these data to a rq-worker, after which I open a basic web-page showing the input data. This page should also hold the results of the asynchronous calculations
• the rq-worker starts the requested download of data from a site and I store the data in the apps (gnsspredict) directory. The rq-worker stops with a success reported in the log.
• not yet implemented, but this is similar to the download thread, I shall further launch 2 to 3 workers to split up the work so that I can inform the user of the progress of the async calculations

But this is where I do not know what to do next.

As far as I understand 

• the rq-worker can send its results using a dictionary to a 'reply-channel', but where does this channel goes to? 
  
• of the few complete working examples on github I found I assume that I have to write a javascript (which I never used) to treat the results from the worker thread? I also remark that the javascript has as name 'function()', is this always the case? If so, for all 3 to 4 workers how do I write a script that takes action on which asyn cthread that has finished?

All help is appreciated

Thanks

Alain

On Thursday, 10 November 2016 11:55:01 UTC+1, ludovic coues wrote:

Websocket provide a way for server to send information to the client
without waiting for input from the client.

Django channels [1] is a project to bring native support of websocket
to django. There are alternatives which might involve a bit more of
work

[1] https://channels.readthedocs.io/en/stable/

2016-11-10 11:38 GMT+01:00 Antonis Christofides <ant...@djangodeployment.com>:
> Is there no mechanism that when the background tasks finishes to have a web
> page called which could display the results?
>
> Web pages cannot be "called". They are loaded by the browser. So, what you
> want is a mechanism that notifies the browser that an event has occurred in
> the server. That mechanism is comet.
>
> Antonis Christofides
> http://djangodeployment.com
>
> On 2016-11-10 11:54, Alain Muls wrote:
>
> Hi
>
> Tx for the suggestion but how do I reload a page after eg 30 seconds?
> Is there no mechanism that when the background tasks finishes to have a web
> page called which could display the results?
> I had a look at the signal mechanism of Django but I think that is not
> working since the background task is in another environment than the django
> apps which called it.
>
> bye/alain
>
> On Thursday, 10 November 2016 09:55:08 UTC+1, Antonis Christofides wrote:
>>
>> (Note: The most popular way to do asynchronous tasks is celery, but indeed
>> some
>> people prefer django-rq, which is said to be simpler. But your question is
>> not
>> affected by that.)
>>
>> I'm not an expert but I think that the "correct" way to do what you want
>> would
>> be to use comet (i.e. the opposite of ajax). However, if the work required
>> to
>> make that work is not justified by the budget or the business case, you
>> might be
>> able to get away with a message like "This information is being
>> (re)calculated.
>> Reload the page after half a minute to view the updated results." (That's
>> what I
>> did last time :-)
>>
>> Regards,
>>
>> Antonis
>>
>> http://djangodeployment.com
>>
>> On 2016-11-10 10:04, Alain Muls wrote:
>> > Hi All
>> >
>> > I am building a website which makes calculations about the visibility of
>> > satellites. These calculations take about half a minute so I do not want
>> > to
>> > block the site during this time. I found django-rq and was able to start
>> > a
>> > asynchronous task which handles the calculations.
>> >
>> > The problem I have is how do I find out when the calculations of the
>> > task
>> > thread are done so that I can direct the results to another web page
>> > which
>> > will display them?
>> >
>> > Thanks for your help
>> >
>> > Alain Muls
>> >
>> >
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/6ae68ef3-6a52-4a59-a87b-3fcd627b4f26%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/0dcd7af0-6833-8174-363a-a68c5b3ff5c9%40djangodeployment.com.
>
> For more options, visit https://groups.google.com/d/optout.



--

Cordialement, Coues Ludovic
+336 148 743 42

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/c7aa0cfa-0371-4a85-a248-8f2f9715aadd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFwN1uqroxLN7RBeaX3_%2BeEtOEz9W6Hdu%3DSfAC56fL%2BRtUBQNQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: makemigrations generates new migration when nothing has changed

On 1/12/2016 3:56 AM, Francis Fisher wrote:
> Any idea why makemigrations would fail to recognise that a migration
> has already been generated?

I think migrations is seeing field choices differently each time it
scans the model. Can you use a list rather than a tuple for choices? (Or
vice versa - I don't remember) Or import the choices from another module?


>
> makemigrations.py is a script which calls makemigrations with
> appropriate django settings. If I run this once, it will generate the
> initial migration, but every time I run it subsequently, it will
> regenerate a migration even though nothing has changed in the model.
> This is with django 1.10.3 and postgres. The field sk that is
> constantly regenerated is a foreign key relation to an unmanaged
> database, but I have referred to this in a different app with no bother.
>
> I am using django 1.10.3.
>
> I found a post in this group that had a similar issue with
> instantiating a class in the model that was missing an implementation
> for __eq__ but that doesn't apply in this case as I don't instantiate
> any class in the model file.
>
> Is there any common mistake that leads to this outcome?
>
> ------------------
>
> user@testenv:~/eit/testproj$ ./makemigrations.py
> Migrations for 'sk':
> testproj/sk/migrations/0001_initial.py:
> - Create model SKM
> user@testenv:~/eit/testproj$ ./makemigrations.py
> Migrations for 'sk':
> testproj/sk/migrations/0002_auto_20161130_1643.py:
> - Alter field s on skm
> user@testenv:~/eit/testproj$
> user@testenv:~/eit/testproj$ ./makemigrations.py
> Migrations for 'sk':
> testproj/sk/migrations/0003_auto_20161130_1643.py:
> - Alter field s on skm
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users+unsubscribe@googlegroups.com
> <mailto:django-users+unsubscribe@googlegroups.com>.
> To post to this group, send email to django-users@googlegroups.com
> <mailto:django-users@googlegroups.com>.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/ad7c5eb6-4542-456e-8e53-675c02e897b1%40googlegroups.com
> <https://groups.google.com/d/msgid/django-users/ad7c5eb6-4542-456e-8e53-675c02e897b1%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/0ed4c35b-753b-3737-223c-a63e2c45601a%40dewhirst.com.au.
For more options, visit https://groups.google.com/d/optout.

Django and wsgiref simple_server.py line 33

REF: /usr/lib64/python2.7/wsgiref/simple_server.py", line 33,

On a fresh install of Python 2.7 and Django 1.6.12 attempting to resolve the following error.

When running... 

 python manage.py runserver 0.0.0.0:8000

And then loading a page on a browser I get the following.

********************************************************************************

  File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 33, in close

   self.status.split(' ',1)[0], self.bytes_sent

   AttributeError: 'NoneType' object has no attribute 'split'

********************************************************************************

I've googled and read entries in this group but can find no suggestions that work. 

Please advise 

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/89fb52b4-9828-4922-a666-f9db8ff9e849%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

django HttpResponse() issue when literal template characters are part of query string

TL;DR:

Under certain conditions providing literal "{{ %" as the value of a query string parameter caused the HTTP Response to my browser to be un-renderable.

Request Modified via Burp: http://django/charts/?q={{ blahblah %
or
Request Modified via Burp: http://django/charts/?q={{ xxxxxxx &

This results in the response to the browser being treated as plaintext. Firefox generates the following error message:

"The character encoding of the plain text document was not declared. The document will render with garbled text in some browser configurations if the document contains characters from outside the US-ASCII range. The character encoding of the file needs to be declared in the transfer protocol or file needs to use a byte order mark as an encoding signature."

It seems as though django is expecting all query string content to be encoded but obviously with a proxy we can alter this.

Does the HttpResponse() function handle literal templating characters properly when they are part of the request variable that is passed to it?

Additional Information:

In this particular app request.Get.get("variable") is not used.

Based on my limited understanding of django it looks like our programmers have implemented a flow that is something like this:

-Project settings.py:

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',                     <------Request Processor

-Frontend urls.py:

from django.conf.urls import patterns, url, include
from blah.contrib.project.frontend import views

urlpatterns = patterns('',
    url(r'^$', views.index),
    url(r'^charts/', views.charts),

-Frontend views.py:

from django.shortcuts import render, HttpResponse, render_to_response, RequestContext
from django.contrib.auth.decorators import login_required
from django.contrib.auth.decorators import user_passes_test

@user_passes_test(is_user)
def charts(request):
    return render(request, 'main_html/charts.html')

-charts.html:

{% extends "base_html/base.html" %}

-base.html: (Where ?q= is defined finally. Is this considered a global django var?)

    <!-- Left side column. contains the logo and sidebar -->
    <aside class="main-sidebar">
        <!-- sidebar: style can be found in sidebar.less -->
        <section class="sidebar">

            <!-- search form -->
            <form action="#" method="get" class="sidebar-form">
                <div class="input-group">
                    <input type="text" name="q" class="form-control" placeholder="Search...">
              <span class="input-group-btn">
                <button type="submit" name="search" id="search-btn" class="btn btn-flat"><i class="fa fa-search"></i>
                </button>
              </span>
                </div>
            </form>
            <!-- /.search form -->

-Rendering:

https://docs.djangoproject.com/en/1.10/_modules/django/shortcuts/#render

So the built-in request is being passed to the shortcut function render():

• render(request, 'main_html/charts.html')

https://docs.djangoproject.com/en/1.10/_modules/django/template/loader/#render_to_string

• render() under the hood calls render_to_string()
• render_to_string() returns a string to render()
• render() finally calls HttpResponse() with the string-ified content

-Finally:
So is HttpResponse() having an issue because the unparsed request contains the literal templating characters and the result of HttpResponse() is now corrupted?

Or is the problem on our developers side because they did not explicitly define the response parameters when they called render()-->HttpResponse()?

Screenshot: (Modify via proxy to ensure no encoded characters)

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/cd67de62-a1e6-484b-82e5-2377de905d07%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.