Field available via Django shell but not via web application (crossposted from StackOverFlow)

On the web page, I get the following error:

FieldError at /foo/bar/
Cannot resolve keyword 'foos' into field. Choices are: __unused__,
[snip]

The problem code is

User.objects.filter(foos__name='bar')

When I run this in the shell, it works and I get a recordset:

>>> User.objects.filter(foos__name='bar')
[<User: JordanReiter>]

But on the webpage I get the exception above.

Already checked:
- INSTALLED_APPS are identical for both settings
- runserver version also works (as would be expected)
- User used is identical in both cases and is
`django.contrib.auth.models.User`
- related names for the shell's User and the web app's User are
*definitely* different. User._meta.get_all_related_objects() in the
shell displays around 7 more related fields than the if I dump that
from the web app.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Re: Suggestions on "Loading" for possible long load-times?

On Wed, Nov 2, 2011 at 7:37 AM, Kurtis <kurtis.mullins@gmail.com> wrote:
> Hey,
>
> I've got a part of my site that is just prototyped at the moment. It's
> not particularly optimized since it's still in the works.
>
> I don't think it's going to take a considerable amount of time to load
> while we have few users. But, when we start migrating our old users to
> the new system, I can imagine there will be a delay before the
> calculations are finished and the data is ready to display.
>
> Unfortunately, this is real-time calculations so it's not a simple
> matter of caching it to death. I'm not ready to do any major
> optimization at this point anyways since it's still a work-in-
> progress.
>
> Anyways, I want to display a "loading" page while the data is
> prepared. I don't mind if it's a matter of a redirect, done using
> AJAX, or just using some sort of a Javascript technique that will
> display the "Loading" dialog while the rest of the data is sent to the
> browser. What are some suggestions on doing this with Django? I prefer
> a simple, straight-forward method that won't cause a lot of
> complication with the existing code base if at all possible.

There's a very common pattern for this kind of activity:

1) User requests /start page to get some data that is expensive to calculate

2) View for /start creates a Task object that represents the work to
be executed in the background.

3) An external process is running, waiting for new Task objects to be
created. Exactly how this is serviced is up to you, but the important
part is that the expensive calculation isn't performed in the view
code -- it's handled by a process external to the web server. Celery
is a common way to handle this sort of task; you can also knock
something together quickly with a script running under a cron (run
once per minute, find all new tasks, process each, then quit. Not
ideal for production, but it works as a proof of concept).

4) Once the task has been *created* (Not finished -- just created),
/start redirects to a /loading page.

5) /loading contains an ajax lookup that polls the completion status
of the task

6) The task executes and completes; on completion, the task is marked
as "complete".

7) Once the task is complete, /loading reports the completion, and
redirects to /finished to display the results of the calculation.

At every step, the web server's interaction is short lived -- create a
task; check if the task is complete; display the result of the task --
so it scales well. If you're careful about the design of your task
processor, you can make sure that this processor scales too; e.g., add
a second Celery processor for the task, and you've just doubled your
capacity.

Hope this helps!

Yours,
Russ Magee %-)

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Re: User data being exposed with mod_wsgi/apache

On Tue, Nov 1, 2011 at 4:40 PM, Jennifer Bell <jenniferlianne@yahoo.ca> wrote:

   def __init__(self,data=None,files=None,initial={},first_update=False,user=None, report=None):
      if user and user.is_authenticated() and UserProfile.objects.filter(user=user).exists():
              initial[ 'author' ] = user.first_name + " " + user.last_name
              initial[ 'phone' ] = user.get_profile().phone
              initial[ 'email' ] = user.email
      super(ReportUpdateForm,self).__init__(data,files=files,initial=initial)

---------------------------------------->

... I'm guessing because the 'initial' declaration in the form
constructor prototype is not on the stack, like I would have thought.
Changing the view to construct the ReportUpdateForm like so:

                 "update_form": ReportUpdateForm(user=request.user, initial={}),

This is a classic learning Python gotcha, see for example: http://www.ferg.org/projects/python_gotchas.html#contents_item_6

which describes the why in a fair amount of detail.

You should really replace initial={} in the __init__ definition with initial=None and replace None with an empty dictionary inside the method body...otherwise you are relying on all ReportUpdateForm callers to do the right thing (that is, always specify initial, even though it's not "required") to prevent that default initial dictionary from getting modified.

Karen
--
http://tracey.org/kmt/

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Suggestions on "Loading" for possible long load-times?

Hey,

I've got a part of my site that is just prototyped at the moment. It's
not particularly optimized since it's still in the works.

I don't think it's going to take a considerable amount of time to load
while we have few users. But, when we start migrating our old users to
the new system, I can imagine there will be a delay before the
calculations are finished and the data is ready to display.

Unfortunately, this is real-time calculations so it's not a simple
matter of caching it to death. I'm not ready to do any major
optimization at this point anyways since it's still a work-in-
progress.

Anyways, I want to display a "loading" page while the data is
prepared. I don't mind if it's a matter of a redirect, done using
AJAX, or just using some sort of a Javascript technique that will
display the "Loading" dialog while the rest of the data is sent to the
browser. What are some suggestions on doing this with Django? I prefer
a simple, straight-forward method that won't cause a lot of
complication with the existing code base if at all possible.

Thanks!

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Django aggregation does excessive GROUP BY clauses

I am doing a very simple aggregation using the Django ORM, and it is
producing a GROUP BY clause that includes the data field, which is
very large, and is slowing down the query by over 100-fold.

Here is a simplified version of the model:

class Document(models.Model):
data = models.TextField()

class Attachment(models.Model):
document = models.ForeignKey(Document)

And the query I am running:

Document.objects.annotate(num_attachments=Count('attachment'))

And the SQL output:

SELECT
`document_document`.`id`,
`document_document`.`data`,
COUNT(`document_attachment`.`id`) AS `num_attachments`
FROM `document_document`
LEFT OUTER JOIN `document_attachment`
ON (`document_document`.`id` =
`document_attachment`.`document_id`)
GROUP BY
`document_document`.`id`,
`document_document`.`id`,
`document_document`.`data`
ORDER BY NULL

Doing GROUP BY on the data field is unnecessary and ridiculous. I can
stop this by doing a values query:

Document.objects.values('pk').annotate(num_attachments=Count('attachment'))

But then how do I get a real, annotated Document query as the result?

Note: I cross-posted this to StackOverflow:
http://stackoverflow.com/questions/7973461/django-aggregation-does-excessive-group-by-clauses

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Re: User data being exposed with mod_wsgi/apache

Well... you were right. The problem was with my code. As a public
service, the code below will expose the data of a logged in user for
anyone viewing the site:

BAD code -------------------------------->

view.py:

def show( request, report_id ):
report = get_object_or_404(Report, id=report_id)
return render_to_response("reports/show.html",
{ "report": report,
"update_form": ReportUpdateForm(user=request.user),
context_instance=RequestContext(request))

form.py:

class ReportUpdateForm(forms.ModelForm):

class Meta:
model = ReportUpdate
fields = ('desc','author','email','phone','is_fixed')

def
__init__(self,data=None,files=None,initial={},first_update=False,user
= None, report=None):
if user and user.is_authenticated() and
UserProfile.objects.filter(user=user).exists():
initial[ 'author' ] = user.first_name + " " +
user.last_name
initial[ 'phone' ] = user.get_profile().phone
initial[ 'email' ] = user.email
super(ReportUpdateForm,self).__init__(data,files=files,
initial=initial)

---------------------------------------->

... I'm guessing because the 'initial' declaration in the form
constructor prototype is not on the stack, like I would have thought.
Changing the view to construct the ReportUpdateForm like so:

"update_form": ReportUpdateForm(user=request.user,
initial={}),

puts the values on the stack, instead of in the apparently persistent
dict declared in the constructor prototype. This was confirmed with a
unit test:

-------------------------------->

def test_update_form(self):
# check that default values are already filled in.
c = Client()
r = c.login(username='user1',password='user1')
url = '/reports/4'
r = c.get( url )
self.assertEquals( r.status_code, 200 )
self.assertContains(r,"Clark Kent")
self.assertContains(r,"user1@test.com")
self.assertContains(r,"555-111-1111")

# check that default values are NOT already filled in
# for a second anonymous client (problem in the field)

c2 = Client()
r = c2.get( url )
self.assertEquals( r.status_code, 200 )
self.assertNotContains(r,"Clark Kent")
self.assertNotContains(r,"user1@test.com")
self.assertNotContains(r,"555-111-1111")

----------------------->

Which passes or fails according to the change above. Thank you for
your advice.

Jennifer

On Oct 25, 2:36 pm, Daniel Roseman <dan...@roseman.org.uk> wrote:
> On Monday, 24 October 2011 23:14:40 UTC+1, Jennifer Bell wrote:
>
> > On my site, some user data is automatically filled in to a form if a
> > user is logged in by accessing request.user in the view code.
>
> > On deployment, it seems that if *any* user is logged in, forms
> > requested via another browser will be filled in with their data.  The
> > data is not filled in if no user is logged in.
>
> > I'm mystified.  Where is this coming from?  I'm using django 1.3, and
> > caching is not enabled in my settings (though I have set
> > CACHE_MIDDLEWARE_ANONYMOUS_ONLY=True just in case).
>
> > The WSGIDeamonProcess is set up like this:
> > WSGIDaemonProcess lalala user=lalala group=lalala threads=1
> > processes=3
>
> > Is this apache?  mod_wsgi?
>
> > Jennifer
>
> No, it's your code. You've got something somewhere that's providing default
> arguments to your form, but is doing so at the module or class level rather
> than per-request. You'd better show your form and view code.
> --
> DR.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

Re: Django app, part 1, error with the database

Hi,
Thanks for your messages. The problem was indeed some missing commas
after 'ENGINE': ' ' and 'NAME': ' '
Best
-Nicolas-

On Nov 1, 6:32 pm, Sandro Dutra <hexo...@gmail.com> wrote:
>  'NAME': 'C:/Users/songbird/Desktop/bdd/' # Or path to database file
> if using sqlite3.
>
> You're using a SQLite database, so you've to provide the full path to
> SQLite database file, for example:  'NAME':
> 'C:/Users/songbird/Desktop/bdd/mydb.db3' instead only the path to
> directory.
>
> 2011/11/1 Nikolas Stevenson-Molnar <nik.mol...@consbio.org>:
>
>
>
>
>
>
>
> > Hi Nicolas,
>
> > It looks like you have a syntax error in your settings file. It's
> > possible you left of a comma after one of your dictionary values, or
> > didn't close a brace. I can't tell for sure without more context: could
> > you please provide the complete "DATABASES" section of your settings file?
>
> > On 11/1/2011 9:46 AM, Nicolas wrote:
> >> Dear all,
> >> I'm completly new to the world of django and python.
> >> I'm trying to go through the tuto "Writing your first Django app, part
> >> 1" and of course I have an error.
> >> This error happens in the step "database setup". I edited the settings
> >> file, wrote 'django.db.backends.sqlite3' for engine and a path to a db
> >> file (as it's said in the tuto that "If the file doesn't exist, it
> >> will automatically be created when you synchronize the database for
> >> the first time" I just set a path to an empty folder).
> >> And then, I run manage.py syncdb and I get this :
>
> >> C:\Python27\Lib\site-packages\django\bin\mysite>manage.py syncdb
> >> Traceback (most recent call last):
> >>   File "C:\Python27\Lib\site-packages\django\bin\mysite\manage.py",
> >> line 9, in <
> >> module>
> >>     execute_from_command_line(sys.argv)
> >>   File "C:\Python27\lib\site-packages\django\core\management
> >> \__init__.py", line
> >> 422, in execute_from_command_line
> >>     utility.execute()
> >>   File "C:\Python27\lib\site-packages\django\core\management
> >> \__init__.py", line
> >> 361, in execute
> >>     self.fetch_command(subcommand).run_from_argv(self.argv)
> >>   File "C:\Python27\lib\site-packages\django\core\management
> >> \__init__.py", line
> >> 234, in fetch_command
> >>     app_name = get_commands()[subcommand]
> >>   File "C:\Python27\lib\site-packages\django\core\management
> >> \__init__.py", line
> >> 102, in get_commands
> >>     apps = settings.INSTALLED_APPS
> >>   File "C:\Python27\lib\site-packages\django\utils\functional.py",
> >> line 185, in
> >> inner
> >>     self._setup()
> >>   File "C:\Python27\lib\site-packages\django\conf\__init__.py", line
> >> 42, in _set
> >> up
> >>     self._wrapped = Settings(settings_module)
> >>   File "C:\Python27\lib\site-packages\django\conf\__init__.py", line
> >> 90, in __in
> >> it__
> >>     mod = importlib.import_module(self.SETTINGS_MODULE)
> >>   File "C:\Python27\lib\site-packages\django\utils\importlib.py", line
> >> 35, in im
> >> port_module
> >>     __import__(name)
> >>   File "C:\Python27\Lib\site-packages\django\bin\mysite\mysite
> >> \settings.py", lin
> >> e 15
> >>     'NAME': 'C:/Users/songbird/Desktop/bdd/' # Or path to database
> >> file if using sqlite3.
> >>           ^
> >> SyntaxError: invalid syntax
>
> >> I'm on windows 7.
> >> So, my question is simple: what am I doing wrong?
> >> Thanks a lot in advance.
> >> Best
> >> -Nicolas-
>
> > --
> > You received this message because you are subscribed to the Google Groups "Django users" group.
> > To post to this group, send email to django-users@googlegroups.com.
> > To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
> > For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.