GeoDjango distance lookup on Subquery: AttributeError: 'tuple' object has no attribute 'extend'

Good evening,

I'm having a problem doing a distance lookup on a Subquery. I always get an AttributeError (see above).

Put simply, I have a table of Locations where a User may have many Locations (one-to-many). Now, I want to look up which users were close to a Point at a specific time, i.e., get the newest Location which is equal or older than my lookup time T for each user and do a distance query to only include users close by. I chose to do this with a Subquery, but correct me if my approach is not optimal.

This error is triggered in django/contrib/gis/db/models/lookups.py", line 143.

Interestingly, sql_params is a Tuple in this case, which of course doesn't allow .extend() to be called. However, I am not proficient enough to fully understand the stacktrace, especially at which point sql_params suddenly becomes a Tuple instead of a List, which it should be.

Full stacktrace at the bottom of this post.

Simplified reproduction:

models.py

from django.contrib.auth.models import User
from django.contrib.gis.db import models

class Location(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE,
                             related_name='locations')
    location = models.PointField()
    created_at = models.DateTimeField(auto_now=True)

views.py or in shell

from datetime import timedelta
from django.contrib.auth.models import User
from django.contrib.gis.geos import Point
from django.contrib.gis.measure import D
from django.db.models import OuterRef, Subquery
from django.utils import timezone

from .models import Location

some_time = timezone.now() - timedelta(days=1)
some_location = Point(49.1, 110.2)
radius = D(m=1000)

# get the a location for each user which is less than or equal
# to our lookup time some_time
locations = Location.objects \
                    .filter(user=OuterRef('pk'),
                            created_at__lte=some_time) \
                    .order_by('-created_at')

# select the users which were close by at our lookup time some_time
# this yields the attribute error.
users = User.objects \
            .annotate(location_at_time=Subquery(
                                         locations.values('location')[:1])
                     ) \
            .filter(location_at_time__distance_lte=(some_location, radius))

# interestingly, this similar query without a subquery works
users = User.objects \
            .filter(location__location__distance_lte=(some_location, radius))

when running pdb in django/contrib/gis/db/models/lookups.py in GISLookup.as_sql(),  the values of some variables are these:

(Pdb) sql_params

(datetime.datetime(2017, 7, 30, 22, 5, 44, 866379, tzinfo=<UTC>),)   # <-- Tuple!

(Pdb) lhs_sql

'(SELECT U0."location" FROM "api_location" U0 WHERE (U0."user_id" = (V0."id") AND U0."created_at" <= %s) ORDER BY U0."created_at" DESC LIMIT 1)'

(Pdb) rhs_sql

'%s'

(Pdb) rhs_params

[<django.contrib.gis.db.backends.postgis.adapter.PostGISAdapter object at 0x11275d4a8>, 1000.0]

the next command sql_params.extend(rhs_params) then fails, obviously.

So I suppose there's something fishy going on due to the Subquery.

Would love some help, as I'm pretty much just starting out with Django (and Python) and my source code reading skills are very limited.

Thanks! :)

Full stacktrace:

Traceback (most recent call last):

  File "myproject/philter/tests/test_api.py", line 284, in test_play

    response = self.create_game(self.users[0])

  File "myproject/philter/tests/test_api.py", line 248, in create_game

    response = self.game_view(request)

  File "myproject/venv/lib/python3.6/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view

    return view_func(*args, **kwargs)

  File "myproject/venv/lib/python3.6/site-packages/django/views/generic/base.py", line 68, in view

    return self.dispatch(request, *args, **kwargs)

  File "myproject/venv/lib/python3.6/site-packages/rest_framework/views.py", line 489, in dispatch

    response = self.handle_exception(exc)

  File "myproject/venv/lib/python3.6/site-packages/rest_framework/views.py", line 449, in handle_exception

    self.raise_uncaught_exception(exc)

  File "myproject/venv/lib/python3.6/site-packages/rest_framework/views.py", line 486, in dispatch

    response = handler(request, *args, **kwargs)

  File "myproject/philter/api/views.py", line 149, in post

    possible_question_count = questions.count()

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/query.py", line 364, in count

    return self.query.get_count(using=self.db)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/sql/query.py", line 499, in get_count

    number = obj.get_aggregation(using, ['__count'])['__count']

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/sql/query.py", line 463, in get_aggregation

    outer_query.add_subquery(inner_query, using)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/sql/subqueries.py", line 209, in add_subquery

    self.subquery, self.sub_params = query.get_compiler(using).as_sql(with_col_aliases=True)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 420, in as_sql

    where, w_params = self.compile(self.where) if self.where is not None else ("", [])

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 373, in compile

    sql, params = node.as_sql(self, self.connection)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/sql/where.py", line 79, in as_sql

    sql, params = compiler.compile(child)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 373, in compile

    sql, params = node.as_sql(self, self.connection)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/fields/related_lookups.py", line 96, in as_sql

    return super(RelatedIn, self).as_sql(compiler, connection)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/lookups.py", line 381, in as_sql

    return super(In, self).as_sql(compiler, connection)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/lookups.py", line 170, in as_sql

    rhs_sql, rhs_params = self.process_rhs(compiler, connection)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/lookups.py", line 372, in process_rhs

    return super(In, self).process_rhs(compiler, connection)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/lookups.py", line 229, in process_rhs

    return super(FieldGetDbPrepValueIterableMixin, self).process_rhs(compiler, connection)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/lookups.py", line 100, in process_rhs

    sql, params = compiler.compile(value)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 373, in compile

    sql, params = node.as_sql(self, self.connection)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 420, in as_sql

    where, w_params = self.compile(self.where) if self.where is not None else ("", [])

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 373, in compile

    sql, params = node.as_sql(self, self.connection)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/sql/where.py", line 79, in as_sql

    sql, params = compiler.compile(child)

  File "myproject/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 373, in compile

    sql, params = node.as_sql(self, self.connection)

  File "myproject/venv/lib/python3.6/site-packages/django/contrib/gis/db/models/lookups.py", line 143, in as_sql

    sql_params.extend(rhs_params)

AttributeError: 'tuple' object has no attribute 'extend'

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/e7ccec2f-244c-424f-9603-f532931c5109%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.