Friday, November 1, 2013

Complex query reduction

I have been exploring a rather complex query:

# get all the MPTT descendants of entity
entities = entity.get_descendants()

# get all the Researchers in these Entities
researchers = Researcher.objects.filter(person__member_of__entity__in=entities)

# get all the BibliographicRecords for these Researchers
items = BibliographicRecord.objects.filter(authored__researcher__in=researchers).distinct()

In practice I use some tweaks (such as values_list) to speed this up, and caching, but the fundamental pattern is the same. It's slow, because there are 30 thousand BibliographicRecords.

I'm trying to rearrange the construction of the query in different ways to find a speed improvement. I don't think that either prefetch_related or select_related will help here, but perhaps there are some other tools that would.

Any suggestions?

At <https://dpaste.de/8vwP> I've shown the SQL generated, and the models being used.

Thanks,

Daniele

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/20131101174524.207157947%40smtp.modern-world.net.
For more options, visit https://groups.google.com/groups/opt_out.

No comments:

Post a Comment