Monday, November 26, 2012

Re: count resulting rows in sliced valuesqueryset

I did some research on windowing functions since your post. They tend to be slower since they ignore optimizations. For example, when using count(*), indexes are used whenever possible to avoid hitting the table data and outer joins are ignored altogether.  And limited searches quit as soon as they reach the proper count.

One case where it might be faster to combine them is looking up the last page. For example:
count = qs.count()
qs = qs[ count-100 : count ]

When I run my slow query on that, I get 0.2 sec + 1.9 sec. YMMV. That's why there's https://docs.djangoproject.com/en/1.5/ref/models/querysets/#reverse 

On Nov 24, 2012, at 4:06 PM, ?manu* wrote:

On Saturday, November 24, 2012 8:03:06 AM UTC+1, Peter of the Norse wrote:
On Nov 21, 2012, at 3:53 AM, ?manu* wrote:

> Suppose I have a queryset qs. For paginating purposes I need to do something like:
>
> count = qs.count()
> qs = qs[0:100]
>
> Unfortunately this executes the query twice, which I don't want.

Are you sure? This is such a common pattern that I suspect that it's not slower than making it into one query. I ran some tests on the slowest query I have, and the two statements were faster than trying to combine them. 0.2 + 1.5 sec vs. 1.9 sec.

You are right! (thanks also to Javier). It is not clear to me how it is possible but effectively it seems that the two queries are not slower than a single one...

Thank you also for the other answers.

E.
 
Peter of the Norse



No comments:

Post a Comment