Re: Large Queryset Calculation In Background?

On Mon, Nov 28, 2011 at 3:17 PM, Javier Guerra Giraldez
<javier@guerrag.com> wrote:
> On Mon, Nov 28, 2011 at 10:10 AM, Tom Evans <tevans.uk@googlemail.com> wrote:
>> Django currently always fetches the entire result all at once,
>> regardless of how you then fetch the data from the queryset.
>
> but this result isn't the whole queryset result, it's a chunk of it.
> the ORM adds 'LIMIT' arguments to the query.  I think the answer to
> Nan's question is that there's no lock across chunked reads.
>

(NB: you snipped the bit where I say that I am specifically talking
about MySQL - I still am)

No-one mentioned slicing or adding LIMITs into the query - but it is
irrelevant. When you issue a query, the DB tables that are read from
are locked until that data is returned to the client. That happens as
soon as mysql_store_result() in the MySQL C API finishes, at which
point all the data has been transferred from the server to the client.
This happens at the moment that you evaluate your query in django.

When you iterate through or otherwise access that query result in
django, it is no longer talking to the DB server, it does not fetch
any additional data, in chunks or otherwise. The queryset object cache
is built in chunks, but this relates to when Django creates model
instances, not communication with the database.

Cheers

Tom

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.