Re: Bulk delete - performance / object collection

That's an interesting perspective actually..

"Don't be a slave of your mind (or ORM)"

Cal

On Wed, Jan 2, 2013 at 12:01 PM, Larry Martell <larry.martell@gmail.com> wrote:

On Wed, Jan 2, 2013 at 4:29 AM, George Lund <glund@mintel.com> wrote:
> I'm trying to bulk-delete several million rows from my database.
>
> The docs for Django 1.3 say "this will, whenever possible, be executed
> purely in SQL". A pure-SQL delete is what I want in this case, so that's
> fine.
>
> However, the code is never getting as far as running any SQL.
>
> Interrupting the script shows that the delete method on the QuerySet is
> trying to use a "Collector" to construct model instances for each row I'm
> trying to delete. This is going to take too long (and may in fact consume
> all the memory available) -- I don't think it's practical to wait in this
> case. (I've tried waiting over half an hour!)
>
> (I'm looking at django.db.models.query.QuerySet.delete and
> django.db.models.deletion.Collector.collect / Collector.add.)
>
> What's the point in doing the delete "purely in SQL" if all of the objects
> are getting constructed anyway? Why do they need to be "collected" before
> the SQL DELETE is run? The model instance in this case has no child rows, to
> which the delete might need to be cascaded.
>
> Meanwhile I can construct the SQL by hand easily enough, but I feel this
> isn't doing things the right way.

I've had this discussion with other developers. Many feel they need to
slavishly adhere to the ORM and if you don't you're some type of evil
entity. I could not disagree more. Django and the ORM are tools, and
should be used only when they are the right tools for the job. I'd
just go ahead and do the delete in SQL.

--
You received this message because you are subscribed to the Google Groups "Django users" group.

To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.