Re: Massive insertion of records

Thanks Craig - that explains it very clearly indeed ("textbook answer"!)

On 27 December 2010 16:50, W. Craig Trader <craig.trader@gmail.com> wrote:

Derek ...

Quoting from http://docs.djangoproject.com/en/1.2/topics/db/transactions/:

Django's default behavior is to run with an open transaction which it commits automatically when any built-in, data-altering model function is called. For example, if you call model.save() or model.delete(), the change will be committed immediately.

This is much like the auto-commit setting for most databases. As soon as you perform an action that needs to write to the database, Django produces the INSERT/UPDATE/DELETE statements and then does the COMMIT. There's no implicit ROLLBACK.

Using the Django transaction API, you can manually control transactions.  In this case, Silva must be doing something like this:

from django.db import transaction

@transaction.commit_manually
def viewfunc(request):
       ...
    # You can commit/rollback however and whenever you want
    i = 0
    for x in data:
        ... save something ...
        i += 1
        if i % 1000:
             transaction.commit()

    # One last commit to persist the last batch of saves
    transaction.commit()
   

I will also note that the transaction API can be (and should be) used when you're writing management commands.

 - Craig -

On Mon, Dec 27, 2010 at 05:29, derek <gamesbook@gmail.com> wrote:

Can you explain what you mean by "manual commitments" (preferably with
a code example)?

Thanks
Derek

On Dec 26, 4:38 am, Silva Paulo <psdasi...@yahoo.com> wrote:
> OK. I found a "solution".
> Using manual commitments after lots of insertions (ex. >1000) makes the massive insertion very fast (at least for my needs).
>
> Thanks to all who answered.
>

> --- On Fri, 12/24/10, Silva Paulo <psdasi...@yahoo.com> wrote:
>
>
>
>
>
>
>
> > From: Silva Paulo <psdasi...@yahoo.com>

> > Subject: Massive insertion of records
> > To: "newbie" <django-users@googlegroups.com>
> > Date: Friday, December 24, 2010, 6:26 PM
> > I need to do a massive insertion of
> > records in two tables "connectd" by "foreignkey". Is there a
> > way, using the Django db API, to do it. e=Foo(...);e.save()
> > seems too slow.
>
> > Thanks
>
> >      
>
> > --
> > You received this message because you are subscribed to the
> > Google Groups "Django users" group.
> > To post to this group, send email to django-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-users+unsubscribe@googlegroups.com.

> > For more options, visit this group athttp://groups.google.com/group/django-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.