Re: Possible interest in a webcast/presentation about Django site with 40mil+ rows of data??

I was thinking.. what would be really nice would be a monkey patch that could catch any inserts/updates/saves from the ORM, and defer them until the user triggered the bulk update some how.. That way, DSE would be completely interchangeable with existing code, making it much more likely for people to adopt it. 

On Wed, Jun 22, 2011 at 11:39 PM, Thomas Weholt <thomas.weholt@gmail.com> wrote:

On Wed, Jun 22, 2011 at 4:47 PM, Cal Leeming [Simplicity Media Ltd]

<cal.leeming@simplicitymedialtd.co.uk> wrote:
>
>

> On Wed, Jun 22, 2011 at 3:25 PM, Andre Terra <andreterra@gmail.com> wrote:
>>
>> Hello, Cal
>>
>> First of all, congrats on the newborn! The Django community will surely
>> benefit from having yet another success story, especially considering how
>> big this project sounds. Is there any chance you could open-source some of
>> your custom made improvements so that they could eventually be merged to
>> trunk?
>
> Thank you! Yeah, the plan is to release as much of the improvements as open
> source as possible. Although I'd rely heavily on the community to make them
> 'patch worthy' for the core, as the amount of spare time I have is somewhat
> limited.
> The improvements list is growing by the day, and I usually try and post as
> many snippets as I can, and/or tickets etc.
> It sounds like Thomas's DSE might be the perfect place for the bulk update
> code too.

FYI: Inspired by this discussion I've allready started on a similar
feature ( allthough somewhat simplified ) for DSE v2.2.0 and you're
right; the speed increase is huge using the method described here,
even compared to my current solution ( using cursor.executemany ),
which is considerably faster than the django orm allready. My testing
so far have been using postgresql, not sure how mysql will perform. I
expect to release DSE v.2.2.0 with this feature in the next few days.

--

Mvh/Best regards,
Thomas Weholt
http://www.weholt.org

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.