Re: Large Queryset Calculation In Background?

I wouldn't expect it to lock the database (though someone with more database expertise should address that). I would expect it to consume significant CPU. If you're on UNIX, you could address this issue by making your process 'nice': http://docs.python.org/library/os.html#os.nice The nicer a process (higher the value), the less CPU it will hog. IIRC, nice values default to 0 for processes and range from -20 (biggest CPU usage) to +20 (smallest CPU usage).

_Nik

On 11/22/2011 2:37 PM, Nan wrote:

Hi folks --  I need to run a fairly CPU-intensive calculation nightly over a dataset that's already large and growing quickly.  I'm planning to run this via a cron job, but would like to make sure that it neither eats up the entire CPU nor locks the database, so that my site can continue functioning in the meantime.  The rough outline of what it needs to do is as follows:  class OtherThing(models.Model):     anotherthing = models.ManyToManyField(Whatever)     ...  class Thing(models.Model):     other_things = models.ManyToManyField(OtherThing, through='SomethingElse')     ...  for thing in Thing.objects.select_related('other_things', 'other_things__anotherthing__etc'):     calculated = calculation_on_thing_and_its_otherthings(thing) # this mainly involves serialization to a great depth     thing.calculated_data = calculated     thing.save()  Will the above approach lock the database for a while or eat tons of CPU?  Any suggestions?  I'm using Django 1.2, btw.  Thanks, -Nan