Thursday, January 19, 2017

Re: A Backup Model in Django?

I'm finally round to implementing this, and so wanted to send a warm thanks to those that contributes their thoughts and ideas. I think I'm on a winner with an abstract class and two children, one the backup of the other.

To those you recommended full database backup and/or transaction, my apologies for my lacking clarity. I'm well aware of the need for both and both are on the agenda, but not mysteries (well documented) and no need for me to ask about. Abstract classes were the clue I missed in my read of docs.

To clarify for anyone interested perchance, not to justify, just to clarify, the use case is a little like this:
  • Raw data stored
  • An expensive set of statics is calculated on that raw data and the aim of the whole site is to see these statistics. They too are abundant. But very expensive to calculate from the raw data. Well truth be told it's taking about 5 minutes with a few hundreds of raw data records to process. Pricey.
  • For this reason the statistics are calculated on the fly as raw data is added and stored, so that they can be retrieved quickly on demand.
  • Said statistics depend on a handful of configuration parameters, that need to be tuned as data is collected. Changing them demands a rebuild of all the stats.
  • When doing such a rebuild it's very desirable to know what changed. In short to rebuild a table of stats and compare it with the table as it was before the rebuild and summarise what was impacted by the rebuild, and conceivably to undo if the impact is deemed undesirable (not an improvement).
  • This could all over time involve rather serious quantities of data too - and stats on stats are the consequence, that is a table of calculated stats from raw data and configs, a backup of those stats before a config change, and stats on the differences between these tables ...

And the best way I see of doing that is to have two identically defined  tables, back one up, rebuild, compare.

To wit, the abstract model and two identical tables suits my needs and is what I'm now in process of implementing.

Of course data entry is transactioned and the database will be backed up regularly, but they are separate issues.


Regards,

Bernd.

On Sunday, 16 October 2016 17:40:15 UTC+11, Bernd Wechner wrote:
A curious question I've had trouble finding an answer for alas. I have a model that I'd like to backup in the database in a backup model. This being the pro-forma so to speqk:

from django.db import models

class MyModel(models.Model):
 
# Declare fields ....

class MyModel_backup(MyModel):
 
def create(self):
 
self.objects = MyModel.objects.all()
 
But there are two immediate problems.
  1. Deriving from MyModel reveals itself in the migration to be generating a model which has a single OneToOne reference to MyModel. That is ti does not appear to create a duplicate model at all. Which leaves me wondering how to create a duplicate model without repeating the code. 

  2. I have no really idea how to copy all the objects of MyModel to a new model.
I may be approaching it poorly and am open to better ideas. I'm used to doing it in SQL, essentially having an identically defined backup table, just copying data to that table before doing a (risky) table wide operation on the first. 

I could of course export a serialized backup to a disk file, but am exploring options for keeping one backup in the database itself. 

I'd rather, I admit hear options for doing that than philosophic appraisals of the benefits of an in-database copy vs, database exports. 

Regards,

Bernd.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/9e5fa74d-3027-4928-8be4-7fcdc1b51beb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

No comments:

Post a Comment