Re: Importing file to Models - Temporary store to allow confirming?

Shawn,

Thanks for the quick reply =).

If we go with the third approach, what advantages are there to persisting the models in MongoDB (or something similar like Redid.or Cassandra), as opposed to a traditional RDBMS? (I just want to get a good handle on the issues).

Furthermore, is there a particular reason you picked MongoDB over the other NoSQL solutions?

Also, in terms of actually persisting the temporary models to MongoDB, I can just use PyMongo and store the dicts themselves - and just user a nested dict to represent all the model instances from a given import file. Any issues with that approach?

Thanks for the tip about using asynchronous processing for the import file - I didn't think the file would be that big/complex to process, but I suppose it's probably the better approach to do it all asynchronously, instead of just hoping it won't grow larger I the future. In this case, I suppose I can just use Ajax on the page itself to check on the status of the queue?

Cheers,
Victor

On Mon, Jun 27, 2011 at 04:36, Shawn Milochik <shawn@milochik.com> wrote:

If you're using Django 1.2 or higher you can take advantage of the multi-database support by adding the 'using' kwarg to your model.save(). This will allow you to ensure that the model saves successfully (is valid) in the 'holding' database, and move it to your 'live' database later.

You could add a field to the model indicating whether it's 'live' or 'pending,' load all the temp models as pending, then just flip the flag as each one is "approved." That would front-load all the processing.

You can use a ModelForm to accept the CSV data (preferably in dict form, from csv.DictReader) to validate the data, then just check for is_valid() without saving anything to the database. You could then store those dictionaries somewhere (such as MongoDB) for retrieval when you actually want to save them to the database.

Each of these options has different advantages. I don't know about performance. In any case, you may have to use asynchronous processing (via ZeroMQ or django-celery) so the page can refresh before you Web server's timeout.

Use whichever seems best for your needs, and maybe someone else has another option. I'd prefer option #3, because it keeps all the temporary data out of your production database and makes good use of the validation properties of the ModelForm class.

--
You received this message because you are subscribed to the Google Groups "Django users" group.

To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.