Re: Sync local Database with remote REST API

Has anyone ever come up with a best practices approach for this? Particularly if the remote service is the primary one.

On Monday, March 17, 2014 at 7:40:01 PM UTC-4, Leonard wrote:

Hello everybody, I'm having an hard time to figuring this out.
I'm designing a django project based on data retrieved from a remote public REST API that will:

• Retrieve data from a REST API
  
• Saving the data in my models (my local db)
  
• Analyze and process the local data asynchronously and create statistics and trends
  

When a user request the data:

• Check if the local data (models) are in sync with the remote, otherwise fetch the remote data and save them in the local db
  
• Present the data
  

What is the best approach to do this? 
My questions:

1. Should I design my models replicating all the remote API fields?
   
• Can I use JSonField and store data in json format to speed things up?
  
• In this case would it be complex to calculate statistics on the data?
  In other words I can't use the django ORM and foreign keys if I store data in JSON format, right?
2. Where should I call the remote API?
• Considering that the remote data should "replicate" the database models, first I tough to add model methods that call the REST API inside the models
  
• Or I could put them in the manager, considering that when a user needs the data he/she doesn't need to know if they are local or remote
• How do I tell django to grab the remote data and put them in db before presenting them?
• Maybe this should be an asynchronous task? 

Sorry for the complexity, I hope I was clear!

Thanks in advance for your help

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/d9ab7a2b-1266-40b1-a478-2f7362b600e6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.