Tuesday, August 25, 2015

Re: Loading scikit model in Django


> Thanks you James for your suggestion. I am going to look into the memcached now.
>
> I was thinking that the ideal solution will be to load the model into a global variable, then the request can use refer to this variable for the model. I don't know if that possible.
>

Possible...yes. Good idea? Well, let's just say a cache is what you need in this case. I'm hoping that your objects can be serialized or pickled, that will make them much easier to store in the cache. This will also make it easier to support concurrent users with different objects in the cache.

The tough part is going to be maintaining a consistent state for the object in the cache if the user is making changes to it (updating the instance and then updating the cache with the new object so that further request are working with current data).

You may also want to consider handling changes via a batch job processor such as Celery if the updates are lengthy (locking the object for updates, retrieving the object, updating it, saving it to the DB or a file, and then updating the cache with the new version). That way your users get an instant response saying that their changes have been submitted and the updated version should be available shortly. It really depends on your user base and the timing of operations.

Start with implementing the cache, once that is working, look at a batch processor like celery if the response is still less than desirable, since there is a fair amount of setup required to get a batch processor online.

I've never worked with custom memcached entries, so I've reached the end of my knowledge scope here.

-James

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CA%2Be%2BciWAWUOBetHzxWysq4WXiUhnvftf%2BJ0N-p%3Djs2zWYt4Hhw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

No comments:

Post a Comment