Tuesday, August 25, 2015

Re: Loading scikit model in Django


> Hi guys, I have searched the answers for days with no avail, thus I would like to ask here.
>
> I am running a svm scikit model in Django, currently, every request have to load the ~20mb scikit model, and it was really slow.
>
> I wonder if I should load the model in settings.py, and store in in session? However, the scikit model can be updated by a certain user.
>
> Any comment? I am rather new to django, thank in advance.

Have you thought at all about using a cache system such as memcached? The initial load would probably be slow, but it's possible that subsequent access to that same object would see a speed improvement. You would need to establish a workflow that ensures the object stays up to date whenever changes are made, but repeated reads would likely see a substantial improvement.

Storing that amount of data in the session would not have the desired effect, if anything, things would probably get worse, depending on the backend used to store sessions. Sessions are reloaded during every request (and therefore probably not addressing your issue), whereas a cache system is meant to hold things in an easy/quickly accessible format and location, independent of the request/response cycle.

-James

https://docs.djangoproject.com/en/1.8/topics/cache/#memcached

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CA%2Be%2BciWYbdmvuTcsf2XN%3DUT4ZCCmMfOVOQgSq4pgLKboOYHqQg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

No comments:

Post a Comment