Tuesday, February 1, 2011

Re: Storing *big* texts in db fields

(I tried to post this once before, but the browser crashed so I am not
sure if it got posted. Apologies for dupe content if so)

Is it possible the wordcount function is slowing everything down? Does
it run each time you view the admin page?
If so, I would try adding it as an attribute to the model and counting
the words when the model is saved (using the
post_save signal), instead of counting the words each time the admin
page is displayed (if that's what you are doing).

On Feb 1, 2:22 am, Karen McNeil <karenlmcn...@gmail.com> wrote:
> I've created an application to manage texts, storing the content in a
> TextField, along with metadata in other fields. Now that I've
> completed the model and started actually adding the texts, the admin
> is getting verrrry slow. The app is just for the use of me and my
> team, so the slowness is not a deal-breaker, but it's annoying to work
> with and I still have a lot of texts to add to the corpus.
>
> Although I may be adding a large amount of smaller texts in the
> future, the texts that I have now are large, mostly in the tens of
> thousands of words, with the largest currently at 101,399 words.
> (Which I know because I added a method to the model to calculate the
> wordcount, and have it displayed in the admin list. Which gives me no
> end of pleasure.)
>
> So, is it a bad idea to be storing texts this large in a database
> field? I really hope not, because when I first started this project
> (granted, before I started using Django), I was reading the data from
> files and running into constant encoding/decoding problems. (These
> texts I'm collecting are in Arabic.)
>
> If it's not a totally horrible idea to do this like I'm doing, is
> there anything I can do to improve performance?  I tried implementing
> caching and it didn't make any difference.
>
> Thanks,
> Karen

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

No comments:

Post a Comment