Friday, July 30, 2010

Re: generate cache by visiting protected links

On 29/07/2010 21:10, Jirka Vejrazka wrote:
>> That's why i asked on what Django uses as a key to set and entry in the cache.
>> If i generate the page and put it on the cache and then rely on Django to get it,
>> the key i use needs to be the same as Django uses otherwise the page isn't found in the
>> cache and the page is generated again and then cached by Django.
>> And that's just what i want to avoid.
>> I'm happy to leave the caching to Django, i just want to "help" cache it initially :)
> OK - now I see what you're trying to achieve :)
> What is slow for you? Is it the data calculation, or actualy
> constructing the HTML page from the data (read:context) and page
> template?
> If it's the former, I'd still cache inside the function that
> calculates the data (with a fixed string key) to get the "transparency
> of the program". Using the "hack" with the cache key you plan to use
> sounds clever, but a bit difficult to pick up if someone else needs to
> read and understand the code later.
> Just my 2 cents :)
> Jirka

The actual data calculation is slowest. The HTML page construction
isn't fast either but i can live with that.

Since putting the cache code inside my view could eventualy lead to problems,
i'm going to take it outside the view and put it in a seperate file.

I still have 2 approaches i can take.

The first using urllib2 to retrieve the page. I just tested this and i can
login to the site and open the correct url.
I'll paste the code here as it shows how to login to a site and setting
up a HTTPCookieProcessor so subsequent calls to the site don't complain
about not being logged in:

import urllib2, urllib

stats_url = 'http://calltracking:8000/management/statistics/top/user/yearly/'
login_url = "http://calltracking:8000/accounts/login/"
username = 'xyz'
password = '123'

if __name__ == "__main__":
# Log in to the site
o = urllib2.build_opener( urllib2.HTTPCookieProcessor() )
p=urllib.urlencode({"username" : username, "password" : password}), p)

# Now open the protected statistics page
pagehandle =
data =

# Not necessary, but to verify:
fp = file(r"""c:\test.html""","w")

Second approach is to use Django to access the view function and store
the result in cache. You make a very good point about the transparency.
I haven't thought about it and it's a very good reason to set/get the
pages in cache as it would indeed be clear what i'm trying to achieve.
What i would do then is make a decorator where i cache the page or get it.

If i can get the urllib approach to fully work as intended, i can use the
default Django caching.
If not i'll use the 2nd approach and both would be from an external file
that i can then schedule.


You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to
To unsubscribe from this group, send email to
For more options, visit this group at

No comments:

Post a Comment