Re: generate cache by visiting protected links

On 29/07/2010 21:10, Jirka Vejrazka wrote:
>> That's why i asked on what Django uses as a key to set and entry in the cache.
>> If i generate the page and put it on the cache and then rely on Django to get it,
>> the key i use needs to be the same as Django uses otherwise the page isn't found in the
>> cache and the page is generated again and then cached by Django.
>> And that's just what i want to avoid.
>> I'm happy to leave the caching to Django, i just want to "help" cache it initially :)
>
> OK - now I see what you're trying to achieve :)
>
> What is slow for you? Is it the data calculation, or actualy
> constructing the HTML page from the data (read:context) and page
> template?
>
> If it's the former, I'd still cache inside the function that
> calculates the data (with a fixed string key) to get the "transparency
> of the program". Using the "hack" with the cache key you plan to use
> sounds clever, but a bit difficult to pick up if someone else needs to
> read and understand the code later.
>
> Just my 2 cents :)
>
> Jirka
>

The actual data calculation is slowest. The HTML page construction
isn't fast either but i can live with that.

Since putting the cache code inside my view could eventualy lead to problems,
i'm going to take it outside the view and put it in a seperate file.

I still have 2 approaches i can take.

The first using urllib2 to retrieve the page. I just tested this and i can
login to the site and open the correct url.
I'll paste the code here as it shows how to login to a site and setting
up a HTTPCookieProcessor so subsequent calls to the site don't complain
about not being logged in:

====================================
import urllib2, urllib

stats_url = 'http://calltracking:8000/management/statistics/top/user/yearly/'
login_url = "http://calltracking:8000/accounts/login/"
username = 'xyz'
password = '123'

if __name__ == "__main__":
# Log in to the site
o = urllib2.build_opener( urllib2.HTTPCookieProcessor() )
urllib2.install_opener(o)
p=urllib.urlencode({"username" : username, "password" : password})
f=o.open(login_url, p)
data=f.read()
f.close()

# Now open the protected statistics page
pagehandle = o.open(stats_url)
data = pagehandle.read()
pagehandle.close()

# Not necessary, but to verify:
fp = file(r"""c:\test.html""","w")
fp.write(data)
fp.close()
====================================

Second approach is to use Django to access the view function and store
the result in cache. You make a very good point about the transparency.
I haven't thought about it and it's a very good reason to set/get the
pages in cache as it would indeed be clear what i'm trying to achieve.
What i would do then is make a decorator where i cache the page or get it.

If i can get the urllib approach to fully work as intended, i can use the
default Django caching.
If not i'll use the 2nd approach and both would be from an external file
that i can then schedule.

Regards,
Benedict

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.