Friday, July 30, 2010

Re: generate cache by visiting protected links

On 30/07/2010 12:06, Jirka Vejrazka wrote:
>> I still have 2 approaches i can take.
>
> I still believe that there is a 3rd option there - to bypass the
> view / auth / login machinery completely :)
>
>
> If your primary goal is to cache the statistics and the statistics
> does not depend on the user that is logged in (you did pass a user
> with pk=1 in one of your previous emails), then you don't really need
> to call any view to cache the data. You can call the function that
> calculates the statistics (that would cache the data internally within
> the function), then call this function from anywhere upon start of
> your webserver (or cron job, as mentioned before).
>
> As long as you have a global cache (i.e. memcached), this would work
> very nicely and you would not have to do any urllib2 login magic.
>
> Let me give you a short example:
>
> def stats_top_callers_per_year(some_parameters_not_tied_to_a_specific_user):
> cache_key = 'stats_top_callers_per_year'
> data = cache.get(cache_key)
> if data:
> return data
> # calculate the data somehow
> cache.set(cache_key, data) # <- set appropriate timeout there
>
>
> Then have small script called before (or shortly after) your web
> server start, that would import this function from your code, hence
> calculating the statistics. Then, as long as you use memcached, your
> web would use the cached statistics every time the
> stats_top_callers_per_year() would be called frmo your views.
>
> Jirka
>

Hi Jirka,

your approach is better, and cleaner.
I will do it that way.
It boils down to caching the results of querysets rather than caching pages.
I'll post the stats function i've been using to test all this:

@check_can_view_stats
@csrf_exempt
def stats_top_callers_per_year(request):
# Statistics for the top 30 callers
# Choose different base ( & menu) according for staff or other user
if ( request.user.is_staff ):
extends_from = "management/base.html"
else: extends_from = "management/public_base.html"
# We'll use a simple class that will serve as the base for the individual data
class Stats(object):
pass
stats_all = {}
# We want the last 5 years
current_year = datetime.datetime.now().year
archive_year = current_year - 4
years = xrange(archive_year, current_year+1)
for year in years:
stats_year = []
counts = {}
# Initialize count dict
# If we don't do this, we get a KeyError
for user in User.objects.all():
counts[user]=0
# Count the times an user has called (called initiator here)
for i in Call.objects.filter(date_created__year=year):
for _init in i.initiator.all():
counts[_init] += 1
# Sort the dictionary
count_sorted = sorted(counts.items(), lambda x, y: cmp(x[1], y[1]), reverse=True)
# Make the stats of the top 30 users ready for the template
for user_stat in count_sorted[0:30]:
if ( user_stat[1] > 0 ):
st = Stats()
st.calls = user_stat[1]
st.user = user_stat[0]
stats_year.append(st)
stats_all[year]=stats_year
stats_sorted = sorted(stats_all.items(), lambda x, y: cmp(x[0], y[0]), reverse=True)
return render_to_response('management/stats.html', {'extends_from': extends_from, 'stats_year': stats_sorted},
context_instance=RequestContext(request))


I'm sure there is still room for improving the way i construct my data but i haven't
yet reached that phase in the project :)

Caching the stats_sorted data would be enough i think.
I calculate here which users have been bothering our ICT the most. It's a vital stat :) ;)

Also, my 1ste approach comes with a hitch.
According to who logs in, you get a different menu and there is also a login button with
the username on it.
This is also cached so when you visit the stats page under your own user, you see
the stats correcty but the logout part contains the administrator name instead of your own.
In other words, the menu & login/logout button needs to be excluded from cache.
As i've found out, there is no standard way of doing that in Django and you need to use
a custom tag & middleware:
http://www.holovaty.com/writing/django-two-phased-rendering/

I've tried to set it up but it failed so far.

As it turns out, your method is clearer, simpler and does what i want :)
Building a nice decorator from it that takes the function name as key would be ideal
so i'll do it that way.

Thanks,
Benedict


--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

No comments:

Post a Comment