Friday, October 28, 2011

CacheMiddleware with generation-style caching

Hi all,

I'm trying to implement a generation-style caching system with django
[1]. I found django-jimmypage [2] which implements an alternative
version of @cache_page, and it uses johnny-cache [3] for an "infinite"
cache timeout. However, most of the code in django-jimmypage appears
quite similar to what we've got in the default CacheMiddleware, so I'm
trying to implement this without jimmypage by making the following
minor tweaks:

1) Update settings.py to use the johnny backend for infinite-timeout
support and instruct the cache to use a custom key function:

CACHES = {
'default': {
'BACKEND': 'johnny.backends.filebased.FileBasedCache',
'LOCATION': 'c:\cache',
'KEY_FUNCTION': 'utils.cache.make_key',
'TIMEOUT': 0
}
}

CACHE_GENERATION_KEY = 'generation'

2) Write a custom key function to include the "generation" in the key:

def make_key(key, key_prefix, version):
from django.conf import settings
from django.core.cache.backends.base import default_key_func

if key != settings.CACHE_GENERATION_KEY:
from django.core.cache import cache
return default_key_func(key, key_prefix,
str(cache.get(settings.CACHE_GENERATION_KEY, 1)) + '.' + str(version))
else:
return default_key_func(key, key_prefix, version)

3) Connect to some signals to increment the generation:

def new_cache_generation(sender, **kwargs):
from django.conf import settings
from django.core.cache import cache
try:
cache.incr(settings.CACHE_GENERATION_KEY)
except ValueError:
cache.set(settings.CACHE_GENERATION_KEY, 1)

from django.db.models.signals import post_save, pre_delete
post_save.connect(new_cache_generation)
pre_delete.connect(new_cache_generation)

So far, this part works quite well--no crazy hacking of django. :)

However, my trouble is with CACHE_MIDDLEWARE_SECONDS. From what I can
tell, CACHE_MIDDLEWARE_SECONDS defines *both* the cache timeout *and*
the HTTP headers. For my purposes, this means that if I set
CACHE_MIDDLEWARE_SECONDS = 0 (meaning I want an infinite cache), then
the HTTP Cache-Control headers include max-age=0 which is not what I
want. I'd prefer to define the cache timeout via the cache's TIMEOUT
field, and define the Cache-Control via CACHE_MIDDLEWARE_SECONDS.
'Course, if a view explicitly uses @cache_page() to define a non-
infinite (non-zero) custom timeout, then that timeout should
reasonably impact both the cache timeout and the Cache-Control.

So my question is: does it seem reasonable for me to tweak django to
only use CACHE_MIDDLEWARE_SECONDS for the Cache-Control headers?
Obviously, I can make this change locally, but I'd like to know if I'm
missing something and if this change has any chance of making it
upstream. :)

Thanks!

Jonathan

[1]
http://assets.en.oreilly.com/1/event/27/Accelerate%20your%20Rails%20Site%20with%20Automatic%20Generation-based%20Action%20Caching%20Presentation%201.pdf
[2] https://github.com/yourcelf/django-jimmypage
[3] http://packages.python.org/johnny-cache/

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

No comments:

Post a Comment