DeWitt Clinton

A caching decorator pattern for django views in App Engine

Step 1)

Download and put decorator.py from Michele Simionato’s decorator module in your path.

Step 2)

Add the following generic memcaching decorator to your views.py:

import logging
from google.appengine.api import memcache
from decorator import *
CACHE_EXPIRATION = 3600

def cacheable(keygen=None, expiration=CACHE_EXPIRATION):
  """A decorator that caches results in memcache.
  
  keygen: 
    A function that returns the cache key based on the *args and
    **kwargs of the function being called.  If keygen is 
    not specified, the first positional argument will be used.
  expiration:
    The length of time to cache the response in seconds.
  """
  # Define the decorator itself as a closure within cacheable
  def call(f, *args, **kwargs):
    # Don't use the cache at all if there is no expiration
    if not expiration:
      return f(*args, **kwargs)

    # Use the supplied keygen to create a local cache key
    # or use the first positional arg if none is supplied
    if keygen:
      local_key = keygen(*args, **kwargs)
    else:
      local_key = args[0]

    # Create a global cache key that remains stable across instances
    global_key = '%s:%s:%s' % (f.__module__, f.__name__, local_key)

    logging.debug('Checking cache for %s' % local_key)
    result = memcache.get(global_key)
    if result:
      logging.debug('Found %s in cache.' % local_key)
    else:
      logging.debug('Cache miss for %s.' % local_key)
      result = f(*args, **kwargs)
      logging.debug('Caching %s' % local_key)
      if not memcache.add(global_key, result, expiration):
        logging.error('Error caching response for %s.' % local_key)
    return result

  return decorator(call)

Step 3)

Define a key generation function for http requests. This one ignores query parameters, but yours probably shouldn’t. By default the caching decorator uses the first parameter of the function call as the key, but in the case of http requests, we’ll use the request path.

def request_keygen(request, *args, **kwargs):
  """Returns a key based on the request path.                                                                                                                                                                                     
                                                                                                                                                                                                                                  
  Args:                                                                                                                                                                                                                           
    request: The http request object.                                                                                                                                                                                             
  """
  if not request or not request.path:
    raise ServerError('First parameter must be a request with a path.')
  return request.path

Step 4)

Apply the decorator to any view corresponding to a GET request. The key will be the request.path, so if your logic uses query parameters be sure to modify request_keygen accordingly.

@cacheable(keygen=request_keygen)
def HomeView(request):
  """Prints the homepage"""
  return render_to_response('home.tmpl')

Step 5)

Use the decorator anywhere you’d like to cache the response in memcache.

@cacheable()
def get_url(url):
  """Retrieves a URL and caches the results.                                                                                                                                                                                      
                                                                                                                                                                                                                                  
  Args:                                                                                                                                                                                                                           
    url: A url to be fetched                                                                                                                                                                                                      
  Returns:                                                                                                                                                                                                                        
    a http response                                                                                                                                                                                                               
  """
  return urlfetch.fetch(url)

Step 6)

Sprinkle liberally.