Here at matchFWD we A/B test our templates, both web and e-mail. To facilitate this we use a handy little wrapper function for Django’s render_to_response shortcut which I’ll share with you below, then describe the how/why.

# encoding: utf-8
 
from random import randrange
from os import listdir
from os.path import join, exists, isdir, splitext
 
from django.conf import settings
from django.core.cache import cache
from django.shortcuts import render_to_response
 
from matchfwd.utils.caching import generate_keyname
 
 
def render_round_robin_to_response(template, *args, **kw):
    # First we split the name at the extension.
    base, extension = splitext(template)
    possibilities = []
 
    # Now we calculate if this is a single file (silly) or a directory (round-robin).
    for django_template_dir in settings.TEMPLATE_DIRS:
        fname = join(django_template_dir, template)
        dname = join(django_template_dir, base)
 
        if exists(fname):
            possibilities.append(template)
            break
 
        if isdir(dname):
            for fname in sorted(listdir(dname)):
                # Protect ourselves from hidden files and unexpected file types.
                if fname[0] == '.' or splitext(fname)[1] != extension:
                    continue
 
                possibilities.append(join(base, fname))
 
            break
 
    else:
        raise ValueError("Unknown template: " + template)
 
    # No point mucking with the cache if there is only one option!
    if len(possibilities) == 1:
        return render_to_response(possibilities[0], *args, **kw)
 
    key = generate_keyname(template)
    index = cache.get(key, randrange(0, len(possibilities)))
    template = possibilities[index]
 
    cache.set(key, (index + 1) % len(possibilities))
 
    return render_to_response(template, *args, **kw)

That’s quite a function; now let’s see what’s going on in there! Skipping the import preamble:

def render_round_robin_to_response(template, *args, **kw):

Since this is a drop-in replacement for render_to_response we accept unlimited positional and keyword arguments to pass along to the renderer. We only care about the template name being requested.

    for django_template_dir in settings.TEMPLATE_DIRS:
        fname = join(django_template_dir, template)
        dname = join(django_template_dir, base)

This calculates the real filename and filename without extension as a directory name for every template directory registered in our settings.py file. Those with sharp eyes will notice that fname is re-used later, but only if the file doesn’t actually exist.

        if exists(fname):
            possibilities.append(template)
            break

We “exit early” if a template with the given name actually exists. This allows us (if we wanted to) to use this function everywhere we use render_to_response without worrying about breakage.

        if exists(dname) and isdir(dname):
            for fname in sorted(listdir(dname)):

Now, if the directory with the same name as the template (without extension) exists, we iterate the directory contents.

                # Protect ourselves from hidden files and unexpected file types.
                if fname[0] == '.' or splitext(fname)[1] != extension:
                    continue
 
                possibilities.append(join(base, fname))

Here we skip over files that are ‘hidden’—such as the bane of every Mac developer’s life, .DS_Store files, and more esoterically the extended attribute ‘ghost’ file which is associated with all other files—and explicitly search for files sharing the original template’s filename extension. For each valid file found we append the path to the list of possibilities.

    # No point mucking with the cache if there is only one option!
    if len(possibilities) == 1:
        return render_to_response(possibilities[0], *args, **kw)

The comment pretty much says it all; for two cases (real template filename or directory with only one valid match) we exit early and avoid overhead of playing with the cache.

    key = generate_keyname(template)
    index = cache.get(key, randrange(0, len(possibilities)))
    template = possibilities[index]
 
    cache.set(key, (index + 1) % len(possibilities))
 
    return render_to_response(template, *args, **kw)

Now we use our private generate_keyname function to get ourselves a cache hash and attempt to get the next index into possibilities. There’s an important trick here; since the entire cache is invalidated every time we deploy, if we don’t randomize the default index there would be a statistical gradient (bias) towards the first template, less so for the second, then third, and so on, which would make tracking A/B conversion rates a fair amount more difficult. (We would have to track views for each template as the denominator instead of assuming equal distribution!)

Finally we set the cache value, wrapped to the number of available possibilities, and return the result of the original render_to_response function. Fin.

Of course, this doesn’t cover actually tracking the effect of the template variation. You will still need to add appropriate JS tracking events or modify links to include a tracking value, but that is left as an exercise for the reader.

Updated to add: The above function assumes even distribution of the available template choices, but often if you are A/B testing page templates you want each user to consistently get the same template. To do this, replace the code at the end of the function, after the length-of-one early exit with:

    index = unpack('L', inet_aton(ip))[0] % len(possibilities)
    template = possibilities[index]
 
    return render_to_response(template, *args, **kw)

No more need to involve a cache as we are literally hashing a user-supplied value. (The ip variable would come from the REMOTE_ADDR WSGI environment variable of the active request.) You’ll also need to import unpack from the struct module and inet_aton from the socket module. You can also hash pretty much any value (such as a combination of REMOTE_ADDR and HTTP_USER_AGENT) using MD5:

int(md5('hello').hexdigest(), 16). % 7 == 4

Enjoy!