Neil’s Notes and Thoughts – Building a Django Session Backend using Riak

Building a Django Session Backend using Riak

July 03, 2011

Introduction

At MochiMedia we use 'Django in non-standard ways'. One of the non-standard things we do is use a Django custom session backend for MochiGames. Not too long ago, MochiGames’ session lookup became increasingly slow because of the volume of sessions we were storing. This is mostly because of the usage behavior of visitors and our non-default session expiration time (3 weeks vs. Django’s default of 2 weeks.) So, we wrote a custom session backend backed by the distributed datastore Riak. We decided on using Riak since it can effortlessly scale horizontally and allows our Erlang-based services to access Django sessions. This post discusses how to create a custom Django session backend. But, before we do that let’s understand how sessions work together with Django’s auth middleware.

How Djanog Sessions and Auth Middlewares Work

When a user logs in using django.contrib.auth.login, Django stashes the user’s id and the name of the authenticating backend in the user’s session. It also sets the user attribute on the request object for the current request.

When a user logs out using django.contrib.auth.logout, their session is flushed (deleted in most backends) and request.user is set to an AnonymousUser instance.

On all authenticated requests, the auth and session middlewares handle instantiating the user session and user objects for use in the request lifetime. The session middleware (django.contrib.session.middleware.SessionMiddleWare) is the first middleware executed on requests and looks for the session cookie (cookie name sessionid, by default) and creates the request session object using the configured session backend (defined by settings.SESSION_ENGINE). Not surprisingly, session data is not fetched at this point. Django’s session is lazy and the session data is only fetched when a session item is accessed.

On requests, the auth middleware (django.contrib.auth.middleware.AuthenticationMiddleware) sets the request.user class to LazyUser, this allows the request.user object to be fetched only on attribute access. When the user needs to be fetched, Django grabs the stashed user id and authenticating backend from the session to create the user object.

On response, the session middleware updates the session cookie if the session object was modified or the settings.SAVE_ON_EVERY_REQUEST is set.

The session and auth middlewares work together but are logically decoupled, this makes it very simple to create and plug-in your own session backend.

Creating new Session Backends

To create a custom session backend, create a class named SessionStore that inherits from django.contrib.sessions.backends.base.SessionBase and implements the following methods:

exists: Returns True if the session_key already exists in the session store.
create: Creates a new empty session store instance, calling create() should guarantee the session store instance exists
save: Saves session data to the session backend. Note, this does not serialize the data; serialization is done automatically by SessionBase.
delete: Deletes sessions in the session store.
load: Fetches the session and returns a dictionary.

For MochiGames, we use a slightly modified version of the standard Riak Python client library. Here is how our custom session backend calls map to Riak’s API calls.

Session method	Riak API
load	RiakBucket.get()
exists	RiaKBucket.get() (return `False` if `get()` returns `None`, `True` otherwise)
save	RiakObject.store()
delete	RiakObject.delete()
create	RiakObject.store() indirectly (actually calls `session.save(must_create=True)`

To enable your session backend, set settings.SESSION_ENGINE to the Python module containing your session class. That’s all there is to creating and setting up a custom session backing using Riak.

When running our standalone unittests, we do not want to require a Riak instance, so we just set our test configuration to use the Cache session backend and the localmem cache backend.

Summary

Writing a custom Session backend is simple as implementing five methods of a SessionBase subclass. The Django sessions API to allow deep customization; making it easy to configure a customize session backend.

There are a couple of unfortunate things when using the session middleware with a couple of other middlewares:

When using with the auth middleware, the auth middleware will force the fetching of the user’s session to retrieve the stashed authenticated user's id and authenticating backend.
When using the locale middleware, there’s a similar stashing of the 'django_language' in the session.

Both of these middlewares nullify the usefulness of lazy sessions. So, even if your view code does not touch session data, the session is fetched on every request.

With the addition of cookie signing to Django core, these pieces of information can now be securely stashed in a cookie instead of a session, making sessions truly lazy and only fetched when you truly need them.

Tags: scale, database, python, django, middleware, riak, sessions, authentication