Building a Django Session Backend using RiakJuly 03, 2011
At MochiMedia we use 'Django in non-standard ways'. One of the non-standard things we do is use a Django custom session backend for MochiGames. Not too long ago, MochiGames’ session lookup became increasingly slow because of the volume of sessions we were storing. This is mostly because of the usage behavior of visitors and our non-default session expiration time (3 weeks vs. Django’s default of 2 weeks.) So, we wrote a custom session backend backed by the distributed datastore Riak. We decided on using Riak since it can effortlessly scale horizontally and allows our Erlang-based services to access Django sessions. This post discusses how to create a custom Django session backend. But, before we do that let’s understand how sessions work together with Django’s auth middleware.
How Djanog Sessions and Auth Middlewares Work
When a user logs in using
django.contrib.auth.login, Django stashes the user’s id and the name of the authenticating backend in the user’s session. It also sets the
user attribute on the request object for the current request.
When a user logs out using
django.contrib.auth.logout, their session is flushed (deleted in most backends) and
request.user is set to an
On all authenticated requests, the auth and session middlewares handle instantiating the user session and user objects for use in the request lifetime. The session middleware (
django.contrib.session.middleware.SessionMiddleWare) is the first middleware executed on requests and looks for the session cookie (cookie name
sessionid, by default) and creates the request session object using the configured session backend (defined by
settings.SESSION_ENGINE). Not surprisingly, session data is not fetched at this point. Django’s session is lazy and the session data is only fetched when a session item is accessed.
On requests, the auth middleware (
django.contrib.auth.middleware.AuthenticationMiddleware) sets the
request.user class to
LazyUser, this allows the
request.user object to be fetched only on attribute access. When the user needs to be fetched, Django grabs the stashed user id and authenticating backend from the session to create the user object.
On response, the session middleware updates the session cookie if the session object was modified or the
settings.SAVE_ON_EVERY_REQUEST is set.
The session and auth middlewares work together but are logically decoupled, this makes it very simple to create and plug-in your own session backend.
Creating new Session Backends
To create a custom session backend, create a class named
SessionStore that inherits from
django.contrib.sessions.backends.base.SessionBase and implements the following methods:
session_keyalready exists in the session store.
- Creates a new empty session store instance, calling
create()should guarantee the session store instance exists
- Saves session data to the session backend. Note, this does not serialize the data; serialization is done automatically by SessionBase.
- Deletes sessions in the session store.
- Fetches the session and returns a dictionary.
For MochiGames, we use a slightly modified version of the standard Riak Python client library. Here is how our custom session backend calls map to Riak’s API calls.
|Session method||Riak API|
|create||RiakObject.store() indirectly (actually calls
To enable your session backend, set
settings.SESSION_ENGINE to the Python module containing your session class. That’s all there is to creating and setting up a custom session backing using Riak.
When running our standalone unittests, we do not want to require a Riak instance, so we just set our test configuration to use the Cache session backend and the
localmem cache backend.
Writing a custom Session backend is simple as implementing five methods of a
SessionBase subclass. The Django sessions API to allow deep customization; making it easy to configure a customize session backend.
There are a couple of unfortunate things when using the session middleware with a couple of other middlewares:
- When using with the auth middleware, the auth middleware will force the fetching of the user’s session to retrieve the stashed authenticated user's id and authenticating backend.
- When using the locale middleware, there’s a similar stashing of the 'django_language' in the session.
Both of these middlewares nullify the usefulness of lazy sessions. So, even if your view code does not touch session data, the session is fetched on every request.
With the addition of cookie signing to Django core, these pieces of information can now be securely stashed in a cookie instead of a session, making sessions truly lazy and only fetched when you truly need them.