Local state is harmful

Last updated 2014-02-17

Don't lynch me just yet :)

Picture a traditional webapp. We have a bunch of stateless workers connected to a stateful, relational database. This is a setup with a number of excellent properties:

The database state is also pervasive, mutable and global.

Now let's zoom in and look at our imperative, object-oriented workers:

Functional programmers need not look so smug at this point. The Haskell/OCaml family struggles to redefine types at runtime or handle live data migrations (the declaration of a nominal type is a side-effect in a live language). Clojure does better on these points but still gets burned by nominal types (eg extend a deftype/defrecord and the reeval the definition) and more generally by treating the definition of new code as mutation of state (which has to be papered over by tools.namespace).

Why are these points important? We spend most of our time not writing code but reasoning about code, whether hunting for bugs, refactoring old code or trying to extend a module. We end up with questions like:

How do we answer these questions?

In the database we have a transaction log containing for each transaction: the queries involved, the commit time, the client name etc. We can write code that specifies the condition we are interested in via an sql query, locates the relevant transactions by running through the log and then recreates the state of the database at that point. This works even if the error happened elsewhere - just have the user ship you their transaction log.

In the worker, we have two familiar workhorses:

What these two have in common is that they are both achingly manual. There is no easy way to automate the process. There are no libraries full of debugging strategies that you can deploy. The questions we have are about time and causality but our current tools restrict us to looking at tiny slices of space (print statements) or time (debuggers) and offer no way to automate our actions.

I propose that if we were to manage state more like a database and less like a traditional imperative language then understanding and debugging programs would become easier.