Hacking Django runserver to run multiple Django instances
03 Nov 2013
Recently at work we've been on a "servicifying" kick, meaning we're slowly converting our monolithic Django app into separate services. To start, this just means breaking up the existing runtime into pieces. Instead of one logical web process, we now have different ones for the web app, admin, login, apis, etc.
For production, this doesn't change the deployment model all that much. We just have separate servers for various roles. For development, we still want to be able to easily run all the services at once on your local machine.
Enter runservices, an extension of the Django runserver command that just launches a bunch of processes in a screen session (a natural choices for us, as we're already using screen intensively). It turns out that screen has the ability to launch multiple windows on startup using a custom .screenrc file, passed on the command-line with -c my.screenrc. The format needed inside the .screenrc file is as follows:
When I first saw this, I thought we could use it to launch our various services. What I did was write a custom Django management command that dynamically writes a .screenrc file and executes it. You can run all the services, or specify just a few. Our services are launched by settings corresponding environment variables, which can be done on the command line itself.
The resulting .screenrc files ends up looking like (added comments):
The only tricky bit was getting virtualenv to activate properly inside the screen session. Because it uses the bash source command, I needed to have screen actually execute bash directly, and pass the source command.
The result is a single session of multiple windows that you can toggle through with the standard screen commans like C-a.
Note: I'm also running into intermittent issues with sqlite3 locking the database, due to many processes trying to access it. Running with --nothreading and reducing celery to one worker seems to have helped, but we may need to move to mysql for development.
I'm currently working at NerdWallet, a startup in San Francisco trying to bring clarity to all of life's financial decisions. We're hiring like crazy. Hit me up on Twitter, I would love to talk.