Paste and Pray
It’s pretty common to Google a technical issue, and come up with a blog or a Stackoverflow article that promises to fix the issue by copying and pasting a simple one-liner into a terminal. At first it’s just a quick
chmod, or a simple
apt-get. It’s completely unsafe unless you know exactly what the command is doing, but it’s pretty damn effective. Then you graduate to doing few lines at a time without really thinking about it too much, like a bad drug habit. Repetition slowly convinces you that it’s not insane.
In the last couple of years, I’ve seen a few extreme versions of this same idea, but taken to the next level. Homebrew can be installed by running
ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)". Heroku Toolbelt for Linux is just a
wget -qO- https://toolbelt.heroku.com/install-ubuntu.sh | sh away. I call these Paste and Pray installers.
Sounds like fun, right? I thought so, so I came up with a version that installs a pretty vanilla Django development environment, from scratch. It doesn’t assume anything except cURL and Python, both of which come pre-installed on OSX and Ubuntu. It does a fully automated install of Homebrew (including its dependency XCode on OSX), as well as pip, virtualenv, your code from GitHub, Django and any other Python requirements you have. It also sets some environment variables and updates your hosts file.
Oh, did I mention that some of these steps require root access? That’s right! It’s more like Super Paste and Pray™.
Note: this code is not intended to be directly re-usable. Instead, I thought I would share what I learned while writing it.
You can view the code and documentation. The code itself can be run with
curl -fsS https://raw.github.com/chase-seibert/paste-and-pray/go/run.py |sudo -E python. You should be able to modify it pretty easily for any Django app.
Although I didn’t want any dependencies required for the installer itself, I did want to take this chance to play around with sh, a nice Python API for interfacing with the shell. Because I could not rely on pip to be installed, I just downloaded this dependency right in Python:
sh turned out to be a little trickier than I imagined. First off, many of my commands required root access. The easiest method was to start the script with
sudo, and wrap any steps that did not require root in a
sudo -E -u username prefix. This is exactly what sh.bake is for.
I found it useful to redirect
sys.stdout for trouble-shooting purposes. Similarly, if you need to take user input, you will need to redirect stdin with something like:
Initially, I had tried calling setuid to downgrade to a non-root user. But, I was not able to go back to root, meaning that you would need to do all the root steps in one chunk, then do all the non-root stuff. This was a show-stopper as the Homebrew install step must be run as a regular user, but subsequent
virtualenv tool installs required root.
Another wrinkle was getting the original username from inside sudo. This was simple, thought I did need to hunt around before I discovered that both OSX and Linux set a
SUDO_USER environment variable for just this purpose.
Installing Homebrew manually is straight-forward, but does require the XCode command line tools to be installed first. Normally, this is a headache as you have to create an Apple ID and hunt and peck trough their developer website for a binary installer.
It turns out there there are direct download links, they just are not publicised. Here is some python code to download and install the correct version.
As I said, the Homebrew install is easy. I did have to include a small fixup gist to get it to work on one of my test machines.
Working with Python made a lot of the script easier. Specifically, working with
sh made tasks like getting the list of items currently installed by brew pretty simple:
It also allows chaining, just like in a bash shell:
One area that was a little tricky was activating virtualenv from inside Python. However, there is actually a supported method for this.
For the shortest URL possible, you can set the default branch in GitHub to be something short, like “go”. Looking at the homebrew installer, I noticed that it’s possible to leave the file name off, too. GitHub’s raw file service apparently picks the first file by alphabetical order if you do this. However, I didn’t want to mess with trying to get my code to show up alphabetically before
.gitignore, so I skipped this optimization.
Random other thoughts on writing installers:
- Create a temp directory if you need to do anything like download files.
- Make the installer idempotent; you want to be able to retry cleanly.
- Checking whether a step has already been run is a good idea.
- Don’t try to hide the debug output; at least for a development audience.