Writing a Python decorator that can be called as a function or a callable
17 Dec 2013
A Python decorator wraps a function with another function. Classing examples are a @cache decorator or a @log decorator, which call the wrapped function and either cache its results or log the fact that it was called, respectively. Decorators can be implemented as functions or as classes; they just need to be callable.
Here is the basic decorator pattern. This one does nothing but prints that it was called.
Here is an example of a @cache decorator implemented in this fashion (as a function). It uses Django's caching layer as the actual cache implementation.
This uses a very simplistic cache key generation scheme. It assumes that the args and kwargs that your wrapped function will be passed are all castable to strings. Django's default cache key generator looks like:
There are also a number of caveats. For example, Django will throw an exception if the cache key is over 250 characters. Writing your own key generation is out of the scope of this post.
You will also notice that I'm using a functools.wraps. This ensures that when callers introspect the function_to_wrap function, it shows its __name__ attribute as function_to_wrap and not cache. This is especially useful for not mucking up your logging and performance stacktraces (for example, New Relic stats).
Here is an example of the same decorator written as a class:
Both implementations have the same usage syntax. You just decorate the function definition that you want to wrap with the @ syntax.
Sometimes you want to pass parameters to your decorators. The trick here is to add another layer of indirection and create a function that takes parameters and returns your original decorator. As you can see, the naming also gets a little mind-bending here; as we struggle to propery name what should really be anonymous functions for the callable we're returning, and the function that defines the logic of our decorator.
Of course, you can also do the same thing in the class style. Again, the trick is that a decorator can be a callable, or return a callable.
Now, a whole in the design of decorators, in my opinion, is that while you're deciding to make your decorator a callable or return a callable, you may also be struggling with how to make it do both at once.
What if I don't want the seconds argument to be mandatory? With either the functional or class based implementations, you will end up using your decorator like so:
This is just ugly. It introduces a source of errors (leaving off the () will throw a somewhat mysterious exception:
With a little ingenuity, you can have your callable and return it, too. Here is a functional decorator that can be used as @cache(seconds=60), or just @cache.
First, you decide whether your decorator has been called as a callable or not. If not, you pull out your optional parameters (and default them if needed). Then you dynamically return either your decorator or a callable. Admittedly this is pretty ugly, but the resulting API is nice and clear. I've also failed repeatedly to produce a class based version of this. Submissions welcome!
I'm currently working at NerdWallet, a startup in San Francisco trying to bring clarity to all of life's financial decisions. We're hiring like crazy. Hit me up on Twitter, I would love to talk.