I was looking at Phillip's CLOS-style generic
functions recently, and it got me thinking about metaprogramming
in Python. If you haven't looked at those generic functions, it's a
very neat idea and implementation, and can be expressed quite
elegantly in Python (at least after adding 2.4's @
decorator syntax). (This
post is another recent item that got me thinking about
metaprogramming.)
Python doesn't have much of anything like macros (which CLOS generic methods are implemented with, I believe), but it doesn't seem like a big impediment. And there's a lot of metaprogramming that is done in Python, without a macro in sight. By metaprogramming I mean programming structures that operate on other programming structures. Every decorator is a case of metaprogramming; the decorator is a function that takes a function as an argument, and returns a function (or function-like) object.
There's a lot of ways you can do metaprogramming in Python. Decorators are one way, and metaclasses are another. But it can be easier than that; you can modify classes on the fly, you can you can do all sorts of things with higher-order functions (i.e., functions that take functions as arguments), and even do funny sorts of things with import hooks, the new module, etc.
What makes this possible is the fact Python is a scripting language.
I know some people hate that term; but I think it means something very
specific, not merely derogatory. A Python module is a script;
when Python loads a module, it reads the file, evaluates the first
line, the second line, etc., until it's done. Functions aren't
declared; rather def creates a function on the spot; you
can redefine the function later, or define it inside another scope, or
delete it, or whatever. This causes problems (like the problem of
changing code in long-running processes, or circular references); but
it also has some advantages.
When we use a module, we can forget that we aren't using the source we
see in its .py file. We're using the objects that are
created by that source. When loading a module, we're running a
bunch of statements; some statements are rather benign with few side
effects (mostly def); others are somewhere in between
(like class); and others are executed immediately, and we
only have access to their effects. Decorators and metaclasses are two
instances; but you can put any statements in a module, and any of them
can modify what's in the module. You can even modify modules after
they are loaded (a monkey
patch).
There are some problems. You can't look inside code programmitically (or at least it isn't easy), and lazy evaluation is fairly limited. The generics package is forced to use strings for its expressions, then compile and execute them in a closed environment. SQLObject uses SQLBuilder which uses Python's magic methods, but is limited; Numeric uses something similar. Still, there's a huge amount you can do, and all without any macros, all simply due to the highly imperative nature of Python source.