The pure dependency based is the reason why make is much better tool than say ant, nant, etc…

It’s great when properly written makefile completes in

I’ve had the same problem - building a tool to deploy developers.org.ua site (uses Pylons, Django, Wordpress, Openads and lots of other smaller libs).

Thus your reflections with fassember/zc.buildout were helpful to “diff against”.

I use Tempita to write deployment-specific files, such as paste .ini or apache vhost files. Configuration data stored in YAML format; there is a common.yaml which holds shared config info plus a .yaml file per each deployment, which contain deployment-specific options like database creds.

Each build/deployment is “self-contained”, that is, all files are put under a specific dir. The system env is configured (with symlinks or direct configuration) to use corresponding bits from the deployment. E.g. crond looks for its script under DROOT/environ/cron.* dirs.

My main “issue” with it - since it’s built by me (in-house) it means I have to maintain it. ;) Therefore I’d like to migrate/use some existing tool if/when it is available. Some capistrano for python or whatever it would be.

Thanks for writing up all of your thoughts about build tools (and for mentioning Paver). I should note that Paver uses a modified path.py that supports “dry-run”. It could easily be extended further to support prompting, which is something I just hadn’t thought of.

I’ve been considering zc.buildout integration as my next step for Paver, but I’m going to think about minitage and the process you’ve created around virtualenv, because I have noticed some of the same annoyances with zc.buildout that you have. One thing that made buildout integration appealing to me is that there are so many recipes that people have written for a variety of things. However, I think that many of these recipes exist simply because that’s the only way to write code for a buildout, which is buildout’s most annoying trait, imho. (I believe someone has written a recipe that lets you put some code in your buildout config file, but eww…)

By the way, so far I have not at all regretted my decision for Paver’s configuration and task definition to be entirely in Python. For Python programmers, there’s very little that is unclear about what will happen. (The trickiest things at present are those areas around distutils integration.)

Paver supports lazy evaluation of configuration items via callables (lambdas, generally). This has worked fine for my needs thus far.

On the whole, I’m quite happy to see all of this activity around build tools, because it has been a source of annoyance (obviously for many).

Kevin

The zc.buildout rebuilding’s habit is one of the main things that make me made minitage. I’m happy with zc.buildout but rebuilding everything is simply not acceptable. So i just do it in an atomic way now. It builds one thing, at one place, that’s all. So we can afford to rebuild something entirely as just that will be rebuilt.

Although minitage is working at the moment, it can be rough to be used by a classical user.

Why?

• Because the documentation is not really all up to date to the new version.

• Because it is made by a geek :p

• Because this is inspired by complicated tools (Gentoo’s portage, or other ports systems.)

What can that brings to you ?

• Minitage lives on top of virtualenv, virtualenv will set its prefix.

• More than just installing python libraries, it installs also system software pieces like postgresql or even python. Its final goal is to deploy a project from start to end in a re-usable manner.

• Free from zc.recipe.egg. I didn’t like too the approach of that recipe. Offline mode / local indexes were very problematic with it. So i implemented a recipe for minitage that borrowed code from zc.recipe.egg at the difference that it ll search in the cache before looking online for dependencies. That means if you have putted the egg inside the cache prior from this buildout execution, there will be no Online access at all, no index check.

• SVN (and others scms, ftp or http urls) checkouts can be up to you too with that recipe. See this buildout template: http://tinyurl.com/5c93lq You can have it with paster and minitage.paste (minitage.geodjango template) It uses custom GeoDjango egg that is not on pypi, and svn checkout to get the sources to build :)

• An enhanced CMMI build recipe wich integrates well with minitage for building “compiled” dependencies and interlink with other stuff installed into minitage.

More than an abstraction, minitage is in spirit a meta package manager. It does nothing by itself and relies on ‘install methods’. At this time, only buildout is available but it is as simply as to make recipes to implement install methods. Its only goal is to maintain dependencies to schedule the build process order and the install layout : http://www.minitage.org/doc/rst/spec.html#the-installation-layout.

For the log verbosity, minitage is pretty loud :p .

NA: The documentation moved to http://www.minitage.org/doc/rst/, you seemed to use a pretty old link.

The metaphor for the new build system was much simpler: do a list of things, top to bottom. There’s no effort into detecting changes in the build, or changes in the settings, or anything else.

While this approach is obviously working for them, I can’t help but think it could cause problems in the future. You’ve essentially linearized your dependency graph, and while the current linearization is presumably correct, any change to the way the system works could render it incorrect, silently or with a big bang. Explicitly encoding your dependencies can be hard, but in my experience it results in a much less fragile, much more easily understandable system. I’ve even used graphviz to visualize dependency graphs, which can make clumsy architecture easier to spot.

EDIT: partly this is because, as Ian noted, most of our stuff isn’t compiled. For instance, a build from scratch of our most complex product (opencore) takes less than ten minutes on old hardware, the bulk of which is download time. Rebuilds take less than a minute; largely this is because we delegate to tools that already handle dependency checking (easy-install, make, etc).

For the few things we install from big tarballs (like Zope), we just manually added a check for an up-to-date tarball. This takes a few lines of code, whereas in Make it’d “just work”. But there are relatively few cases where we care, so this hasn’t been much burden.

This build tool is for deploying our stack, for production, testing, and development. It doesn’t do anything special for remote deployments (unlike Capistrano). It isn’t focused on compiling because nearly everything we have runs from source (as a result we’d save no time figuring out file dependencies).