Comments on: Configuration management: push vs. pull http://blog.ianbicking.org/2010/03/10/configuration-management-push-vs-pull/ Tue, 07 Sep 2010 06:35:54 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Mark Burgess http://blog.ianbicking.org/2010/03/10/configuration-management-push-vs-pull/comment-page-1/#comment-156287 Mark Burgess Tue, 16 Mar 2010 10:55:04 +0000 http://blog.ianbicking.org/?p=180#comment-156287 There are certainly unscalable aspects of push and pull, but in fact they have different optimizations. Push is easier on the server, since it can more easily manage its resource caching, but it is highly unpredictable as you don't know whether the clients are available or not. Moreover, a server is usually equipped to deal with the load where as a client isn't. With push you are letting one system drive the availability of many, which makes adaptation and fault tolerance very difficult. Pull can automatically perform distributed load balancing and each machine can download stuff at its convenience. This especially applies to hosts that are down at the moment of push. Another problem with push is that the clients have no say in what they get from the "pusher". This is a potential security disaster waiting to happen. Clients are in a better position to decide when and what to download than a central pusher is to decide when and what they need from its single viewpoint. So adaptive behaviour makes a strong case for pull based (subscription based) services. As for the rest, Cfengine introduced pull and is of course far superior to Puppet both in capabilities and implementation ;-) There are certainly unscalable aspects of push and pull, but in fact they have different optimizations. Push is easier on the server, since it can more easily manage its resource caching, but it is highly unpredictable as you don’t know whether the clients are available or not. Moreover, a server is usually equipped to deal with the load where as a client isn’t. With push you are letting one system drive the availability of many, which makes adaptation and fault tolerance very difficult. Pull can automatically perform distributed load balancing and each machine can download stuff at its convenience. This especially applies to hosts that are down at the moment of push.

Another problem with push is that the clients have no say in what they get from the “pusher”. This is a potential security disaster waiting to happen. Clients are in a better position to decide when and what to download than a central pusher is to decide when and what they need from its single viewpoint. So adaptive behaviour makes a strong case for pull based (subscription based) services.

As for the rest, Cfengine introduced pull and is of course far superior to Puppet both in capabilities and implementation ;-)

]]> By: Ian Bicking http://blog.ianbicking.org/2010/03/10/configuration-management-push-vs-pull/comment-page-1/#comment-155831 Ian Bicking Fri, 12 Mar 2010 19:52:43 +0000 http://blog.ianbicking.org/?p=180#comment-155831 Your example makes more sense to me than most use cases I've heard. The way [Paul Smith mentions](#comment-155587) that EC2 instances can be started with a script to be run on boot seems kind of like a hybrid. It's not exactly push, as the commands are executed offline on restart. But it's not pull, in that everything is initiated when the server is requisitioned, and initiated by the requisitioner. I guess it's async push...? Your example makes more sense to me than most use cases I’ve heard. The way Paul Smith mentions that EC2 instances can be started with a script to be run on boot seems kind of like a hybrid. It’s not exactly push, as the commands are executed offline on restart. But it’s not pull, in that everything is initiated when the server is requisitioned, and initiated by the requisitioner. I guess it’s async push…?

]]> By: Chris AtLee http://blog.ianbicking.org/2010/03/10/configuration-management-push-vs-pull/comment-page-1/#comment-155829 Chris AtLee Fri, 12 Mar 2010 19:17:53 +0000 http://blog.ianbicking.org/?p=180#comment-155829 A pull-based system is useful for test machines that constantly reboot to ensure a clean test environment. Wouldn't it be easier to have the machine come up, pull its configuration, start testing than to have it come up, and wait for another server to configure it and start the tests? A pull-based system is useful for test machines that constantly reboot to ensure a clean test environment. Wouldn’t it be easier to have the machine come up, pull its configuration, start testing than to have it come up, and wait for another server to configure it and start the tests?

]]> By: Jorge Vargas http://blog.ianbicking.org/2010/03/10/configuration-management-push-vs-pull/comment-page-1/#comment-155773 Jorge Vargas Thu, 11 Mar 2010 23:30:08 +0000 http://blog.ianbicking.org/?p=180#comment-155773 Totally agree. This is the main reason I was first turned on to toppcloud/silverlining having push combined with "a setup that does one thing and does it well" is an excellent combination. Will it scale? we'll see that when we need it. For now we are more than happy for running each app entirely on one box. Totally agree. This is the main reason I was first turned on to toppcloud/silverlining having push combined with “a setup that does one thing and does it well” is an excellent combination. Will it scale? we’ll see that when we need it. For now we are more than happy for running each app entirely on one box.

]]> By: Jorge Vargas http://blog.ianbicking.org/2010/03/10/configuration-management-push-vs-pull/comment-page-1/#comment-155771 Jorge Vargas Thu, 11 Mar 2010 23:27:35 +0000 http://blog.ianbicking.org/?p=180#comment-155771 It's in the docs http://cloudsilverlining.org/design.html#fabric IMO Fabric's problem is that it expect you to run a lot of stuff on the server which makes it a messier approach. It’s in the docs http://cloudsilverlining.org/design.html#fabric

IMO Fabric’s problem is that it expect you to run a lot of stuff on the server which makes it a messier approach.

]]> By: Ian Bicking http://blog.ianbicking.org/2010/03/10/configuration-management-push-vs-pull/comment-page-1/#comment-155767 Ian Bicking Thu, 11 Mar 2010 22:27:04 +0000 http://blog.ianbicking.org/?p=180#comment-155767 http://cloudsilverlining.org/design.html#fabric http://cloudsilverlining.org/design.html#fabric

]]> By: Grig Gheorghiu http://blog.ianbicking.org/2010/03/10/configuration-management-push-vs-pull/comment-page-1/#comment-155765 Grig Gheorghiu Thu, 11 Mar 2010 22:02:47 +0000 http://blog.ianbicking.org/?p=180#comment-155765 Interesting Usenix paper from 1998. They swear by the pull mechanism too: http://www.infrastructures.org/papers/bootstrap/bootstrap.html Interesting Usenix paper from 1998. They swear by the pull mechanism too:

http://www.infrastructures.org/papers/bootstrap/bootstrap.html

]]> By: Simon Luijk http://blog.ianbicking.org/2010/03/10/configuration-management-push-vs-pull/comment-page-1/#comment-155756 Simon Luijk Thu, 11 Mar 2010 20:28:28 +0000 http://blog.ianbicking.org/?p=180#comment-155756 Sorry about being slightly off topic, but how do you view Silver lining's overlap with fabric? Sorry about being slightly off topic, but how do you view Silver lining’s overlap with fabric?

]]> By: Ian Bicking http://blog.ianbicking.org/2010/03/10/configuration-management-push-vs-pull/comment-page-1/#comment-155754 Ian Bicking Thu, 11 Mar 2010 19:32:52 +0000 http://blog.ianbicking.org/?p=180#comment-155754 I think I was pretty clear that Silver Lining is a very simple and limited approach to the problem of configuration management. It actually "handles" things like dependencies and starting services in the right order, because Ubuntu and Debian maintainers have been doing that work for ages now, and Silver Lining uses that work. If you are willing to use and trust a quality distribution like Ubuntu, then configuration management becomes a much more approachable project. If you want to be distribution- or OS-agnostic, or you disagree with the way the packagers arranged those packages, then you have to do a lot more work. Puppet does that work. I don't consider the tradeoff worth it, but I can understand why some people would feel differently. That's a [different argument](http://blog.ianbicking.org/2010/02/10/why-toppcloud-not-agnostic/) for a different time. Puppet *also* uses a pull model. It would be entirely possible to use a push model to the same effect. I think I was pretty clear that Silver Lining is a very simple and limited approach to the problem of configuration management. It actually “handles” things like dependencies and starting services in the right order, because Ubuntu and Debian maintainers have been doing that work for ages now, and Silver Lining uses that work. If you are willing to use and trust a quality distribution like Ubuntu, then configuration management becomes a much more approachable project. If you want to be distribution- or OS-agnostic, or you disagree with the way the packagers arranged those packages, then you have to do a lot more work. Puppet does that work. I don’t consider the tradeoff worth it, but I can understand why some people would feel differently. That’s a different argument for a different time.

Puppet also uses a pull model. It would be entirely possible to use a push model to the same effect.

]]> By: Grig Gheorghiu http://blog.ianbicking.org/2010/03/10/configuration-management-push-vs-pull/comment-page-1/#comment-155741 Grig Gheorghiu Thu, 11 Mar 2010 15:09:56 +0000 http://blog.ianbicking.org/?p=180#comment-155741 Also, speaking of large scale file synchronization problems, here's how twitter does it: http://github.com/lg/murder + Capistrano. Interesting use of BitTorrent. Also, speaking of large scale file synchronization problems, here’s how twitter does it: http://github.com/lg/murder + Capistrano. Interesting use of BitTorrent.

]]>