<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Ian Bicking: a blog &#187; Web</title>
	<atom:link href="http://blog.ianbicking.org/category/web/feed/" rel="self" type="application/rss+xml" />
	<link>http://blog.ianbicking.org</link>
	<description></description>
	<lastBuildDate>Sat, 31 Dec 2011 00:12:49 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.2.1</generator>
		<item>
		<title>A Python Web Application Package and Format (we should make one)</title>
		<link>http://blog.ianbicking.org/2011/03/31/python-webapp-package/</link>
		<comments>http://blog.ianbicking.org/2011/03/31/python-webapp-package/#comments</comments>
		<pubDate>Thu, 31 Mar 2011 16:01:54 +0000</pubDate>
		<dc:creator>Ian</dc:creator>
				<category><![CDATA[Packaging]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Web]]></category>

		<guid isPermaLink="false">http://blog.ianbicking.org/?p=250</guid>
		<description><![CDATA[At PyCon there was an open space about deployment, and the idea of drop-in applications (Java-WAR-style). I generally get pessimistic about 80% solutions, and dropping in a WAR file feels like an 80% solution to me. I&#8217;ve used the Hudson/Jenkins installer (which I think is specifically a project that got WARs on people&#8217;s minds), and [...]]]></description>
			<content:encoded><![CDATA[
<div class="document">


<p>At PyCon there was an open space about deployment, and the idea of drop-in applications (Java-WAR-style).</p>
<p>I generally get pessimistic about 80% solutions, and dropping in a WAR file feels like an 80% solution to me. I&#8217;ve used the Hudson/Jenkins installer (which I think is <em>specifically</em> a project that got WARs on people&#8217;s minds), and in a lot of ways that installer is nice, but it&#8217;s also kind of wonky, it makes configuration unclear, it&#8217;s not always clear when it installs or configures itself through the web, and when you have to do this at the system level, nor is it clear where it puts files and data, etc. So a great initial experience doesn&#8217;t feel like a great ongoing experience to me &#8212; and <em>it doesn&#8217;t have to be that way</em>. If those were <em>necessary</em> compromises, sure, but they aren&#8217;t. And because we <em>don&#8217;t have</em> WAR files, if we&#8217;re proposing to make something new, then we have every opportunity to make things better.</p>
<p>So the question then is what we&#8217;re trying to make. To me: we want applications that are easy to install, that are self-describing, self-configuring (or at least guide you through configuration), reliable with respect to their environment (not dependent on system tweaking), upgradable, and respectful of persistence (the data that outlives the application install). A lot of this can be done by the &quot;container&quot; (to use Java parlance; or &quot;environment&quot;) &#8212; if you just have the app packaged in a nice way, the container (server environment, hosting service, etc) can handle all the system-specific things to make the application actually work.</p>
<p>At which point I am of course reminded of my <a class="reference external" href="http://cloudsilverlining.org">Silver Lining</a> project, which defines something very much like this. Silver Lining isn&#8217;t <em>just</em> an application format, and things aren&#8217;t fully extracted along these lines, but it&#8217;s pretty close and it addresses a lot of important issues in the lifecycle of an application. To be clear: Silver Lining is an application packaging format, a server configuration library, a cloud server management tool, a persistence management tool, and a tool to manage the application with respect to all these services over time. It is a bunch of things, maybe too many things, so it is not unreasonable to pick out a smaller subset to focus on. Maybe an easy place to start (and good for Silver Lining itself) would be to separate at least the application format (and tools to manage applications in that state, e.g., installing new libraries) from the tools that make use of such applications (deploy, etc).</p>
<p>Some opinions I have on this format, exemplified in Silver Lining:</p>
<ul class="simple">
<li>It&#8217;s not zipped or a single file, unlike WARs. Uploading zip files is not a great API. Geez. I know there&#8217;s this desire to &quot;just drop in a file&quot;; but there&#8217;s no getting around the fact that &quot;dropping a file&quot; becomes a <em>deployment protocol</em> <strong>and</strong> <em>it&#8217;s an incredibly impoverished protocol</em>. The format is also not subtly git-based (ala Heroku) because <tt class="docutils literal"><span class="pre">git</span> <span class="pre">push</span></tt> is not a good deployment protocol.</li>
<li>But of course there isn&#8217;t really any deployment protocol inferred by a format anyway, so maybe I&#8217;m getting ahead of myself ;) I&#8217;m saying a tool that deploys should take as an argument a directory, not a single file. (If the tool then zips it up and uploads it, fine!)</li>
<li>Configuration &quot;comes from the outside&quot;. That is, an application requests services, and the <em>container</em> tells the application where those services are. For Silver Lining I&#8217;ve used environmental variables. I think this one point is really important &#8212; the container <em>tells</em> the application. As a counter-example, an application that comes with a Puppet deployment recipe is essentially <em>telling</em> the server how to arrange itself to suit the application. This will never be reliable or simple!</li>
<li>The application indicates what &quot;services&quot; it wants; for instance, it may want to have access to a MySQL database. The container then provides this to the application. In practice this means installing the actual packages, but also creating a database and setting up permissions appropriately. The alternative is never having <em>any</em> dependencies, meaning you have to use SQLite databases or ad hoc structures, etc. But in fact installing databases really isn&#8217;t that hard these days.</li>
<li><em>All</em> persistence has to use a service of some kind. If you want to be able to write to files, you need to use a file service. This means the container is fully aware of everything the application is leaving behind. All the various paths an application should use are given in different environmental variables (many of which don&#8217;t need to be invented anew, e.g., <tt class="docutils literal"><span class="pre">$TMPDIR</span></tt>).</li>
<li>It uses vendor libraries exclusively for Python libraries. That means the application bundles all the libraries it requires. Nothing ever gets installed at deploy-time. This is in contrast to using a <tt class="docutils literal"><span class="pre">requirements.txt</span></tt> list of packages at deployment time. If you want to use those tools for development that&#8217;s fine, just not for deployment.</li>
<li>There <em>is</em> also a way to indicate other libraries you might require; e.g., you might <tt class="docutils literal"><span class="pre">lxml</span></tt>, or even something that isn&#8217;t quite a library, like <tt class="docutils literal"><span class="pre">git</span></tt> (if you are making a github clone). You can&#8217;t do those as vendor libraries (they include non-portable binaries). Currently in Silver Lining the application description can contain a list of Ubuntu package names to install. Of course that would have to be abstracted some.</li>
<li>You can ask for scripts or a request to be invoked for an application after an installation or deployment. It&#8217;s lame to try to test if is-this-app-installed on <em>every</em> request, which is the frequent alternative. Also, it gives the application the chance to signal that the installation failed.</li>
<li>It has a very simple (possibly/probably too simple) sense of configuration. You don&#8217;t have to use this if you make your app self-configuring (i.e., build in a web-accessible settings screen), but in practice it felt like some simple sense of configuration would be helpful.</li>
</ul>
<p>Things that could be improved:</p>
<ul class="simple">
<li>There are some places where you might be encouraged to use routines from the <tt class="docutils literal"><span class="pre">silversupport</span></tt> package. There are very few! But maybe an alternative could be provided for these cases.</li>
<li>A little convention-over-configuration is probably suitable for the bundled libraries; silver includes tools to manage things, but it gets a little twisty. When creating a new project I find myself creating several <tt class="docutils literal"><span class="pre">.pth</span></tt> files, special customizing modules, etc. Managing vendor libraries is also not obvious.</li>
<li><a class="reference external" href="http://cloudsilverlining.org/services.html">Services</a> are IMHO quite important and useful, but also need to be carefully specified.</li>
<li>There&#8217;s a bunch of runtime expectations that aren&#8217;t part of the format, but in practice would be part of how the application is written. For instance, I make sure each app has its own temporary directory, and that it is cleared on update. If you keep session files in that location, and you expect the environment to clean up old sessions &#8212; well, either all environments should do that, or none should.</li>
<li>The process model is not entirely clear. I tried to simply define one process model (unthreaded, multiple processes), but I&#8217;m not sure that&#8217;s suitable &#8212; most notably, multiple processes have a significant memory impact compared to threads. An application should at least be able to indicate what process models it accepts and prefers.</li>
<li>Static files are <em>all</em> convention over configuration &#8212; you put static files under <tt class="docutils literal"><span class="pre">static/</span></tt> and then they are available. So <tt class="docutils literal"><span class="pre">static/style.css</span></tt> would be at <tt class="docutils literal"><span class="pre">/style.css</span></tt>. I think this is generally <em>good</em>, but putting all static files under one URL path (e.g., <tt class="docutils literal"><span class="pre">/media/</span></tt>) can be good for other reasons as well. Maybe there should be conventions for both.</li>
<li>Cron jobs are important. Though maybe they could just be yet another kind of service? Many extra features could be new services.</li>
<li>Logging is also important; Silver Lining attempts to handle that somewhat, but it could be specified much better.</li>
<li>Silver Lining also supports PHP, which seemed to cause a bit of stress. But just ignore that. It&#8217;s really easy to ignore.</li>
</ul>
<p>There is a <a class="reference external" href="http://cloudsilverlining.org/appconfig.html">description of the configuration file for apps</a>. The <a class="reference external" href="http://cloudsilverlining.org/envvariables.html">environmental variables</a> are also notably part of the application&#8217;s expectations. The file layout is explained (together with a bunch of Silver Lining-specific concepts) in <a class="reference external" href="http://cloudsilverlining.org/devpatterns.html">Development Patterns</a>. Besides all that there is admittedly some other stuff that is only really specified in code; but in Silver Lining&#8217;s defense, specified in code is better than unspecified ;) App Engine provides another example of an application format, and would be worth using as a point of discussion or contrast (I did that myself when writing Silver Lining).</p>
<p>Discussing WSGI stuff with Ben Bangert at PyCon he noted that he didn&#8217;t really feel like the WSGI pieces needed that much more work, or at least that&#8217;s not where the interesting work was &#8212; the interesting work is in the tooling. An application format could provide a great basis for building this tooling. And I honestly think that the tooling has been held back more by divergent patterns of development than by the difficulty of writing the tools themselves; and a good, general application format could fix that.</p>
</div>
]]></content:encoded>
			<wfw:commentRss>http://blog.ianbicking.org/2011/03/31/python-webapp-package/feed/</wfw:commentRss>
		<slash:comments>18</slash:comments>
		</item>
		<item>
		<title>Javascript on the server AND the client is not a big deal</title>
		<link>http://blog.ianbicking.org/2011/03/30/js-on-server-and-client-is-not-a-big-deal/</link>
		<comments>http://blog.ianbicking.org/2011/03/30/js-on-server-and-client-is-not-a-big-deal/#comments</comments>
		<pubDate>Wed, 30 Mar 2011 22:09:24 +0000</pubDate>
		<dc:creator>Ian</dc:creator>
				<category><![CDATA[Javascript]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Web]]></category>

		<guid isPermaLink="false">http://blog.ianbicking.org/?p=245</guid>
		<description><![CDATA[All the cool kids love Node.js. I&#8217;ve used it a little, and it&#8217;s fine; I was able to do what I wanted to do, and it wasn&#8217;t particularly painful. It&#8217;s fun to use something new, and it&#8217;s relatively straight-forward to get started so it&#8217;s an emotionally satisfying experience. There are several reasons you might want [...]]]></description>
			<content:encoded><![CDATA[<div class="document">
<p>All the cool kids love <a class="reference external" href="http://nodejs.org/">Node.js</a>.  I&#8217;ve used it a little, and it&#8217;s fine; I was able to do what I wanted to do, and it wasn&#8217;t particularly painful.  It&#8217;s fun to use something new, and it&#8217;s relatively straight-forward to get started so it&#8217;s an <em>emotionally satisfying</em> experience.</p>
<p>There are several reasons you might want to use Node.js, and I&#8217;ll ignore many of them, but I want to talk about one in particular:</p>
<blockquote><p>
Javascript on the client and the server!</p></blockquote>
<p>Is this such a great feature?  I think not&#8230;</p>
<div class="section" id="you-only-need-to-know-one-language">
<h2>You only need to know one language!</h2>
<p>Sure.  Yay ignorance!  But really, this is <em>fine</em> but unlikely to be relevant to any current potential audience for Node.js.  If you are shooting for an very-easy-to-learn client-server programming system, Node.js isn&#8217;t it.  Maybe <a class="reference external" href="http://couchapp.org/">Couch</a> or something similar has that potential?  But I digress.</p>
<p>It&#8217;s not easy to have expertise at multiple languages.  But it&#8217;s not <em>that hard</em>.  It&#8217;s considerably harder to have expertise at <em>multiple platforms</em>.  Node.js gives you one language across client and server, but <em>not</em> one platform.  Node.js programming doesn&#8217;t <em>feel</em> like the browser environment.  They do adopt many conventions when it&#8217;s reasonable, but even then it&#8217;s not always the case &#8212; in particular because many browser APIs are the awkward product of C++ programmers exposing things to Javascript, and you don&#8217;t want to reproduce those same APIs if you don&#8217;t have to (and Node.js doesn&#8217;t have to!) &#8212; an example is the event pattern in Node, which is similar to a browser but less obtuse.</p>
</div>
<div class="section" id="you-get-to-share-libraries">
<h2>You get to share libraries!</h2>
<p>First: the same set of libraries is probably not applicable.  If you can do it on the client then you probably don&#8217;t <em>have</em> to do it on the server, and vice versa.</p>
<p>But sometimes the same libraries are useful.  Can you really share them?  Browser libraries are often hard to use elsewhere because they rely on browser APIs.  These APIs are frequently <em>impossible to implement in Javascript</em>.</p>
<p>Actually they are possible to implement in Javascript using <a class="reference external" href="https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/Proxy">Proxies</a> (or maybe some other new and not-yet-standard Javascript features). But not in Node.js, which uses V8, and V8 is a pretty conservative implementation of the Javascript language. (<strong>Update</strong>: <a class="reference external" href="http://blog.ianbicking.org/2011/03/30/js-on-server-and-client-is-not-a-big-deal/comment-page-1/#comment-194005">it is noted</a> that you can <a class="reference external" href="https://github.com/isaacs/node-proxy/tree/master/src">implement proxies</a> &#8212; in this case a C++ extension to Node)</p>
<p>Besides these unimplementable APIs, it is also just a different environment.  There is the trivial: the <tt class="docutils literal"><span class="pre">window</span></tt> object in the browser has a Node.js equivalent, but it&#8217;s not named <tt class="docutils literal"><span class="pre">window</span></tt>. Performance is different &#8212; Node has long-running processes, the browser <em>might</em>.  Node can have blocking calls, which are useful even if you can&#8217;t use them at runtime (e.g., <tt class="docutils literal"><span class="pre">require()</span></tt>); but you can&#8217;t really have any of these at any time on the browser.  And then of course all the system calls, <em>none</em> of which you can use in the browser.</p>
<p>All these may simply be surmountable challenges, through modularity, mocking, abstractions, and so on&#8230; but ultimately I think the motivation is lacking: the domain of changing a live-rendered DOM isn&#8217;t the same as producing bytes to put onto a socket.</p>
</div>
<div class="section" id="you-can-work-fluidly-across-client-and-server">
<h2>You can work fluidly across client and server!</h2>
<p>If anything I think this is <em>dangerous</em> rather than <em>useful</em>.  The client and the server are different places, with different expectations.  Any vagueness about that boundary is <em>wrong</em>.</p>
<p>It&#8217;s wrong from a security perspective, as the security assumptions are nearly opposite on the two platforms.  The client trusts itself, and the server trusts itself, and both should hold the other in suspicion (though the client can be more trusting because the <em>browser</em> doesn&#8217;t trust the client code).</p>
<p>But it&#8217;s also the wrong way to treat HTTP.  HTTP is pretty simple until you try to make it simpler.  Efforts to make it simpler mostly make it more complicated.  HTTP lets you send serialized data back and forth to a server, with a bunch of metadata and other do-dads.  And that&#8217;s all neat, but you should always be thinking about <em>sending</em> information.  And never <em>sharing</em> information.  It&#8217;s not a fluid boundary, and code that touches HTTP needs to be explicit about it and not pretend it is equivalent to any other non-network operation.</p>
<p>Certainly you don&#8217;t <em>need</em> two implementation languages to keep your mind clear.  But it doesn&#8217;t hurt.</p>
</div>
<div class="section" id="you-can-do-validation-the-same-way-on-the-client-and-server">
<h2>You can do validation the same way on the client and server!</h2>
<p>One of the things people frequently bring up is that you can validate data on the client and server using the same code.  And of course, what web developer hasn&#8217;t been a little frustrated that they have to implement validation twice?</p>
<p>Validation on the client is primarily a <em>user experience</em> concern, where you focus on bringing attention to problems with a form, and helping the user resolve those problems.  You may be able to avoid errors entirely with an input method that avoids the problem (e.g., if a you have a slider for a numeric input, you don&#8217;t have to worry about the user inputing a non-numeric value).</p>
<p>Once the form is submitted, if you&#8217;ve done thorough client-side validation you can also avoid <em>friendly</em> server-side validation.  Of course all your client-side validation could be avoided through a malicious client, but you don&#8217;t need to give a friendly error message in that case, you can simply bail out with a simple 400 Bad Request error.</p>
<p>At that point there&#8217;s not much in common between these two kinds of validation &#8212; the client is all user experience, and the server is all data integrity.</p>
</div>
<div class="section" id="you-can-do-server-side-javascript-as-a-fallback-for-the-client">
<h2>You can do server-side Javascript as a fallback for the client!</h2>
<p>Writing for clients without Javascript is becoming increasingly less relevant, and if we aren&#8217;t <em>there</em> yet, then we&#8217;ll certainly <em>get there</em> soon.  It&#8217;s only a matter of time, the writing is on the wall. Depending on the project you might have to put in workarounds, but we should keep those concerns out of architecture decisions.  Maintaining crazy hacks is <em>not</em> worth it. There&#8217;s so many terrible hacks that have turned into frameworks, and frameworks that have justified themselves because of the problems they solved that no longer matter&#8230; Node.js deserves better than to be one of those.</p>
</div>
<div class="section" id="in-conclusion-or-whatever">
<h2>In Conclusion Or Whatever</h2>
<p>I&#8217;m not saying Node.js is <em>bad</em>.  There are other arguments for it, and you don&#8217;t need to make <em>any</em> argument for it if you just feel like using it.  It&#8217;s fun to do something new.  And I&#8217;m as optimistic about Javascript as anyone.  But this one argument, I do not think it is very good.</p>
</div>
</div>
]]></content:encoded>
			<wfw:commentRss>http://blog.ianbicking.org/2011/03/30/js-on-server-and-client-is-not-a-big-deal/feed/</wfw:commentRss>
		<slash:comments>29</slash:comments>
		</item>
		<item>
		<title>Doctest.js &amp; Callbacks</title>
		<link>http://blog.ianbicking.org/2010/09/12/doctest-js-callbacks/</link>
		<comments>http://blog.ianbicking.org/2010/09/12/doctest-js-callbacks/#comments</comments>
		<pubDate>Sun, 12 Sep 2010 23:08:40 +0000</pubDate>
		<dc:creator>Ian Bicking</dc:creator>
				<category><![CDATA[Javascript]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Testing]]></category>
		<category><![CDATA[Web]]></category>

		<guid isPermaLink="false">http://blog.ianbicking.org/?p=239</guid>
		<description><![CDATA[Many years ago I wrote a fairly straight-forward port of Python&#8217;s doctest to Javascript. I thought it was cool, but I didn&#8217;t really talk about it that much. Especially because I knew it had one fatal flaw: it was very unfriendly towards programming with callbacks, and Javascript uses a lot of callbacks. On a recent [...]]]></description>
			<content:encoded><![CDATA[<div class="document">
<p>Many years ago I wrote a fairly straight-forward port of Python&#8217;s <a class="reference external" href="http://docs.python.org/library/doctest.html">doctest</a> to Javascript.  I thought it was cool, but I didn&#8217;t really talk about it that much.  Especially because I knew it had one fatal flaw: it was very unfriendly towards programming with callbacks, and Javascript uses a lot of callbacks.</p>
<p>On a recent flight I decided to look at it again, and realized fixing that one flaw wasn&#8217;t actually a big deal.  So now doctest.js really works.  And I think it works well: <a class="reference external" href="http://ianb.github.com/doctestjs">doctest.js (reborn)</a>.</p>
<p>I have yet to really use doctest.js on more than a couple real cases, and as I do (or you do?) I expect to tweak it more to make it flow well. But having tried a couple of examples I am particularly liking how it can be used with callbacks.</p>
<p>Testing with callbacks is generally a tricky thing.  You want to make assertions, but they happen entirely separately from the test runner&#8217;s own loop, and your callbacks may not run at all if there&#8217;s a failure.</p>
<p>I came upon some tests recently that used <a class="reference external" href="http://pivotal.github.com/jasmine/">Jasmine</a>, a <a class="reference external" href="http://en.wikipedia.org/wiki/Behavior_Driven_Development">BDD-style</a> test framework.  I&#8217;m <a class="reference external" href="http://blog.ianbicking.org/behavior-driven-programming.html">not a big fan of BDD</a> but I&#8217;m fairly new to serious Javascript development so I&#8217;m trying to withhold judgement.  The flow of the tests is a bit peculiar until you realize that it&#8217;s for async reasons.  I&#8217;ll try to show something that roughly approximates a real test of an XMLHttpRequest API call:</p>
<div class="dean_ch" style="white-space: wrap;"><br />
it<span class="br0">&#40;</span><span class="st0">&quot;should give us no results&quot;</span>, function<span class="br0">&#40;</span><span class="br0">&#41;</span> <span class="br0">&#123;</span><br />
&nbsp; runs<span class="br0">&#40;</span>function <span class="br0">&#40;</span><span class="br0">&#41;</span> <span class="br0">&#123;</span><br />
&nbsp; &nbsp; var callback = createSpy<span class="br0">&#40;</span><span class="st0">'callback for results'</span><span class="br0">&#41;</span>;<br />
&nbsp; &nbsp; $.<span class="me1">ajax</span><span class="br0">&#40;</span><span class="br0">&#123;</span><br />
&nbsp; &nbsp; &nbsp; url: <span class="st0">'/search'</span>,<br />
&nbsp; &nbsp; &nbsp; data: <span class="br0">&#123;</span>q: <span class="st0">&quot;query unlikely to match anything&quot;</span><span class="br0">&#125;</span>,<br />
&nbsp; &nbsp; &nbsp; dataType: <span class="st0">&quot;json&quot;</span>,<br />
&nbsp; &nbsp; &nbsp; success: callback<br />
&nbsp; &nbsp; <span class="br0">&#125;</span><span class="br0">&#41;</span>;<br />
&nbsp; <span class="br0">&#125;</span><span class="br0">&#41;</span>;<br />
&nbsp; waits<span class="br0">&#40;</span>someTimeout<span class="br0">&#41;</span>;<br />
&nbsp; runs<span class="br0">&#40;</span>function <span class="br0">&#40;</span><span class="br0">&#41;</span> <span class="br0">&#123;</span><br />
&nbsp; &nbsp; expect<span class="br0">&#40;</span>callback<span class="br0">&#41;</span>.<span class="me1">toHaveBeenCalled</span><span class="br0">&#40;</span><span class="br0">&#41;</span>;<br />
&nbsp; &nbsp; expect<span class="br0">&#40;</span>callback.<span class="me1">mostRecentCall</span>.<span class="me1">args</span><span class="br0">&#91;</span><span class="nu0">0</span><span class="br0">&#93;</span>.<span class="me1">length</span><span class="br0">&#41;</span>.<span class="me1">toEqual</span><span class="br0">&#40;</span><span class="nu0">0</span><span class="br0">&#41;</span>;<br />
&nbsp; <span class="br0">&#125;</span><span class="br0">&#41;</span>;<br />
<span class="br0">&#125;</span><span class="br0">&#41;</span>;<br />
&nbsp;</div>
<p>So, the basic pattern is <tt class="docutils literal"><span class="pre">it()</span></tt> creates a group of tests, and each call to <tt class="docutils literal"><span class="pre">run()</span></tt> is a set of items to call sequentially.  Then between these run blocks you can have signals to the runner to wait for some result, either a timeout (which is fragile), or you can setup specific conditions.</p>
<p>Another popular test runner is <a class="reference external" href="http://docs.jquery.com/Qunit">QUnit</a>; it&#8217;s popular particularly because it&#8217;s what jQuery uses, and my own impression is that QUnit is just very simple and so least likely to piss you off.</p>
<p>QUnit has its own style for async:</p>
<div class="dean_ch" style="white-space: wrap;"><br />
<span class="kw3">test</span><span class="br0">&#40;</span><span class="st0">&quot;should give us no results&quot;</span>, function <span class="br0">&#40;</span><span class="br0">&#41;</span> <span class="br0">&#123;</span><br />
&nbsp; stop<span class="br0">&#40;</span><span class="br0">&#41;</span>;<br />
&nbsp; expect<span class="br0">&#40;</span><span class="nu0">1</span><span class="br0">&#41;</span>;<br />
&nbsp; $.<span class="me1">ajax</span><span class="br0">&#40;</span><span class="br0">&#123;</span><br />
&nbsp; &nbsp; url: <span class="st0">'/search'</span>,<br />
&nbsp; &nbsp; data: <span class="br0">&#123;</span>q: <span class="st0">&quot;query unlikely to match anything&quot;</span><span class="br0">&#125;</span>,<br />
&nbsp; &nbsp; dataType: <span class="st0">&quot;json&quot;</span>,<br />
&nbsp; &nbsp; success: function <span class="br0">&#40;</span>result<span class="br0">&#41;</span> <span class="br0">&#123;</span><br />
&nbsp; &nbsp; &nbsp; ok<span class="br0">&#40;</span>result.<span class="me1">length</span> == <span class="nu0">0</span>, <span class="st0">'No results'</span><span class="br0">&#41;</span>;<br />
&nbsp; &nbsp; &nbsp; start<span class="br0">&#40;</span><span class="br0">&#41;</span>;<br />
&nbsp; &nbsp; <span class="br0">&#125;</span><br />
&nbsp; <span class="br0">&#125;</span><span class="br0">&#41;</span>;<br />
<span class="br0">&#125;</span><span class="br0">&#41;</span>;<br />
&nbsp;</div>
<p><tt class="docutils literal"><span class="pre">stop()</span></tt> confused me for a bit until I realized what they were really referring to stopping the <em>test runner</em>; of course the function continues on regardless.  What will happen is that the function will return, but nothing will have really been tested &#8212; the <tt class="docutils literal"><span class="pre">success</span></tt> callback will not have been run, and <em>cannot</em> run until all Javascript execution stops and control is given back to the browser.  So the test runner will use <tt class="docutils literal"><span class="pre">setTimeout</span></tt> to let time pass before the test continues.  In this case it will continue once <tt class="docutils literal"><span class="pre">start()</span></tt> is called. And <tt class="docutils literal"><span class="pre">expect()</span></tt> also makes it fail if it didn&#8217;t get at least one assertion during that interval &#8212; it would otherwise be easy to simply miss an assertion (though in this example it would be okay because if the success callback isn&#8217;t invoked then <tt class="docutils literal"><span class="pre">start()</span></tt> will never be called, and the runner will timeout and signal that as a failure).</p>
<p>So&#8230; now for doctest.js.  Note that doctest.js isn&#8217;t &quot;plain&quot; Javascript, it looks like what an interactive Javascript session might look like (I&#8217;ve used shell-style prompts instead of typical console prompts, because the consoles didn&#8217;t exist when first I wrote this, and because <tt class="docutils literal"><span class="pre">&gt;&gt;&gt;/...</span></tt> kind of annoy me anyway).</p>
<div class="dean_ch" style="white-space: wrap;"><br />
$ success = Spy<span class="br0">&#40;</span><span class="st0">'success'</span>, <span class="br0">&#123;</span>writes: true<span class="br0">&#125;</span><span class="br0">&#41;</span>;<br />
$ $.<span class="me1">ajax</span><span class="br0">&#40;</span><span class="br0">&#123;</span><br />
&gt; &nbsp; url: <span class="st0">'/search'</span>,<br />
&gt; &nbsp; data: <span class="br0">&#123;</span>q: <span class="st0">&quot;query unlikely to match anything&quot;</span><span class="br0">&#125;</span>,<br />
&gt; &nbsp; dataType: <span class="st0">&quot;json&quot;</span>,<br />
&gt; &nbsp; success: success.<span class="me1">func</span><br />
&gt; <span class="br0">&#125;</span><span class="br0">&#41;</span>;<br />
$ success.<span class="me1">wait</span><span class="br0">&#40;</span><span class="br0">&#41;</span>;<br />
success<span class="br0">&#40;</span><span class="br0">&#91;</span><span class="br0">&#93;</span><span class="br0">&#41;</span><br />
&nbsp;</div>
<p>With doctest.js you still get a fairly linear feel &#8212; it&#8217;s similar to how Jasmine works, except every <tt class="docutils literal"><span class="pre">$</span></tt> prompt is potentially a place where the loop can be released so something async can happen.  Each prompt is equivalent to <tt class="docutils literal"><span class="pre">run()</span></tt> (though unless you call wait, directly or indirectly, everything will run in sequence).</p>
<p>There&#8217;s also an implicit assertion for each stanza, which is anything that is written must be matched (<tt class="docutils literal"><span class="pre">{writes:</span> <span class="pre">true}</span></tt> makes the spy/mock object write out any invocations).  This makes it much harder to miss something in your tests.</p>
<p>I&#8217;ve never actually found Python&#8217;s doctest to be a particularly good way to write docs, and I don&#8217;t expect any different from doctest.js, but I find it a very nice way to <em>write</em> and <em>run</em> tests&#8230; and while Python&#8217;s doctest is essentially abandoned and lacks many features to make it a more humane testing environment, maybe doctest.js can do better.</p>
</div>
]]></content:encoded>
			<wfw:commentRss>http://blog.ianbicking.org/2010/09/12/doctest-js-callbacks/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
		<item>
		<title>The Browser Desktop, developer tools</title>
		<link>http://blog.ianbicking.org/2010/05/04/browser-desktop-developer-tools/</link>
		<comments>http://blog.ianbicking.org/2010/05/04/browser-desktop-developer-tools/#comments</comments>
		<pubDate>Tue, 04 May 2010 21:58:00 +0000</pubDate>
		<dc:creator>Ian Bicking</dc:creator>
				<category><![CDATA[Mozilla]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Web]]></category>

		<guid isPermaLink="false">http://blog.ianbicking.org/?p=228</guid>
		<description><![CDATA[I find myself working in a Windows environment due to some temporary problems with my Linux installation. In terms of user experience Windows is not terrible. But more notable, things mostly just feel the same. My computing experience is not very dependent on the operating system&#8230; almost. Most of what I do is in a [...]]]></description>
			<content:encoded><![CDATA[<div class="document">
<p>I find myself working in a Windows environment due to some temporary problems with my Linux installation.  In terms of user experience Windows is not terrible.  But more notable, things mostly just feel the same.  My computing experience is not very dependent on the operating system&#8230; <em>almost</em>.  Most of what I do is in a web browser &#8212; except programming itself.  Probably a lot of you have the same experience: web browser, text editor, and terminal are pretty much all I need.  I occasionally play with other tools, but none of them stick.  Of course underlying the terminal and text editor UI is a whole host of important software &#8212; interpreters, version control tools, checkouts of all my projects, etc.  So really there&#8217;s two things keeping us from a browser-only world: a few bits of UI, and a whole bunch of tools.  Can we bridge this?  I&#8217;m thinking (more speculatively than as an actual plan): could I stay on Windows without ever having to &quot;use&quot; Windows?</p>
<p>Browsers are clearly capable of implementing a capable UI for a terminal or editor; not a trivial endeavor, but not impossible.  We need a way of handling the tools.  The obvious answer in that case is a virtual machine.  The virtual machine would certainly be using Linux, as there&#8217;s clear consensus that if you remove the UI and hardware considerations and just consider tools then Linux is by far the best choice &#8212; who uses Mac servers? And Windows is barely worth mentioning.  I worked in a Linux VM for a while but found it really unsatisfying &#8212; but that was using the Linux UI through a VMWare interface.</p>
<p>So instead imagine: you start up a headless VM (remembering the tools are not about UI, so there&#8217;s no reason to have a graphical user interface on the VM), you point your browser at this VM, and you use a browser-based developer environment that mediates all the tools (the lightest kind of mediation is just simulating a terminal and using existing console-based interfaces).  Look at your existing setup and just imagine a browser window in place of each not-browser-window app you are using.</p>
<p>I&#8217;m intrigued then by the idea of adding more to these interfaces, incrementally.  Like HTML in the console, or applications lightly wrapping individual tools.  IDEs never stick for me, maybe in part because I can&#8217;t commit, and also there&#8217;s collaboration issues with these tools (I&#8217;m never in a team where we would be able to agree on a single environment).  But incremental decentralized improvements seem genuinely workable &#8212; improvement more in the style of the web, the browser providing the central metaphor.</p>
<p>I call this a Browser Desktop because it&#8217;s a fairly incremental change at this point and other terms (Web OS, Cloud OS) are always presented with unnecessarily hyperbole.  What &quot;operating system&quot; you are using in this imagined system is a somewhat uninteresting semantic question; the OS hasn&#8217;t <em>disappeared</em>, it&#8217;s just <em>boring</em>.  &quot;The Cloud&quot; is fine, but too easy to overthink, and there are many technical reasons to use a hybrid of local and remote pieces.  <a class="reference external" href="http://radar.oreilly.com/2010/03/state-of-internet-operating-system.html">&quot;Internet Operating System&quot;</a> is more a framing concept than a thing-that-can-be-built.  <a class="reference external" href="http://www.chromium.org/chromium-os">Chromium OS</a> is essentially the same idea&#8230; I&#8217;m not really sure how they categorize themselves.</p>
<p>What would be painful right now?  Good Javascript terminals exist.  <a class="reference external" href="http://mozillalabs.com/bespin/">Bespin</a> is hard at work on an editor worthy of being used by programmers.  The browser needs to be an extremely solid platform.  Google Chrome has done a lot in this direction, and Firefox is <a class="reference external" href="https://wiki.mozilla.org/Electrolysis">moving the same direction with the Electrolysis project</a>.  It&#8217;s okay to punt for now on all the &quot;consumer&quot; issues like music and media handling&#8230; and anyway, other people are hard at work on those things.  <a class="reference external" href="http://en.wikipedia.org/wiki/Web_Sockets">Web sockets</a> will help with some kinds of services that ideally will connect directly to a port; it&#8217;s not the same as a raw socket, but I feel like there&#8217;s potential for small intermediaries (e.g., imagine a Javascript app that connects to a locally-hosted server-side app that proxies to ssh).  Also AddOns can be used when necessary (e.g., <cite>ChatZilla &lt;https://addons.mozilla.org/en-US/firefox/addon/16&gt;</cite>).</p>
<p>I&#8217;d like much better management of all these &quot;apps&quot; aka pages aka windows or tabs &#8212; things like split screens and workspaces.  Generally I think using such a system heavily will create all sorts of interesting UI tensions.  Which might be annoying for the user, but if it&#8217;s a constructive annoyance&#8230;</p>
<p>On the whole&#8230; this seems doable.  It&#8217;s navel gazing in a sense &#8212; programmers thinking about programming &#8212; but one good thing about navel gazing is that programmers have traditionally been quite good at navel gazing, and while some results aren&#8217;t generally applicable (e.g., VM management) the exercise will certainly create many generally applicable side products.  It would encourage interesting itch-scratching.  There&#8217;s lots of other &quot;web OS&quot; efforts out there, but I&#8217;ve never really understood them&#8230; they copy desktop metaphors, or have weird filesystem metaphors, or create an unnecessarily cohesive experience.  The web is not cohesive, and I&#8217;m pretty okay with that; I don&#8217;t expect my experiences in this context to be any more cohesive than my tasks are cohesive.  In fact it&#8217;s exactly the lack of cohesiveness that interests me in this exercise &#8212; the browser mostly gives me the level of cohesiveness I want, and I&#8217;m open to experimentation on the rest.  And maybe the biggest interest for me is that I am entirely convinced that traditional GUI applications are a dead end; they rise and fall (mobile apps being a current rise) but I can&#8217;t seriously imagine long term (10 year) viability for any current or upcoming GUI system.  I&#8217;m certain the browser is going to be along for the long haul.  Doing this would let us Live The Future ;)</p>
</div>
]]></content:encoded>
			<wfw:commentRss>http://blog.ianbicking.org/2010/05/04/browser-desktop-developer-tools/feed/</wfw:commentRss>
		<slash:comments>11</slash:comments>
		</item>
		<item>
		<title>Silver Lining: More People!</title>
		<link>http://blog.ianbicking.org/2010/04/21/silver-lining-more-people/</link>
		<comments>http://blog.ianbicking.org/2010/04/21/silver-lining-more-people/#comments</comments>
		<pubDate>Thu, 22 Apr 2010 03:20:00 +0000</pubDate>
		<dc:creator>Ian Bicking</dc:creator>
				<category><![CDATA[Python]]></category>
		<category><![CDATA[Silver Lining]]></category>
		<category><![CDATA[Web]]></category>

		<guid isPermaLink="false">http://blog.ianbicking.org/?p=224</guid>
		<description><![CDATA[OK&#8230; so I said before Silver Lining is for collaborators not users. And that&#8217;s still true&#8230; it&#8217;s not a polished experience where you can confidently ignore the innards of the tool. But it does stuff, and it works, and you can use it. So&#8230; I encourage some more of you to do so. Now would [...]]]></description>
			<content:encoded><![CDATA[<div class="document">
<p>OK&#8230; so I said before Silver Lining is for collaborators not users. And that&#8217;s still true&#8230; it&#8217;s not a polished experience where you can confidently ignore the innards of the tool.  But it does stuff, and it works, and you can use it.  So&#8230; I encourage some more of you to do so.</p>
<p>Now would be a perfectly good time, for instance, to port an application you use to the system.  Almost all Python applications should be portable.  The requirements are fairly simple:</p>
<ol class="arabic simple">
<li>The application needs a WSGI interface.</li>
<li>It needs to be Python 2.6 compatible.</li>
<li>Any libraries that aren&#8217;t pure-Python need to be available as deb packages in some form.</li>
<li>Any persistence needs to be provided as a <a class="reference external" href="http://cloudsilverlining.org/services.html">service</a>; if the appropriate service isn&#8217;t already available you may need to write some code.</li>
</ol>
<p>Also PHP applications should work (though you may encounter more rough edges), with these constraints:</p>
<ol class="arabic simple">
<li>No <tt class="docutils literal"><span class="pre">.htaccess</span></tt> files, so you have to implement any URL rewriting in PHP (e.g., <a class="reference external" href="http://bitbucket.org/ianb/silverlining/src/tip/docs/examples/wordpress/runner.php">for WordPress</a>).</li>
<li>Source code is not writable, so self-installers that write files won&#8217;t work.  (Self-installing plugins might be workable, but that hasn&#8217;t been worked out yet.)</li>
<li>And the same constraints for services.</li>
</ol>
<p>So&#8230; take an application, give it a try, and tell me what you encounter.</p>
<p>Also I&#8217;d love to get feedback and ideas from people with more sysadmin background, or who know Ubuntu/Debian tricks.  For instance, I&#8217;d like to handle some of the questions packages ask about on installation (right now they are all left as defaults, not always the right answer).  I imagine there&#8217;s some non-interactive way to handle those questions but I haven&#8217;t been able to find it.</p>
</div>
]]></content:encoded>
			<wfw:commentRss>http://blog.ianbicking.org/2010/04/21/silver-lining-more-people/feed/</wfw:commentRss>
		<slash:comments>5</slash:comments>
		</item>
		<item>
		<title>WebTest HTTP testing</title>
		<link>http://blog.ianbicking.org/2010/04/02/webtest-http-testing/</link>
		<comments>http://blog.ianbicking.org/2010/04/02/webtest-http-testing/#comments</comments>
		<pubDate>Fri, 02 Apr 2010 22:27:37 +0000</pubDate>
		<dc:creator>Ian Bicking</dc:creator>
				<category><![CDATA[Programming]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Web]]></category>

		<guid isPermaLink="false">http://blog.ianbicking.org/?p=215</guid>
		<description><![CDATA[I&#8217;ve yet to see another testing system for local web testing that I like as much as WebTest&#8230; which is perhaps personal bias for something I wrote, but then I don&#8217;t have that same bias towards everything I&#8217;ve written. Many frameworks build in their own testing systems but I don&#8217;t like the abstractions &#8212; they [...]]]></description>
			<content:encoded><![CDATA[<div class="document">
<p>I&#8217;ve yet to see another testing system for local web testing that I like as much as <a class="reference external" href="http://pythonpaste.org/webtest/">WebTest</a>&#8230; which is perhaps personal bias for something I wrote, but then I don&#8217;t have that same bias towards everything I&#8217;ve written.  Many frameworks build in their own testing systems but I don&#8217;t like the abstractions &#8212; they touch lots of internal things, or skip important steps of the request, or mock out things that don&#8217;t need to be mocked out.  WSGI can make this testing easy.</p>
<p>There&#8217;s also a hidden feature here: because WSGI is basically just describing HTTP, it can be a means of representing not just incoming HTTP requests, but also outgoing HTTP requests.  If you are running local tests against your application using WebTest, with just a little tweaking you can turn those tests into HTTP tests (i.e., actually connect to a socket).  But doing this is admittedly not obvious; hence this post!</p>
<p>Here&#8217;s what a basic WebTest test looks like:</p>
<div class="dean_ch" style="white-space: wrap;"><br />
<span class="kw1">from</span> webtest <span class="kw1">import</span> TestApp<br />
<span class="kw1">import</span> json<br />
<br />
wsgi_app = acquire_wsgi_application_somehow<span class="br0">&#40;</span><span class="br0">&#41;</span><br />
app = TestApp<span class="br0">&#40;</span>wsgi_app<span class="br0">&#41;</span><br />
<br />
<span class="kw1">def</span> test_login<span class="br0">&#40;</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; resp = app.<span class="me1">post</span><span class="br0">&#40;</span><span class="st0">'/login'</span>, <span class="kw2">dict</span><span class="br0">&#40;</span>username=<span class="st0">'guest'</span>, password=<span class="st0">'guest'</span><span class="br0">&#41;</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; resp.<span class="me1">mustcontain</span><span class="br0">&#40;</span><span class="st0">'login successful'</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; resp = resp.<span class="me1">click</span><span class="br0">&#40;</span><span class="st0">'home'</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; resp.<span class="me1">mustcontain</span><span class="br0">&#40;</span><span class="st0">'&lt;a href=&quot;/profile&quot;&gt;guest&lt;/a&gt;'</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; <span class="co1"># Or with a little framework integration:</span><br />
&nbsp; &nbsp; <span class="kw1">assert</span> resp.<span class="me1">templatevars</span>.<span class="me1">get</span><span class="br0">&#40;</span><span class="st0">'username'</span><span class="br0">&#41;</span> == <span class="st0">'guest'</span><br />
<br />
<span class="co1"># Or an API test:</span><br />
<span class="kw1">def</span> test_user_query<span class="br0">&#40;</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; resp = app.<span class="me1">get</span><span class="br0">&#40;</span><span class="st0">'/users.json'</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; <span class="kw1">assert</span> <span class="st0">'guest'</span> <span class="kw1">in</span> resp.<span class="me1">json</span><span class="br0">&#91;</span><span class="st0">'userList'</span><span class="br0">&#93;</span><br />
&nbsp; &nbsp; user_info = <span class="kw2">dict</span><span class="br0">&#40;</span>username=<span class="st0">'guest2'</span>, password=<span class="st0">'guest2'</span>, name=<span class="st0">'Guest'</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; resp = app.<span class="me1">post</span><span class="br0">&#40;</span><span class="st0">'/users.json'</span>, content_type=<span class="st0">'application/json'</span>,<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; body=json.<span class="me1">dumps</span><span class="br0">&#40;</span>user_info<span class="br0">&#41;</span><br />
&nbsp; &nbsp; <span class="kw1">assert</span> resp.<span class="me1">json</span> == user_info<br />
&nbsp;</div>
<p>The <tt class="docutils literal"><span class="pre">app</span></tt> object is a wrapper around the WSGI application, and each of those methods runs a request and gets the response.  The response object is a WebOb response with several additional helpers for testing (things like <tt class="docutils literal"><span class="pre">.click()</span></tt> which finds a link in HTML and follows it, or <tt class="docutils literal"><span class="pre">.json</span></tt> which loads the body as JSON).</p>
<p>You <em>don&#8217;t</em> have to be using a WSGI-centric framework like Pylons to use WebTest, it works fine with anything with a WSGI frontend, which is just about everything.  But the point of my post: you don&#8217;t have to use it with a WSGI application at all.  Using <a class="reference external" href="http://pythonpaste.org/wsgiproxy/">WSGIProxy</a>:</p>
<div class="dean_ch" style="white-space: wrap;"><br />
<span class="kw1">import</span> <span class="kw3">os</span><br />
<span class="kw1">import</span> <span class="kw3">urlparse</span><br />
<br />
<span class="kw1">if</span> <span class="kw3">os</span>.<span class="me1">environ</span>.<span class="me1">get</span><span class="br0">&#40;</span><span class="st0">'TEST_REMOTE'</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; <span class="kw1">from</span> wsgiproxy.<span class="me1">exactproxy</span> <span class="kw1">import</span> proxy_exact_request<br />
&nbsp; &nbsp; wsgi_app = proxy_exact_request<br />
&nbsp; &nbsp; parsed = <span class="kw3">urlparse</span>.<span class="me1">urlsplit</span><span class="br0">&#40;</span><span class="kw3">os</span>.<span class="me1">environ</span><span class="br0">&#91;</span><span class="st0">'TEST_REMOTE'</span><span class="br0">&#93;</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; app = TestApp<span class="br0">&#40;</span>proxy_exact_request, extra_environ=<span class="br0">&#123;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class="st0">'wsgi.scheme'</span>: parsed.<span class="me1">scheme</span>,<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class="st0">'HTTP_HOST'</span>: parsed.<span class="me1">netloc</span>,<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class="st0">'SERVER_NAME'</span>: parsed.<span class="me1">netloc</span><span class="br0">&#125;</span><span class="br0">&#41;</span><br />
<span class="kw1">else</span>:<br />
&nbsp; &nbsp; wsgi_app = acquire_wsgi_application_somehow<span class="br0">&#40;</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; app = TestApp<span class="br0">&#40;</span>wsgi_app<span class="br0">&#41;</span><br />
&nbsp;</div>
<p>It&#8217;s a little crude to control this with an environmental variable (<tt class="docutils literal"><span class="pre">$TEST_REMOTE</span></tt>), but it&#8217;s an easy way to pass an option in when there&#8217;s no better way (and many test runners don&#8217;t make options easy).  The <tt class="docutils literal"><span class="pre">extra_environ</span></tt> option puts in the host and scheme information into each request (the default host WebTest puts in is <tt class="docutils literal"><span class="pre">http://localhost</span></tt>).  WSGIProxy lets you send a request to any host, kind of bypassing DNS, so <tt class="docutils literal"><span class="pre">SERVER_NAME</span></tt> is actually the server the request goes to, while <tt class="docutils literal"><span class="pre">HTTP_HOST</span></tt> is the value of the Host header.</p>
<p>Going over HTTP there are a couple features that won&#8217;t work.  For instance, you can pass information about your application back to the test code by putting values in <tt class="docutils literal"><span class="pre">environ['paste.testing_variables']</span></tt> (which is how you&#8217;d make <tt class="docutils literal"><span class="pre">resp.templatevars</span></tt> work in the first example).  It&#8217;s also possible to use <tt class="docutils literal"><span class="pre">extra_environ</span></tt> to pass information into your application, for example to get your application to mock out user authentication; this is fairly safe because in production no request can put those same special keys into the environment (using custom HTTP headers means you must carefully filter requests in production).  But custom environ values won&#8217;t work over HTTP.</p>
<p>The thing that got me thinking about this is the work I&#8217;m doing on <a class="reference external" href="http://cloudsilverlining.org">Silver Lining</a>, where I am taking apps and rearranging the code and modifying the database configuration ad setup to fit this deployment system.  It would be really nice having done that to be able to run some functional tests, and I really want to run them over HTTP.  If an application has tests using something like Selenium or Windmill that would also work great, but those tools can be a bit more challenging to work with and applications still need smaller tests anyway, so being able to reuse tests like these would be most useful.</p>
</div>
]]></content:encoded>
			<wfw:commentRss>http://blog.ianbicking.org/2010/04/02/webtest-http-testing/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>The Web Server Benchmarking We Need</title>
		<link>http://blog.ianbicking.org/2010/03/16/web-server-benchmarking-we-need/</link>
		<comments>http://blog.ianbicking.org/2010/03/16/web-server-benchmarking-we-need/#comments</comments>
		<pubDate>Tue, 16 Mar 2010 23:23:27 +0000</pubDate>
		<dc:creator>Ian Bicking</dc:creator>
				<category><![CDATA[Programming]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Silver Lining]]></category>
		<category><![CDATA[Web]]></category>

		<guid isPermaLink="false">http://blog.ianbicking.org/?p=198</guid>
		<description><![CDATA[Another WSGI web server benchmark was published. It&#8217;s a decent benchmark, despite some criticisms. But it benchmarks what everyone benchmarks: serving up a trivial app really really quickly. This is not very useful to me. Also, performance is not to me the most important differentiation of servers. In Silver Lining we&#8217;re using mod_wsgi. Silver Lining [...]]]></description>
			<content:encoded><![CDATA[<div class="document">
<p>Another <a class="reference external" href="http://nichol.as/benchmark-of-python-web-servers">WSGI web server benchmark</a> was published.  It&#8217;s a decent benchmark, despite some criticisms.  But it benchmarks what everyone benchmarks: serving up a trivial app really really quickly.  This is not very useful to me.  Also, performance is not to me the most important differentiation of servers.</p>
<p>In <a class="reference external" href="http://cloudsilverlining.org">Silver Lining</a> we&#8217;re using <a class="reference external" href="http://code.google.com/p/modwsgi/">mod_wsgi</a>.  Silver Lining isn&#8217;t tied to mod_wsgi (applications can&#8217;t really tell), and we may revisit that decision (mostly because of memory concerns), but it is a deliberate choice.  mod_wsgi is one of the few multiprocess WSGI servers, and it manages its children (the same way Apache manages all its children).  So if a child stops responding, it gets taken out of the pool and killed (brutal efficiency!  Or at least brutal terminology).  Child processes are also recycled, guarding against memory leaks or other peculiarities.  Sometimes these kinds of things are dismissed for covering up bugs, but (a) production is a lousy time to learn about bugs, (b) it&#8217;s like a third tier of garbage collection, and (c) the bugs you are avoiding are often bugs you can&#8217;t fix anyway (for instance, if your mysql driver leaks memory, is that the application developer&#8217;s fault?)</p>
<p>I wish there was competition among servers not to see who can tweak their performance for entirely unrealistic situations, but to see who can implement the most fail-safe server.  We&#8217;re missing good benchmarks.  Unfortunately benchmarks are a pain in the butt to write and manage.</p>
<p>But I hope someone writes a benchmark like that.  Here&#8217;s some things I&#8217;d like to see benchmarked:</p>
<ul class="simple">
<li>A &quot;realistic&quot; CPU-bound application.  <tt class="docutils literal"><span class="pre">for</span> <span class="pre">i</span> <span class="pre">in</span> <span class="pre">xrange(10000000):</span> <span class="pre">pass</span></tt> is a reasonable start.</li>
<li>An application that generates big responses, e.g., <tt class="docutils literal"><span class="pre">&quot;x&quot;*100000</span></tt>.</li>
<li>An I/O bound application.  E.g., one that reads a big file.</li>
<li>A simply slow application (<tt class="docutils literal"><span class="pre">time.sleep(1)</span></tt>).</li>
<li>Applications that wedge.  <tt class="docutils literal"><span class="pre">while</span> <span class="pre">1:</span> <span class="pre">pass</span></tt> perhaps?  Or <tt class="docutils literal"><span class="pre">lock</span> <span class="pre">=</span> <span class="pre">threading.Lock();</span> <span class="pre">lock.acquire();</span> <span class="pre">lock.acquire()</span></tt>.  Wedging in C and wedging in Python are different, so a bunch of different kinds of wedging.</li>
<li>Applications that segfault.  ctypes is specially designed for this.</li>
<li>Applications that leak memory like a sieve, e.g., <tt class="docutils literal"><span class="pre">global_var.extend(['x']*10000)</span></tt>.</li>
<li>Large uploads.</li>
<li>Slow uploads, like a client that takes 30 seconds to upload 1Mb.</li>
<li>Also slow downloads.</li>
<li>In each case it is interesting what happens when something bad happens to just a portion of requests.  E.g., if 1% of requests wedge hard.  A good container will serve the other 99% of requests properly.  A bad container will have its worker pool exhausted and completely stop.</li>
<li>Mixing and matching these could be interesting.  For instance Dave Beazley <a class="reference external" href="http://www.dabeaz.com/blog/2010/02/revisiting-thread-priorities-and-new.html">found some bad GIL results mixing I/O and CPU-bound code</a>.</li>
<li>Add ideas in the comments and I&#8217;ll copy them into this list.</li>
</ul>
<p>The hardest part of writing this is not the applications (they are simple).  One annoyance is wiring up the applications, but handily Nicholas covers that well in <a class="reference external" href="http://nichol.as/benchmark-of-python-web-servers">his benchmark</a>.  You also have to make sure to clean up, as many servers will not exit cleanly from some of the tests.  Another nuisance is that some of these require funny clients.  These aren&#8217;t too hard to write, but you can&#8217;t just use <tt class="docutils literal"><span class="pre">ab</span></tt>.  Then you have to report.</p>
<p>Anyway: I would <strong>love it</strong> if someone did this, and packaged it as repeatable/runnable code/scripts.  I&#8217;ll help some, but I can&#8217;t lead.  I&#8217;d both really like to see the results, and in my ideal world people writing servers would start using these benchmarks to make their servers more robust.</p>
</div>
]]></content:encoded>
			<wfw:commentRss>http://blog.ianbicking.org/2010/03/16/web-server-benchmarking-we-need/feed/</wfw:commentRss>
		<slash:comments>23</slash:comments>
		</item>
		<item>
		<title>What Does A WebOb App Look Like?</title>
		<link>http://blog.ianbicking.org/2010/03/12/a-webob-app-example/</link>
		<comments>http://blog.ianbicking.org/2010/03/12/a-webob-app-example/#comments</comments>
		<pubDate>Fri, 12 Mar 2010 23:50:30 +0000</pubDate>
		<dc:creator>Ian Bicking</dc:creator>
				<category><![CDATA[Programming]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Web]]></category>

		<guid isPermaLink="false">http://blog.ianbicking.org/?p=192</guid>
		<description><![CDATA[Lately I&#8217;ve been writing code using WebOb and just a few other small libraries. It&#8217;s not entirely obvious what this looks like, so I thought I&#8217;d give a simple example. I make each application a class. Instances of the class are &#34;configured applications&#34;. So it looks a little like this (for an application that takes [...]]]></description>
			<content:encoded><![CDATA[<div class="document">
<p>Lately I&#8217;ve been writing code using WebOb and just a few other small libraries.  It&#8217;s not entirely obvious what this looks like, so I thought I&#8217;d give a simple example.</p>
<p>I make each application a class.  Instances of the class are &quot;configured applications&quot;.  So it looks a little like this (for an application that takes one configuration parameter, <tt class="docutils literal"><span class="pre">file_path</span></tt>):</p>
<div class="dean_ch" style="white-space: wrap;"><br />
<span class="kw1">class</span> Application<span class="br0">&#40;</span><span class="kw2">object</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; <span class="kw1">def</span> <span class="kw4">__init__</span><span class="br0">&#40;</span><span class="kw2">self</span>, file_path<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw2">self</span>.<span class="me1">file_path</span> = file_path<br />
&nbsp;</div>
<p>Then the app needs to be a WSGI app, because that&#8217;s how I roll.  I use <a class="reference external" href="http://pythonpaste.org/webob/modules/dec.html">webob.dec</a>:</p>
<div class="dean_ch" style="white-space: wrap;"><br />
<span class="kw1">from</span> webob.<span class="me1">dec</span> <span class="kw1">import</span> wsgify<br />
<span class="kw1">from</span> webob <span class="kw1">import</span> exc<br />
<span class="kw1">from</span> webob <span class="kw1">import</span> Response<br />
<br />
<span class="kw1">class</span> Application<span class="br0">&#40;</span><span class="kw2">object</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; <span class="kw1">def</span> <span class="kw4">__init__</span><span class="br0">&#40;</span><span class="kw2">self</span>, file_path<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw2">self</span>.<span class="me1">file_path</span> = file_path<br />
&nbsp; &nbsp; @<span class="co1">wsgify</span><br />
&nbsp; &nbsp; <span class="kw1">def</span> <span class="kw4">__call__</span><span class="br0">&#40;</span><span class="kw2">self</span>, req<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">return</span> Response<span class="br0">&#40;</span><span class="st0">'Hi!'</span><span class="br0">&#41;</span><br />
&nbsp;</div>
<p>Somewhere separate from the application you actually instantiate <tt class="docutils literal"><span class="pre">Application</span></tt>.  You can use <a class="reference external" href="http://pythonpaste.org/deploy/">Paste Deploy</a> for that, configure it yourself, or just do something ad hoc (a lot of mod_wsgi <tt class="docutils literal"><span class="pre">.wsgi</span></tt> files are like this, basically).</p>
<p>I use <a class="reference external" href="http://pythonpaste.org/webob/reference.html#exceptions">webob.exc</a> for things like <tt class="docutils literal"><span class="pre">exc.HTTPNotFound()</span></tt>.  You can raise that as an exception, but I mostly just return the object (to the same effect).</p>
<p>Now you have Hello World.  I then sometimes do something terrible, I start handling URLs like this:</p>
<div class="dean_ch" style="white-space: wrap;"><br />
@<span class="co1">wsgify</span><br />
<span class="kw1">def</span> <span class="kw4">__call__</span><span class="br0">&#40;</span><span class="kw2">self</span>, req<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; <span class="kw1">if</span> req.<span class="me1">path_info</span> == <span class="st0">'/'</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">return</span> <span class="kw2">self</span>.<span class="me1">index</span><span class="br0">&#40;</span>req<span class="br0">&#41;</span><br />
&nbsp; &nbsp; <span class="kw1">elif</span> req.<span class="me1">path_info</span>.<span class="me1">startswith</span><span class="br0">&#40;</span><span class="st0">'/view/'</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">return</span> <span class="kw2">self</span>.<span class="me1">view</span><span class="br0">&#40;</span>req<span class="br0">&#41;</span><br />
&nbsp; &nbsp; <span class="kw1">return</span> exc.<span class="me1">HTTPNotFound</span><span class="br0">&#40;</span><span class="br0">&#41;</span><br />
&nbsp;</div>
<p>This is lazy and a very bad idea.  So you want a dispatcher.  There are several (e.g., <a class="reference external" href="http://pypi.python.org/pypi/selector/">selector</a>).  I&#8217;ll use <a class="reference external" href="http://routes.groovie.org">Routes</a> here&#8230; the latest release makes it a bit easier (though it could still be streamlined a bit).  Here&#8217;s a pattern I think makes sense:</p>
<div class="dean_ch" style="white-space: wrap;"><br />
<span class="kw1">from</span> routes <span class="kw1">import</span> Mapper<br />
<br />
<span class="kw1">class</span> Application<span class="br0">&#40;</span><span class="kw2">object</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; <span class="kw2">map</span> = Mapper<span class="br0">&#40;</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; <span class="kw2">map</span>.<span class="me1">connect</span><span class="br0">&#40;</span><span class="st0">'index'</span>, <span class="st0">'/'</span>, method=<span class="st0">'index'</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; <span class="kw2">map</span>.<span class="me1">connect</span><span class="br0">&#40;</span><span class="st0">'view'</span>, <span class="st0">'/view/{item}'</span>, method=<span class="st0">'view'</span><span class="br0">&#41;</span><br />
<br />
&nbsp; &nbsp; <span class="kw1">def</span> <span class="kw4">__init__</span><span class="br0">&#40;</span><span class="kw2">self</span>, file_path<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw2">self</span>.<span class="me1">file_path</span> = file_path<br />
<br />
&nbsp; &nbsp; @<span class="co1">wsgify</span><br />
&nbsp; &nbsp; <span class="kw1">def</span> <span class="kw4">__call__</span><span class="br0">&#40;</span><span class="kw2">self</span>, req<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; results = <span class="kw2">self</span>.<span class="kw2">map</span>.<span class="me1">routematch</span><span class="br0">&#40;</span>environ=req.<span class="me1">environ</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">if</span> <span class="kw1">not</span> results:<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">return</span> exc.<span class="me1">HTTPNotFound</span><span class="br0">&#40;</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; match, route = results<br />
&nbsp; &nbsp; &nbsp; &nbsp; link = URLGenerator<span class="br0">&#40;</span><span class="kw2">self</span>.<span class="kw2">map</span>, req.<span class="me1">environ</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; req.<span class="me1">urlvars</span> = <span class="br0">&#40;</span><span class="br0">&#40;</span><span class="br0">&#41;</span>, match<span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; kwargs = match.<span class="kw3">copy</span><span class="br0">&#40;</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; method = kwargs.<span class="me1">pop</span><span class="br0">&#40;</span><span class="st0">'method'</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; req.<span class="me1">link</span> = link<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">return</span> <span class="kw2">getattr</span><span class="br0">&#40;</span><span class="kw2">self</span>, method<span class="br0">&#41;</span><span class="br0">&#40;</span>req, **kwargs<span class="br0">&#41;</span><br />
<br />
&nbsp; &nbsp; <span class="kw1">def</span> index<span class="br0">&#40;</span><span class="kw2">self</span>, req<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; ...<br />
&nbsp; &nbsp; <span class="kw1">def</span> view<span class="br0">&#40;</span><span class="kw2">self</span>, req, item<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; ...<br />
&nbsp;</div>
<p>Another way you might do it is to skip the class, which means skipping a clear place for configuration.  I don&#8217;t like that, but if you don&#8217;t care about that, then it looks like this:</p>
<div class="dean_ch" style="white-space: wrap;"><br />
<span class="kw1">def</span> index<span class="br0">&#40;</span><span class="kw2">self</span>, req<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; ...<br />
<span class="kw1">def</span> view<span class="br0">&#40;</span><span class="kw2">self</span>, req, item<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; ...<br />
<br />
<span class="kw2">map</span> = Mapper<span class="br0">&#40;</span><span class="br0">&#41;</span><br />
<span class="kw2">map</span>.<span class="me1">connect</span><span class="br0">&#40;</span><span class="st0">'index'</span>, <span class="st0">'/'</span>, view=index<span class="br0">&#41;</span><br />
<span class="kw2">map</span>.<span class="me1">connect</span><span class="br0">&#40;</span><span class="st0">'view'</span>, <span class="st0">'/view/{item}'</span>, view=view<span class="br0">&#41;</span><br />
<br />
@<span class="co1">wsgify</span><br />
<span class="kw1">def</span> application<span class="br0">&#40;</span>req<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; results = <span class="kw2">map</span>.<span class="me1">routematch</span><span class="br0">&#40;</span>environ=req.<span class="me1">environ</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; <span class="kw1">if</span> <span class="kw1">not</span> results:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">return</span> exc.<span class="me1">HTTPNotFound</span><span class="br0">&#40;</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; match, route = results<br />
&nbsp; &nbsp; link = URLGenerator<span class="br0">&#40;</span><span class="kw2">map</span>, req.<span class="me1">environ</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; req.<span class="me1">urlvars</span> = <span class="br0">&#40;</span><span class="br0">&#40;</span><span class="br0">&#41;</span>, match<span class="br0">&#41;</span><br />
&nbsp; &nbsp; kwargs = match.<span class="kw3">copy</span><span class="br0">&#40;</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; view = kwargs.<span class="me1">pop</span><span class="br0">&#40;</span><span class="st0">'view'</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; req.<span class="me1">link</span> = link<br />
&nbsp; &nbsp; <span class="kw1">return</span> view<span class="br0">&#40;</span>req, **kwargs<span class="br0">&#41;</span><br />
&nbsp;</div>
<p>Then <tt class="docutils literal"><span class="pre">application</span></tt> is pretty much boilerplate.  You could put configuration in the request if you wanted, or use some other technique (like <a class="reference external" href="http://pypi.python.org/pypi/Contextual">Contextual</a>).</p>
<p>I talked some with <a class="reference external" href="http://be.groovie.org/">Ben Bangert</a> about what he&#8217;s trying with these patterns, and he&#8217;s doing something reminiscent of Pylons controllers (but without the rest of Pylons) and it looks more like this (with my own adaptations):</p>
<div class="dean_ch" style="white-space: wrap;"><br />
<span class="kw1">class</span> BaseController<span class="br0">&#40;</span><span class="kw2">object</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; special_vars = <span class="br0">&#91;</span><span class="st0">'controller'</span>, <span class="st0">'action'</span><span class="br0">&#93;</span><br />
<br />
&nbsp; &nbsp; <span class="kw1">def</span> <span class="kw4">__init__</span><span class="br0">&#40;</span><span class="kw2">self</span>, request, link, **config<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw2">self</span>.<span class="me1">request</span> = request<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw2">self</span>.<span class="me1">link</span> = link<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">for</span> name, value <span class="kw1">in</span> config.<span class="me1">items</span><span class="br0">&#40;</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class="kw2">setattr</span><span class="br0">&#40;</span><span class="kw2">self</span>, name, value<span class="br0">&#41;</span><br />
<br />
&nbsp; &nbsp; <span class="kw1">def</span> <span class="kw4">__call__</span><span class="br0">&#40;</span><span class="kw2">self</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; action = <span class="kw2">self</span>.<span class="me1">request</span>.<span class="me1">urlvars</span>.<span class="me1">get</span><span class="br0">&#40;</span><span class="st0">'action'</span>, <span class="st0">'index'</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">if</span> <span class="kw2">hasattr</span><span class="br0">&#40;</span><span class="kw2">self</span>, <span class="st0">'__before__'</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class="kw2">self</span>.__before__<span class="br0">&#40;</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; kwargs = req.<span class="me1">urlsvars</span>.<span class="kw3">copy</span><span class="br0">&#40;</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">for</span> attr <span class="kw1">in</span> <span class="kw2">self</span>.<span class="me1">special_vars</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">if</span> attr <span class="kw1">in</span> kwargs:<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">del</span> kwargs<span class="br0">&#91;</span>attr<span class="br0">&#93;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">return</span> <span class="kw2">getattr</span><span class="br0">&#40;</span><span class="kw2">self</span>, action<span class="br0">&#41;</span><span class="br0">&#40;</span>**kwargs<span class="br0">&#41;</span><br />
<br />
<span class="kw1">class</span> Index<span class="br0">&#40;</span>BaseController<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; <span class="kw1">def</span> index<span class="br0">&#40;</span><span class="kw2">self</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; ...<br />
&nbsp; &nbsp; <span class="kw1">def</span> view<span class="br0">&#40;</span><span class="kw2">self</span>, item<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; ...<br />
<br />
<span class="kw1">class</span> Application<span class="br0">&#40;</span><span class="kw2">object</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; <span class="kw2">map</span> = Mapper<span class="br0">&#40;</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; <span class="kw2">map</span>.<span class="me1">connect</span><span class="br0">&#40;</span><span class="st0">'index'</span>, <span class="st0">'/'</span>, controller=Index<span class="br0">&#41;</span><br />
&nbsp; &nbsp; <span class="kw2">map</span>.<span class="me1">connect</span><span class="br0">&#40;</span><span class="st0">'view'</span>, <span class="st0">'/view/{item}'</span>, controller=Index, &nbsp; &nbsp; action=<span class="st0">'view'</span><span class="br0">&#41;</span><br />
<br />
&nbsp; &nbsp; <span class="kw1">def</span> <span class="kw4">__init__</span><span class="br0">&#40;</span><span class="kw2">self</span>, **config<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw2">self</span>.<span class="me1">config</span> = config<br />
<br />
&nbsp; &nbsp; @<span class="co1">wsgify</span><br />
&nbsp; &nbsp; <span class="kw1">def</span> <span class="kw4">__call__</span><span class="br0">&#40;</span><span class="kw2">self</span>, req<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; results = <span class="kw2">self</span>.<span class="kw2">map</span>.<span class="me1">routematch</span><span class="br0">&#40;</span>environ=req.<span class="me1">environ</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">if</span> <span class="kw1">not</span> results:<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">return</span> exc.<span class="me1">HTTPNotFound</span><span class="br0">&#40;</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; match, route = results<br />
&nbsp; &nbsp; &nbsp; &nbsp; link = URLGenerator<span class="br0">&#40;</span><span class="kw2">self</span>.<span class="kw2">map</span>, req.<span class="me1">environ</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; req.<span class="me1">urlvars</span> = <span class="br0">&#40;</span><span class="br0">&#40;</span><span class="br0">&#41;</span>, match<span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; controller = match<span class="br0">&#91;</span><span class="st0">'controller'</span><span class="br0">&#93;</span><span class="br0">&#40;</span>req, link, **<span class="kw2">self</span>.<span class="me1">config</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">return</span> controller<span class="br0">&#40;</span><span class="br0">&#41;</span><br />
&nbsp;</div>
<p>That&#8217;s a lot of code blocks, but they all really say the same thing ;)  I think writing apps with almost-no-framework like this is pretty doable, so if you have something small you should give it a go.  I think it&#8217;s especially appropriate for applications that are an API (not a &quot;web site&quot;).</p>
</div>
]]></content:encoded>
			<wfw:commentRss>http://blog.ianbicking.org/2010/03/12/a-webob-app-example/feed/</wfw:commentRss>
		<slash:comments>30</slash:comments>
		</item>
		<item>
		<title>toppcloud renamed to Silver Lining</title>
		<link>http://blog.ianbicking.org/2010/03/03/toppcloud-renamed-to-silver-lining/</link>
		<comments>http://blog.ianbicking.org/2010/03/03/toppcloud-renamed-to-silver-lining/#comments</comments>
		<pubDate>Thu, 04 Mar 2010 02:18:30 +0000</pubDate>
		<dc:creator>Ian Bicking</dc:creator>
				<category><![CDATA[Packaging]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Silver Lining]]></category>
		<category><![CDATA[Web]]></category>

		<guid isPermaLink="false">http://blog.ianbicking.org/?p=168</guid>
		<description><![CDATA[After some pondering at PyCon, I decided on a new name for toppcloud: Silver Lining. I&#8217;ll credit a mysterious commenter &#34;david&#34; with the name idea. The command line is simply silver &#8212; silver update has a nice ring to it. There&#8217;s a new site: cloudsilverlining.org; not notably different than the old site, just a new [...]]]></description>
			<content:encoded><![CDATA[<div class="document">
<p>After some pondering at PyCon, I decided on a new name for toppcloud: <strong>Silver Lining</strong>.  I&#8217;ll credit a mysterious commenter &quot;david&quot; with the name idea.  The command line is simply <tt class="docutils literal"><span class="pre">silver</span></tt> &#8212; <tt class="docutils literal"><span class="pre">silver</span> <span class="pre">update</span></tt> has a nice ring to it.</p>
<p>There&#8217;s a new site: <a class="reference external" href="http://cloudsilverlining.org">cloudsilverlining.org</a>; not notably different than the old site, just a new name.  The product is self-hosting now, using a <a class="reference external" href="http://bitbucket.org/ianb/silverlining/">simple app</a> that runs after every commit to regenerate the docs, and with a small extension to Silver Lining itself (to make it easier to host static files).  Now that it has a real name I also gave it a <a class="reference external" href="http://groups.google.com/group/silverlining-dev">real mailing list</a>.</p>
<p>Silver Lining also has its <a class="reference external" href="http://bitbucket.org/ianb/silverlining/src/tip/tests/functional/runtest.py">first test</a>.  Not an impressive test, but a test.  I&#8217;m hoping with a <a class="reference external" href="http://mail-archives.apache.org/mod_mbox/incubator-libcloud/201003.mbox/browser">VM-based libcloud backend</a> that a full integration test can run in a reasonable amount of time.  <em>Some</em> unit tests would be possible, but so far most of the bugs have been interaction bugs so I think integration tests will have to pull most of the weight.  (A continuous integration rig will be very useful; I am not sure if Silver Lining can self-host that, though it&#8217;d be nice/clever if it could.)</p>
</div>
]]></content:encoded>
			<wfw:commentRss>http://blog.ianbicking.org/2010/03/03/toppcloud-renamed-to-silver-lining/feed/</wfw:commentRss>
		<slash:comments>7</slash:comments>
		</item>
		<item>
		<title>Throw out your frameworks! (forms included)</title>
		<link>http://blog.ianbicking.org/2010/03/01/throw-out-your-frameworks-forms-included/</link>
		<comments>http://blog.ianbicking.org/2010/03/01/throw-out-your-frameworks-forms-included/#comments</comments>
		<pubDate>Mon, 01 Mar 2010 06:29:04 +0000</pubDate>
		<dc:creator>Ian Bicking</dc:creator>
				<category><![CDATA[HTML]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Web]]></category>

		<guid isPermaLink="false">http://blog.ianbicking.org/?p=164</guid>
		<description><![CDATA[No, I should say forms particularly. I have lots of things to blog about, but nothing makes me want to blog like code. Ideas are hard, code is easy. So when I saw Jacob&#8217;s writeup about dynamic Django form generation I felt a desire to respond. I didn&#8217;t see the form panel at PyCon (I [...]]]></description>
			<content:encoded><![CDATA[<div class="document">
<p>No, I should say <em>forms particularly</em>.</p>
<p>I have lots of things to blog about, but nothing makes me want to blog like <em>code</em>.  Ideas are hard, code is easy.  So when I saw <a class="reference external" href="http://jacobian.org/writing/dynamic-form-generation/">Jacob&#8217;s writeup about dynamic Django form generation</a> I felt a desire to respond.  I didn&#8217;t see the form panel at PyCon (I intended to but I hardly saw <em>any</em> talks at PyCon, and yet still didn&#8217;t even see a good number of the people I wanted to see), but as the author of an <a class="reference external" href="http://formencode.org/htmlfill.html">ungenerator</a> and as a general <a class="reference external" href="http://blog.ianbicking.org/on-form-libraries.html">form library skeptic</a> I have a somewhat different perspective on the topic.</p>
<p>The example created for the panel might display that perspective.  You should go read <a class="reference external" href="http://jacobian.org/writing/dynamic-form-generation/">Jacob&#8217;s description</a>; but basically it&#8217;s a simple registration form with a dynamic set of questions to ask.</p>
<p>I have created a <a class="reference external" href="http://svn.colorstudy.com/home/ianb/formencode_answer.py">complete example</a>, because I wanted to be sure I wasn&#8217;t skipping anything, but I&#8217;ll present a trimmed-down version.</p>
<p>First, the basic control logic:</p>
<div class="dean_ch" style="white-space: wrap;"><br />
<span class="kw1">from</span> webob.<span class="me1">dec</span> <span class="kw1">import</span> wsgify<br />
<span class="kw1">from</span> webob <span class="kw1">import</span> exc<br />
<span class="kw1">from</span> formencode <span class="kw1">import</span> htmlfill<br />
<br />
@<span class="co1">wsgify</span><br />
<span class="kw1">def</span> questioner<span class="br0">&#40;</span>req<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; questions = get_questions<span class="br0">&#40;</span>req<span class="br0">&#41;</span> <span class="co1"># This is provided as part of the example</span><br />
&nbsp; &nbsp; <span class="kw1">if</span> req.<span class="me1">method</span> == <span class="st0">'POST'</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; errors = validate<span class="br0">&#40;</span>req, questions<span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">if</span> <span class="kw1">not</span> errors:<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ... <span class="me1">save</span> response ...<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">return</span> exc.<span class="me1">HTTPFound</span><span class="br0">&#40;</span>location=<span class="st0">'/thanks'</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; <span class="kw1">else</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; errors = <span class="br0">&#123;</span><span class="br0">&#125;</span><br />
&nbsp; &nbsp; <span class="co1">## Here's the &quot;form generation&quot;:</span><br />
&nbsp; &nbsp; page = page_template.<span class="me1">substitute</span><span class="br0">&#40;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; action=req.<span class="me1">url</span>,<br />
&nbsp; &nbsp; &nbsp; &nbsp; questions=questions<span class="br0">&#41;</span><br />
&nbsp; &nbsp; page = htmlfill.<span class="me1">render</span><span class="br0">&#40;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; page,<br />
&nbsp; &nbsp; &nbsp; &nbsp; defaults=req.<span class="me1">POST</span>,<br />
&nbsp; &nbsp; &nbsp; &nbsp; errors=errors<span class="br0">&#41;</span><br />
&nbsp; &nbsp; <span class="kw1">return</span> Response<span class="br0">&#40;</span>page<span class="br0">&#41;</span><br />
<br />
<span class="kw1">def</span> validate<span class="br0">&#40;</span>req, questions<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; <span class="co1"># All manual, but do it however you want:</span><br />
&nbsp; &nbsp; errors = <span class="br0">&#123;</span><span class="br0">&#125;</span><br />
&nbsp; &nbsp; form = req.<span class="me1">POST</span><br />
&nbsp; &nbsp; <span class="kw1">if</span> <span class="br0">&#40;</span>form.<span class="me1">get</span><span class="br0">&#40;</span><span class="st0">'password'</span><span class="br0">&#41;</span><br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">and</span> form<span class="br0">&#91;</span><span class="st0">'password'</span><span class="br0">&#93;</span> != form.<span class="me1">get</span><span class="br0">&#40;</span><span class="st0">'password_confirm'</span><span class="br0">&#41;</span><span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; errors<span class="br0">&#91;</span><span class="st0">'password_confirm'</span><span class="br0">&#93;</span> = <span class="st0">'Passwords do not match'</span><br />
&nbsp; &nbsp; fields = questions + <span class="br0">&#91;</span><span class="st0">'username'</span>, <span class="st0">'password'</span><span class="br0">&#93;</span><br />
&nbsp; &nbsp; <span class="kw1">for</span> field <span class="kw1">in</span> fields:<br />
&nbsp; &nbsp; &nbsp; &nbsp; <span class="kw1">if</span> <span class="kw1">not</span> form.<span class="me1">get</span><span class="br0">&#40;</span>field<span class="br0">&#41;</span>:<br />
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; errors<span class="br0">&#91;</span>field<span class="br0">&#93;</span> = <span class="st0">'Please enter a value'</span><br />
&nbsp; &nbsp; <span class="kw1">return</span> errors<br />
&nbsp;</div>
<p>I&#8217;ve just manually handled validation here.  I don&#8217;t feel like doing it with FormEncode.  Manual validation isn&#8217;t that big a deal; FormEncode would just produce the same <tt class="docutils literal"><span class="pre">errors</span></tt> dictionary anyway.  In this case (as in many form validation cases) you can&#8217;t do better than hand-written validation code: it&#8217;s shorter, more self-contained, and easier to tweak.</p>
<p>After validation the template is rendered:</p>
<div class="dean_ch" style="white-space: wrap;"><br />
page = page_template.<span class="me1">substitute</span><span class="br0">&#40;</span><br />
&nbsp; &nbsp; action=req.<span class="me1">url</span>,<br />
&nbsp; &nbsp; questions=questions<span class="br0">&#41;</span><br />
&nbsp;</div>
<p>I&#8217;m using <a class="reference external" href="http://pythonpaste.org/tempita/">Tempita</a>, but it really doesn&#8217;t matter.  The template looks like this:</p>
<div class="dean_ch" style="white-space: wrap;"><br />
&lt;form action=<span class="st0">&quot;{{action}}&quot;</span> method=<span class="st0">&quot;POST&quot;</span>&gt;<br />
New Username: &lt;input <span class="kw2">type</span>=<span class="st0">&quot;text&quot;</span> name=<span class="st0">&quot;username&quot;</span>&gt;&lt;br /&gt;<br />
Password: &lt;input <span class="kw2">type</span>=<span class="st0">&quot;password&quot;</span> name=<span class="st0">&quot;password&quot;</span>&gt;&lt;br /&gt;<br />
Repeat Password:<br />
&nbsp; &lt;input <span class="kw2">type</span>=<span class="st0">&quot;password&quot;</span> name=<span class="st0">&quot;password_confirm&quot;</span>&gt;&lt;br /&gt;<br />
<span class="br0">&#123;</span><span class="br0">&#123;</span><span class="kw1">for</span> question <span class="kw1">in</span> questions<span class="br0">&#125;</span><span class="br0">&#125;</span><br />
&nbsp; <span class="br0">&#123;</span><span class="br0">&#123;</span>question<span class="br0">&#125;</span><span class="br0">&#125;</span>: &lt;input <span class="kw2">type</span>=<span class="st0">&quot;text&quot;</span> name=<span class="st0">&quot;{{question}}&quot;</span>&gt;&lt;br /&gt;<br />
<span class="br0">&#123;</span><span class="br0">&#123;</span>endfor<span class="br0">&#125;</span><span class="br0">&#125;</span><br />
&lt;input <span class="kw2">type</span>=<span class="st0">&quot;submit&quot;</span>&gt;<br />
&lt;/form&gt;<br />
&nbsp;</div>
<p>Note that the only &quot;logic&quot; here is to render the form to include fields for all the questions.  Obviously this produces an ugly form, but it&#8217;s <em>very obvious</em> how you make this form pretty, and how to tweak it in any way you might want.  Also if you have deeper dynamicism (e.g., <tt class="docutils literal"><span class="pre">get_questions</span></tt> start returning the type of response required, or weird validation, or whatever) it&#8217;s <em>very obvious</em> where that change would go: display logic goes in the form, validation logic goes in that validate function.</p>
<p>This just gives you the raw form.  You wouldn&#8217;t need a template at all if it wasn&#8217;t for the dynamicism.  Everything else is added when the form is &quot;filled&quot;:</p>
<div class="dean_ch" style="white-space: wrap;"><br />
page = htmlfill.<span class="me1">render</span><span class="br0">&#40;</span><br />
&nbsp; &nbsp; page,<br />
&nbsp; &nbsp; defaults=req.<span class="me1">POST</span>,<br />
&nbsp; &nbsp; errors=errors<span class="br0">&#41;</span><br />
&nbsp;</div>
<p>How exactly you want to calculate <tt class="docutils literal"><span class="pre">defaults</span></tt> is up to the application; you might want query string variables to be able to pre-fill the form (use <tt class="docutils literal"><span class="pre">req.params</span></tt>), you might want the form bare to start (like here with <tt class="docutils literal"><span class="pre">req.POST</span></tt>), you can easily implement wizards by stuffing <tt class="docutils literal"><span class="pre">req.POST</span></tt> into the session to repeat a form, you might read the defaults out of a user object to make this an edit form.  And errors are just handled automatically, inserted into the HTML with appropriate CSS classes.</p>
<p>A great aspect of this <em>pattern</em> if you use it (I&#8217;m not even sure it deserves the moniker <em>library</em>): when <a class="reference external" href="http://www.whatwg.org/specs/web-apps/current-work/multipage/forms.html#forms">HTML 5 Forms</a> finally come around and we can all stop doing this stupid server-side overthought nonsense, <strong>you</strong> won&#8217;t have overthought your forms.  Your mind will be free and ready to accept that the world has actually become simpler, not more complicated, and that there is knowledge worth forgetting (forms are so freakin&#8217; stupid!)  If at all possible, dodging complexity is far better than cleverly responding to complexity.</p>
</div>
]]></content:encoded>
			<wfw:commentRss>http://blog.ianbicking.org/2010/03/01/throw-out-your-frameworks-forms-included/feed/</wfw:commentRss>
		<slash:comments>27</slash:comments>
		</item>
	</channel>
</rss>

<!-- Dynamic Page Served (once) in 0.770 seconds -->

