Ian Bicking: a blog

I’ve seen talk of MS Silverlight and Adobe AIR. People talk them up like the future of web applications or something. I don’t know much about them, but I almost completely certain I don’t want anything to do with them.

Here’s a general rule I have: I don’t accept anything made by people who hate the web. If you hate the web and you want to improve the web, I don’t want anything to do with you. If you think the web is some kind of implementation detail then I probably don’t care what you are doing. If you still think the web is a fad, then you are just nuts; if all you can think of is reasons why the web is stupid and awkward, and you think it’s some giant step backward (from what?), then you haven’t thought very deeply about what’s happened in the world of technology and why.

To me Silverlight and AIR reek of a distaste for the web.

So it is with great delight that I read the announcement of Mozilla Prism: a bridge between the desktop and the web, written by people who don’t hate the web.

The premise of Prism from what I can tell is this: to make a web application into a desktop application, you just give the browser a shallowly separate identity. It lives in its own window, has its own icon, it’s own launcher. Maybe runs in its own process for reliability. You take away the chrome (URL bars, back button, etc). Unlike previous ideas for Mozilla (like xulrunner) you don’t add chrome back in with XUL, you just write all the controls with HTML and Javascript.

All the things that Prism isn’t is what makes it great, because it’s so damn simple. The only thing that really seems weird to me is that it is very separate from the browser itself; hopefully this is just temporary, and by the time it’s really "done" you’ll be able to just make any page an application directly from Firefox.

This doesn’t make web applications perfect, of course. There’s the offline issue. There’s usability issues, like keybindings. There’s lots of issues, but all of those issues apply just as much to the web as to a desktop application, and should be solved both places. Those things can be worked on orthogonally (most interestingly in Web Forms 2.0 and Web Applications 1.0 at WHAT-WG). Still, HTML and Javascript right now are totally workable for most applications.

Finally, someone wrote a version of doctest for Ruby.

Recently I’ve been writing most of my tests using stand-alone doctest files. It’s a great way to do TDD — mostly because the cognitive load is so low. Also, I write my examples but don’t write my output, then copy the output after visually confirming it is correct. So the basic pattern is:

• Figure out what I want to do
• Figure out how I want to test it
• Automate my conditions
• Manually inspect whether the output is correct (i.e., implement and debug)
• Copy the output so that in the future the manual process is automated (doctest-mode for Emacs makes this particularly easy)

The result is a really good balance of manual and automated testing, I think giving you the benefit of both processes — the ease of manual testing, and the robustness of automated testing.

Another good thing about doctest is it doesn’t let you hide any boilerplate and setup. If it’s easy to use doctest, it’s probably easy to use the library.

There’s nothing Python-specific about doctest (e.g., doctestjs), so it’s good to see it moving to other languages. Even if the language doesn’t have a REPL, IMHO it’s worth inventing it just for this.

One of the thinmgs I would like to do is to interact with Atompub (aka Atom Publishing Protocol) stores in Javascript through the browser. Since this effectively the browser itself interacting with the Atompub server, browser-like authentication methods would be nice. But services like Atompub don’t work nicely with the kinds of authentication methods that normal websites use. One of these is OpenID, which is particularly browser-focused.

From the perspective of a client, OpenID basically works like this:

• You need to login. You tell the original server what your OpenID URL is, somehow.
• The original server does some redirects, maybe some popups, etc.
• Your OpenID server (attached to your OpenID URL) authenticates you in some fashion, and then tells the original server.
• The original server probably sets a signed cookie so that in subsequent requests you stay logged in. You cannot do this little redirection dance for every request, since it’s actually quite intrusive.

So what happens when I have an XMLHttpRequest that needs to be authenticated? Neither the XMLHttpRequest nor Javascript generally can do the authentication. Only the browser can, with the user’s interaction.

One thought I have is a 401 Unauthorized response, with a header like:

WWW-Authenticate: Cookie location="http://original.server/login.html"

Which means I need to open up http://original.server/login.html and have the user log in, and the final result is that a cookie will be set. XMLHttpRequest sends cookies automatically I believe, so once the browser has the cookie then all the Javascript requests get the same cookie and hence authentication.

One problem, though, is that you have to wait around for a while for the login to succede, then continue on your way. A typical situation is that you have to return to the original page you were requesting, and people often do something like /login?redirect_to=original_url. In this case we might want something like /login?opener_call=reattempt_request, where when the login process is over we call window.opener.reattempt_request() in Javascript.

Maybe it would make sense for that location variable to be a URI Template, with some predefined variables, like opener, back, etc.

For general backward compatibility, would it be reasonable to send 307 Temporary Redirect plus WWW-Authenticate, and let XMLHttpRequests or other service clients sort it out, while normal browser requests do the normal login redirect?

Update: Another question/thought: is it okay to send multiple WWW-Authenticate headers, to give the client options for how it wants to do authentication? It seems vaguely okay, according to RFC 2616 14.47.

I’ve been doing a bit more with Atom lately.

First, I started writing a library to manipulate Atom feeds and entries. For the moment this is located in taggerclient.atom. It uses lxml, as does everything markup related I do these days.

I came upon a revelation of sorts when I was writing the library. I first started with a library that looked like this:

class Feed(object):
    def __init__(self, title, ...):
        self.title = title
        ..
    @classmethod
    def parse(cls, xml):
        if isinstance(xml, basestring):
            xml = etree.XML(xml)
        title = xml.xpath('//title').text
        ...
        return cls(title, ...)
    def serialize(self):
        el = etree.Element('{%s}feed' % atom_ns)
        title = etree.Element('{%s}title' % atom_ns)
        title.text = self.title
        el.append(title)
        ...
        return el

Obviously there’s ways to improve this and make it less verbose, and I went down that path for a while. But then I decided the whole path was wrong. Atom is XML. It’s not the representation of some object I’m creating. If I have something that can’t be represented in XML, it isn’t Atom, and it doesn’t belong in my Atom-related objects.

So instead I started making lxml more convenient when using Atom. I don’t keep any information except what is in the markup, I just make it more convenient to access that information.

I used lots of descriptors to do this, as the same patterns happened over and over. For instance, the Feed object is fairly simple:

class Feed(AtomElement):
    entries = _findall_property('entry')
    author = _element_property('author')

Which basically means that feed.entries returns all <entry> elements, and feed.author returns the single author element.

There’s also accessors for text elements (like <id>) and date containing elements (like <updated>) and just to access XML attributes as Python attributes.

There’s a number of advantages:

• No hidden state.
• No deferred errors, since everything is always represented in the XML infoset.
• All XML extensions work, even though my classes don’t know anything in particular about them. There’s a full API for manipulating the XML that you can use, you don’t have to use my APIs.
• Even more obscure kinds of extensions work fine, like a custom attribute on an element. There’s absolutely zero normalization that happens.
• I only have to write the parts where the normal XML (lxml) APIs are inconvenient, so the implementation stays simple.
• There’s no confusion over which object I might be talking about in my code. There’s no distinction between the XML object and the domain object.

Since then I’ve been working on a Javascript library for handling Atom. It’s not as elegant. I am trying to keep to this same principle, but of course I can’t actually extend the DOM and so I can’t add convenience methods. So instead I’m making a class that lightly wraps the DOM objects, with explicit getters and setters that simply read and modify those DOM objects.

One thing that I have found very useful in my development on the Javascript side is doctest-style testing. You can see the test, but to run it you have to check it out (it uses some svn:externals which you don’t get through the direct svn access). After using that testing some more and being pleased with the result, I decided to package the Javascript doctest runner a bit better. I removed the framework dependencies, did a bit of renaming (now it is doctestjs or doctest.js instead of jsdoctest), wrote up fairly comprehensive docs, and uploaded it to JSAN (though at the moment the trunk from svn is probably better to use). I think it’s an excellent way of doing unit testing in Javascript, much better than any of the alternatives I’ve seen. It even has some notable advantages over Python’s doctest, like if you are using Firebug (which you must if you do Javascript development) you get a console session that runs in the same namespace as your tests, so you can easily do inspection of the objects if there’s a failure.

I’m not sure about JSAN. It’s nice to have an index. But I think they copy stuff from CPAN a bit too much. Why should you have a text README file? That’s just silly; of course Javascript documentation should be HTML. They batch processing. Processing one package a day
on the fly shouldn’t be overwhelming. They want a MANIFEST file. The standard metadata file is YAML, not JSON. This should all be a little more Javascripty in my opinion. But they also accept any kind of upload, so there’s nothing stopping you from ignoring what you don’t
care about. I’ll probably improve the packaging of doctestjs a bit in the future, and still ignore the parts I think are silly.