Here’s a bit more than a starting point:

http://svn.berlios.de/svnroot/repos/less4j/trunk/src/org/less4j/protocols/Doctest.java

For sample output see:

http://laurentszyster.be/less4j/doc/#org.less4j.protocols.IRTD2.digested

The implementation is simple: I hijacked javadoc and used the best known REPL for Java: Rhino.

Enjoy …

Frederick Bicking From Wikipedia, the free encyclopedia Jump to: navigation, search Frederick Bicking was born in Winterburg, a municipality in the district of Bad Kreuznach in Rhineland-Palatinate, in western Germany and came probably first to Philadelphia and then to East Brandywine Township, Chester County, Pennsylvania, before the revolution.

In Pennsylvania he owned and operated a paper mill, establishing the Bicking paper dynasty that would last well into the 19th century. The Continental Congress allocated funds to purchase Bicking’s paper for currency production. He is mentioned in several of the minutes of the Continental Congress and in the George Washington Papers.

Frederick Bicking married Mary Catherine Unverzagt of Otwiller, Germany on 26 May 1752 at St. Michael’s & Zion Lutheran Church in Germantown, Pennsylvania. Mary Catherine Unverzagt, daughter of Johannes Unverzagt, was also a German Palatine.

Of Frederick Bicking’s five sons, three were paper makers in Pennsylvania.

John Bicking had a paper mill near present day Fisherville.

Retrieved from “http://en.wikipedia.org/wiki/Frederick_Bicking”

You picked an example spec with the greatest possible overhead-to-meaningful-code ratio: 9 lines of setup (much of which, like the import statements, is forced by Java), and only 3 lines of an actual spec (one of which is just an annotation saying that it is, in fact, a spec, so maybe that’s overhead, too).

What the framework gives you, like xUnit, is the ability to do some setup in the setUp() method, then share it among some other spec methods (test methods, in xUnit). It’s pretty rare that there’s only one spec method.

And only the code is shared: the actual instance (or whatever other context you set up) is different for each spec method, which helps keep one problem from kicking up a cloud of spurious follow-on problems to hide itself. I’m not sure how you’d go about that in doctest without writing the equivalent of a setUp() method and calling it repeatedly.

The motivations for BDD are to get people to do TDD right. That means ditching anachronistic test-related names that date from back when we thought this stuff was unit testing. It’s not: it’s design, and we should use words related to design instead. (If there’s one thing XP people agree on, it’s that names actually matter.)

Finally, I think you see a conflict between doctest and BDD. I don’t: doctest is a tool, BDD is an approach, and you could clearly use doctest to help you do BDD. (Of course, you could, instead, use it as after-the-fact unit tests, functional tests, executable documentation, and possibly a dessert/floor wax.)

So the question is, does making the mental translation between what you see and what you interpret it as affect your ability to understand the intent of what is being expressed? I agree with you, I can do the translation in my head quickly also (for the tools I’m used to working with), however I find a natural DSL type syntax to be easier to write (good tool support) and more readable than the equivalent. So I get a better process and a better result.

Having a non-specialist as a target audience makes sense for top level “is this done” and “does it do what I want” type of tests (acceptance/functional/story tests). I don’t think anyone is trying to dumb things down to a point where anyone can read and reason about code. We should strive to make it as simple and clear as possible, but clearly there will always be complexity and a barrier to entry. What we are trying to do is use the same language as what a non-specialist would use, so that they can at least get an idea of what we’re up to (is it what I asked for, etc.) as well as helping us to not have to translate between how we think of things and how they think of things, this is a recipe for disaster. If your code and your “tests” all use the same language as that of the domain, things become a lot simpler, leaving you to worry about other things.

I guess as an example I will switch to the design and verification of digital chips which is the field I am in. We have always had a written design spec that both the verification team and the design team interpret. lately we are using assertion languages such as PSL to accurately and unambiguously define what the spec. says.

The spec may say: “Signal apply_power occurs after the breaks are released”.

Which in PSL becomes: assert always {rose(apply_power)} |-> { ! brake_applied };

Which re-interpreted in English becomes something like: “Ensure that whenever the signal apply_power rises, the signal brake_applied is not high at that same time”

You can formally check the PSL against the design. You can simulate the design and ensure it does not contradict the assertion. Domain specialists can accurately reason about the specs interpretation using PSL. ( And pedants can flag the spelling mistakes :-)

Management has to either learn PSL, get someone they trust who can explain PSL to them, monitor the process of creating and using PSL, or probably do all of the above to be a good manager. The domain specialist works together with management so the process of PSL creation and monitoring ends up as accurate reports and graphs of progress but PSL and other assertion languages are optimised for the domain . It needs to be writeable and maintainable by the domain specialists.

To get back to programming, I think we should be going down a similar route; what is maintainable, precise, and readable for the programmer creating tests will create better software, and, I think, prove easier for the manager in the long run.

In reply to Paddy3118, I wasn’t intending to imply by my IDE remark that the code becomes noise. One of the things Instinct (not BDD per se) aims to get rid of is noisy infrastructure. The point I was trying to make was that it is more verbose, but by using tools, the verboseness doesn’t add any extra cost in creation, and the benefits you get from the readability are worth it regardless (of whether it’s harder or easier to create).

In fact the Instinct expectation API is type-safe, making it easier to write these more verbose statements (the IDE leads you to the correct methods via the type) than their less verbose equivalents.

As always, there’s a tradeoff between verboseness and terseness. BDD aims at producing spec code that is readable, and in an argument between readability and terseness I’d take readability every time.

I don’t believe the Instinct expectation API is just noise, it’s been specifically crafted to minimise noise (in the eyes of its developers admittedly). The API aims at readability and accurately reflecting the intent of the author. So for example there are several ways to say a collection/list/array/string has a certain length, you pick the one that most accurately reflects your intent.

Picking on doctest again, the example given:

>>> len(stack)
0

Is not as readable to me as the Instinct (or RSpec) equivalent. Perhaps that’s because I haven’t used python for 7 years or so, or perhaps because I like the ability to read the expectation aloud and have it make sense. “len stack 0″ doesn’t read as well to me as “expect that stack is of size 0″ or “stack should be empty” (as it would be in RSpec) though they both assert the same thing.

In the end it comes down to the audience and how much implicit conversion you do in your head between what’s written and what it means.

In BDD, you embed this readable text in the names of your functions and classes, and in RSpec there are opportunities to use strings with arbitrary text (that look a little like docstrings). In no case does the computer interpret these strings, or confirm the programmer actually lived up to what they said they’d do. The only thing really consumable by non-programmers is the non-code text. So I see two things that we are attempting to accomplish: (a) provide a good space for narrative text about requirements, and (b) attach that to testing code so that the programmer can live up to those specifications, keep them in sync, and for the requirements document to be a useful and functional tool to help the programmer.

Doctests (at least the tests that don’t go in docstrings) really are editable by non-programmers — from their perspective the format is pretty trivial. I don’t think BDD systems like RSpec really are editable by non-programmers, as the text is embedded in the syntax of the language (which is pretty fragile); doctest embeds the code in the text.

FITnesse is a more business-oriented (and less programmer-oriented) approach that is very close to doctest. I personally think its infrastructure is too complex and there’s too much indirection. The amount of test data that can usefully go in a table is fairly low in my experience. It’s high enough that I think it can still be useful, but not high enough to build an entire test framework on it. If I was writing doctests where tabular data was appropriate, I might just write a table reader that can be used by the doctest, driving the test off the data while keeping the all important code close to the functional specification as well. In Python this would unfortunately expose a flaw in doctest: the framework doesn’t expose enough to the tests. I’d like to provide hints about failures and what table data I was using, but there’s no way to do that. But doctest is old, and it really needs some more love. I would hope that new implementations would build in a few more features.

 >>> stack.isEmpty()
 true

It would be:

 >>> len(stack)
 0

I find it difficult to believe Kevin Teague’s point (8.); that Business types who might find a doctest hard to read, might find a BDD test easier to understand.

Tom Adams (4.) Makes the point that “you don’t type much of this stuff, an IDE will auto-complete the matchers”. That point never sits well with me. If the computer can do this stuff then we should move towards not having to see it at all - its just noise. I moved from C because I was wasting too much time chasing pointers, and along came other languages that handled that for me.

- Paddy.

Oh, another doctest introductory link