On an internal ThoughtWorks mailing list, somebody recently asked if anybody had tried using the RSpec stories framework on a project yet. I considered using it for a Selenium-based acceptance test suite, but decided not to. Here is my response to the mailing list.
I considered switching to the RSpec stories framework at [client name removed], but I have one big lingering issue with doing it.
How well do acceptance criteria on a story correlate to focused selenium-based acceptance tests?
Often I need several selenium tests to cover one acceptance criteria on a story. Other times, a couple acceptance criteria can be covered in a single test, although it is certainly not as common as having multiple tests for a single criteria.
There's a couple scenarios there with using the story runner.
(1) Automated acceptance tests that correspond to the exact acceptance criteria from a story.
(2) Automated acceptance tests that don't correspond exactly to the acceptance criteria from the story, but are more suited to automated testing, written in a given/when/then style.
Regarding (2), I mostly consider a test name/description a comment. Given/when/then can be inferred from the test implementation. Given = setup, when = action, then = assertions. So as far as leaving a comment on the test to perhaps provide a little more insight into intent, describe/it seems sufficient.
Regarding (1), I like the idea of having traceability from a test to an acceptance criteria. But I feel acceptance criteria on a story card often do not correspond well with the tests that need to be written to verify the behavior. That may reflect on the quality of the acceptance criteria, but I'm not convinced they should be the same.
Outside of this issue around acceptance criteria, I personally don't like how much the using the story framework fragments the test implementation. I also question the reusability of a template "given", "when", or "then."
I would like to try using the story runner, but it doesn't seem any more beneficial than using describe/it.