Belgian Testing Days 2013 - Behavior Driven Testing with Cucumber demystified

by admin Email

This is the talk I gave on the Belgian Testing Days, I added the slide deck as well as some extra information of things that were discussed during the talk and the Q&A afterwards.


Behavior Driven Testing (BDT) is the lesser known companion of Behavior Driven Development (BDD). However, BDT can be used without BDD. When looking at the V-Model BDT can be used at the requirement definition level and the functional testing level. In Agile BDT is often used in the form of user stories using the 'given-when-then' format.

These can be used not only to define the behavior of an application but also as input for automated test tools using the Cucumber framework.

In order to create and maintain these user stories in a structured way, a Domain Specific Language (DSL) must be defined. There are some pitfalls when creating and maintaining a DSL both for requirement definition and as input for the automated test process.

These pitfalls will be listed and several solutions will be shown during the presentation. Steven also will highlight the effort required to start with a combination of BDT and an automated as well as the expected knowledge.

With some examples the return on investment can easily be explained, these examples can be translated to other companies without too much difficulty.

Slide deck

The slide deck can be seen here.



I cannot stress enough the importance of the ?should? in the then steps, this little word will make sure the author of the user story (being a requirement writer or tester) challenges the premise of the test while writing it. The quality improvement gained when doing this is often underestimated. The quality of the user stories will be reflected in the quality of the automated tests that follow and will impact the quality of the software product.


DSL definition and story robustness

The same goes for the DSL, a healthy amount of nitpicking should be done when creating sentences for the user stories, they should be descriptive without being too long and they should be as much as possible technology independent.

Example for typing text in a textbox on a website:

When I type ?xyz? in the ?address_field? textbox

This is a typical sentence one would end up with when creating steps for the first time.

It is too technical because the textbox could change in a text area, in that case the step would not be accurate anymore. We can improve this to:

When I type ?xyz? in ?address_field?

This is still too dependent on technical knowledge since the address_field is the name of the textbox as programmed in the HTML, this name is subject to change and then all your stories would need updating. We can improve this by making the address field abstract in the story and programming the correct identifier for said field in the HTML in our step.rb file where all the steps are defined (i.e. programmed):

When I type ?xyz? in the address field

Still not perfect, the mention of address as a field is still a little too technical, the user we are describing the behavior off is not interested in that, if an end user is asked what he is doing he would likely say:

When I type ?xyz? as the address

This is where the nitpicking can start, after all there is no guarantee the end user is typing anything, perhaps he is copying text using the mouse buttons. It is better to make the whole description of the behavior even more abstract by noting the step as:

When I enter ?xyz? as the address

This last step is describing the behavior perfectly; it also makes your story more robust, if the address field becomes a text area the step is till accurate, of the field changes from name, it is still valid etc.

Writing testable code

The whole TDD and BDD practices lean of course on the testability of the code created. In practice this means that there must be an easy way to identify the controls that the end user will be using. Giving your GUI controls some kind of identifier that does not change (this means it is only used for testing and only to identify the control) will make your step definition code a LOT more robust. A good example can be found when testing HTML using Selenium web driver from Cucumber, you need to add the tag ID for each control that is needed for testing. The same is true when programming in .net and creating a desktop application using WPF, you will need to make sure the automation ID is filled for each control.



Trackback address for this post

Trackback URL (right click and copy shortcut/link location)

Feedback awaiting moderation

This post has 3595 feedbacks awaiting moderation...