An automated Behavioral Driven Development(BDD) testing framework

Java, Spring Boot, Cucumber, Jenkins, Selenium


JBehave is a web plugin that enables use case testing in accord with the behavior driven development (BDD philosophy). The need for behavior driven development:

In traditional test driven development the developer writes test cases based on the product requirements of the product owner as described by the business analyst. However, this approach disconnects the business analysts and the product team from the development process.

It relies on the capability of the developer to model test cases which may or may not incorporate the expected behavior of the product. In BDD however the business analysts describes the requirement in the form of “stories” Where in each story can contain independent “Features” or “Scenarios”.

JBehave uses the Gherkin language to enable users to write a story containing one or more than one scenarios.

Cucumber and Gherkin

Cucumber executes your .feature files, and those files contain executable specifications written in a language called Gherkin.

Gherkin is plain-text English (or one of 60+ other languages) with a little extra structure. Gherkin is designed to be easy to learn by non-programmers, yet structured enough to allow concise description of examples to illustrate business rules in most real-world domains.

In Gherkin, each line that isn't blank has to start with a Gherkin keyword, followed by any text you like. The main keywords are: Feature
Given, When, Then, And, But (Steps)
Scenario Outline

Read up more on Gherkin syntax and Cucumber here.

Technology Stack Used

1.Spring boot
2. Maven
3. Jenkins
4. Cucumber
5. SVN(Subversion) API from SVNKit
6. Selenium for automated testing

Capabilties of JBehave:

1. Display all scenarios in a feature file given the story ID of the feature file.
2. Enable the user to add a Scenario (in Gherkin) to a feature file.
3. Save the scenario to a SVN repository.
4. Edit a particular scenario and save it to the SVN repository.
5. Add a tag to a scenario.
6. Edit the tag of a scenario (Check for tag name limitations as well as uniqueness).
7. Sanity checking of the Gherkin scenario that has been entered according to the Gherkin syntax before saving.
8. Enable execution of a scenario using Jenkins for build automation. As an example a web application was tested using the Selenium framework.
9. Capture test results of the scenario after execution and show Cucumber test report.
10. Audit all changes under file level audit information.
11. Audit scenario level update information
12. Generate the Cucumber step definition file from the feature file.

Demonstration Video


Login Page

Using the Gherkin 3 parser.

The Gherkin 3 parser allows parsing of the feature file written in Gherkin as a GherkinDocument object which makes accessing individual features, scenarios and sceanrio outlines much easier.

The class diagram which explains all possible nodes the parser can produce.

Optimizing the process of fetching feature files from the SVN repository.

Each feature file has a unique "Story ID" associated with it and hence when the user specifies which feature file is to be read, it would be expensive to read through each feature file in the SVN repository.

To streamline this process the following approach was used:

a. Checking out the repository into a local working copy.

b. If an entry for the specified Story ID exists in a local map which stores StoryID-filename associations, then the found file is parsed to check if it still holds the required feature.(There is a high chance that the StoryID once assigned to a file would not change and hence this step makes sense.)

c. In case a match wasn't found in step 'b', we use the "Unix4j" library in Java to grep through all files in the local working copy.

d. In case a match was found in step 'c' we parse the files which matched to check for the required feature and add an entry for the file that contains it, in the map mentioned in step b.

Loading scenarios from the specified feature file

The scenarios and the file-level audit information are displayed

Adding a new scenario to the feature file

A client side Gherkin syntax validator alerts abuout any syntax errors in the scenario.

Saving the added scenario to the feature file

The save process is time consuming as the following sub-processes occur during each save:
a. The feature file is checked out into a local repository from the source SVN repository.
b. The new scenario is appended to the locally checked out file.
c. The local file is then commited to the SVN repository.

The added scenario is saved in the feature file and all scenarios are re-listed.

The step definition file generated for the feature file by Cucumber which is a boiler plate code where the developer can plug in testing code for the function defined for each scenario.

The scenario can be edited by double-clicking on the card. Clicking on save will initiate the save process as earlier.

Edits made to multiple scenarios can be saved together at once using the "Save All" menu-level button.

An existing scenario tag can be edited however the restrictions of tag length and uniqueness will be validated before a save.

The tag length validation.

Every scenario level card stores scenario-level audit information.

Execution of a scenario is triggered using the execute button.

The execution of a scenario occurs with the help of the steps shown above.

A Jenkins build is triggered for the system under test(a remote host where the web applications to be tested via Selenium are present).

System Under Test(SUT) screen 1 : Selenium script checks a login page(SUT) with empty(invalid) credentials.

System Under Test(SUT) screen 2 : Selenium scripts checks for the validity of an email address.

System Under Test(SUT) screen 3 : Selenium scripts checks for a HTML heading with "Hello World" as the text.

The report generated by Cucumber after all tests have been performed which gives us the time taken to perform the tests as well as the status of the tests.

The test results and the audit information is updated on the respective scenario card.