Escolar Documentos
Profissional Documentos
Cultura Documentos
com
Business
assurance
$ Testing
AuthOr:
Arish Arbab is a Software Engineer at HCL
Singapore Pte Limited, having expertize on Agile
GUI/API Automation methodologies.
WHITEPAPER
April 2015
April 2015
TABLE OF CONTENTS
INTRODUCTION 3
PROBLEM FACED
SOLUTION APPROACH
BENEFITS 6
IMPROVEMENTS 7
APPLICABILITY TO OTHER PROJECTS
UPCOMING FEATURES
REFERENCES 8
APPRECIATIONS RECEIVED
ABOUT HCL
2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED.
April 2015
INTRODUCTION
Behavioral Driven Development (BDD) testing uses natural language to describe
the desired behavior of the system that can be understood by the developer, tester
and the customer. It is a synthesis and refinement of practices stemming from TDD
and ATDD. It describes behaviors in a single notation that is directly accessible to
domain experts, testers and developers for improving communication. It focuses on
implementing only those behaviors, which contribute most directly to the business
outcomes for optimizing the scenarios.
PROBLEM FACED
By using the traditional automation approach, and if given a summary report of
automated tests to the business analysts, then it is guaranteed that the report would be
met with a blank expression. This makes it tricky to prove that the tests are correct
do they match the requirement and, if this changes, what tests need to change to
reflect this? The whole idea behind BDD is to write tests in plain English, describing
the behaviors of the thing that you are testing. The main advantage of this is that
the tests reflect the user stories and the business requirements of the application.
Therefore, the results generated can be read by a non-technical person, e.g., a project
sponsor, a domain expert, a business analyst, and the tests can be validated against the
requirements for better operational efficiency.
To achieve the BDD approach, there is an open source toolCucumberthat
parses the scenarios/tests that are written in Gherkin language (GWT). This tool also
interacts with the API/Junit/Source code of an application, using its step definitions
test code. The best option to get the business intent into our tests is to have clear
deliverables and acceptance criteria in the user stories. These can be then translated
into feature tests containing scenarios written in Gherkin Language.
BDD is the outside-in software development methodology, which has changed
the testers role dramatically in the recent years and bridges the communication gap
between business and technology. This innovative approach has the potential to shape
a new organization standard with new SDET roles.
Problems faced while implementing the new framework:
yy Justifying the coverage done by this framework becomes difficult as the complexity
of the application code increases
yy Exposing APIs/Source Code was difficult as developers disagreed
yy Converting the software requirement into GWT format at the later stages of the
software development utilized more efforts
yy Requiring testers with good technical/coding knowledge
yy Extending the Junit by using the mocking techniques or spring-based test was a
hindrance when the complexity of the code increased
yy Convincing the BTMs to review the scenarios on every day was difficult
2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED.
April 2015
SOLUTION APPROACH
Traditionally, software requirement was written in simple English sentences, but to
convert those requirements into GWT format required an extra effort that should be
properly handled. As GWT approach results in better quality of product, requirement
should be written in GWT format by the business analyst at the early stages of software
development rather than writing the requirement in domain-specific language. Set of
scenarios should depict a particular user story. Implementing this approach in a later
stage may increase the cost of a project.
Example of writing the scenarios in the abovementioned format would be:
April 2015
App Service
Methods
Model
Junit Classes
Service Methods
Cucumber Main(Driver)
Feature File(Scenarios)
Data(.xlsx)
Database
Test Result
HTML Reports, Screen shots, logs of failed
scenarios etc.
Flow of the BDD framework using Cucumber is as followsyy The test data, which is to be used in the scenarios, is stored in Excel sheet or MySql
Database.
yy The test data is then passed to feature file, containing scenarios written in GWT
format. Scenarios are denoted by certain tags.
yy Driver class decides which scenario among the entire features files should be executed
by proving a proper tag value. Before running a particular feature, dependency used
in application is injected to avoid mocking unit testing framework.
yy Appropriate matching method defined in the Cucumber step definition file is
called.
yy From the step definition, appropriate unit test method of the parameterized Junit
class is called and, thus, test data is passed into it.
yy Unit test method, which, in turn, calls the real service method by passing the test
data to it.
yy Response is then returned from the service method to unit test method, which, in
turn, passes the information to Cucumber step definition file.
yy Expected response is validated against the returned (actual) response in the step
definition.
yy Logs are created upon failure or success and an auto HTML report is created,
depicting the passed and failed scenarios in appropriate colors.
2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED.
April 2015
Now, if we have the API available, then, instead of calling the Junit test method, the
API method is called directly and rest of the flow remains the same.
Then the question arises, Whether or not we are covering all the areas of an application
with our test as the scenarios written are very behavioral specific? Here the need
for the code coverage arises. JaCoCo is one of the tools, which can be integrated
with Cucumber Test, and the code coverage report generated can be easily read by
customers/developers/business analysts and testers. The traditional way of testing may
work out with testers, having less technical knowledge. Nevertheless, as we can clearly
point out by seeing the above solutions and exposure of code to tester. The need for a
tester with good coding skills is prerequisite to implement this approach successfully.
JaCoCo report
Cucumber test execution can be integrated into CI (Jenkins) server and can be
executed depending upon the need. Now, when we have everything in place
scenarios, test code and test resultsit becomes difficult to convince the BTMs to
review the scenarios on a daily basis. There is another open-source tool available that
overcomes these difficulties: Pickles, open-source living document generator. This tool
generates the document consisting of the scenarios, which can be easily reviewed by
the customer as it supports advance search capabilities.
BENEFITS
Key Benefits
yy Cucumber and JaCoCo are open-source tools, which provide equal benefits that
other paid tools provide.
yy Around 80% reduction in the test execution time as compared to manual testing.
yy Cucumber can be integrated with existing tools like Selenium, TestComplete and
QTP.
yy Quality of an application increased as the defect is found at an early stage of
development.
yy Cucumber can be used to test REST and SOAP Web services, thereby, easily
adjusting in traditional way of API automation.
2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED.
April 2015
yy Unlike other API testing tools, it produces more detailed, easy-to-understand test
execution report for client, developer, BA and tester regardless of their technical
knowledge.
yy One of the eminent features of Cucumber is that the separation on three levels of
feature, steps and world makes it easier to reuse certain parts of the code.
yy Scenarios can be written in different languages, including French, German, Chinese
and Hindi.
yy Cucumber with Code Coverage tool (JaCoCo) provides efficient and advanced way
of testing applications.
Available for nine programming languages, Cucumber is an open-source tool for BDD.
The tool is available in languages such as Ruby, JVM-based languages, JavaScript and
.NET (.NET languages are supported through the SpecFlow tool.).
IMPROVEMENTS
The key learning is as follows:
yy Team gained knowledge on API automation using smart BDD tools (Cucumber
and JaCoCo).
yy Team was able to understand the requirements better due to the constant interaction
with BAs, and the interaction with developers helped the team increase its technical
knowledge.
yy Detailed test results report was generated in the UI automation by integrating
Selenium with Cucumber.
yy Exposure to continuous integration servers increased, which helped in scheduling
the overnight batch execution.
yy Team collaboration and communication techniquesused for involving
customerssuch as Pair Programming and Code Reviews became frequent.
UPCOMING FEATURES
yy Collaborate across the globe with a real-time shared editor
yy Access the latest version of your executable specifications anytime
2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED.
April 2015
REFERENCES
http://en.wikipedia.org/wiki/Cucumber_(software)
http://cukes.info/
http://www.ibm.com/developerworks/library/a-automating-ria/
https://github.com/cucumber/cucumber/wiki/Cucumber-JVM
http://en.wikipedia.org/wiki/Behavior-driven_development
http://guide.agilealliance.org/guide/bdd.html
APPRECIATIONS RECEIVED
Proof of Concept was accepted and greatly appreciated by the client. Appreciation for
this practice was also provided by the head and the project manager.
ABBREVIATIONS
Abbreviation
Expansion
TDD
ATDD
JaCoCo
UI
SDET
GWT
BA
HTML
CI
API
JVM
BTM
2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED.
ABOUT HCL
About HCL Technologies
HCL Technologies is a leading global IT services company working with clients in the
areas that impact and redefine the core of their businesses. Since its emergence on the
global landscape, and after its IPO in 1999, HCL has focused on transformational
outsourcing, underlined by innovation and value creation, offering an integrated
portfolio of services including software-led IT solutions, remote infrastructure
management, engineering and R&D services and business services. HCL leverages
its extensive global offshore infrastructure and network of offices in 31 countries to
provide holistic, multi-service delivery in key industry verticals including Financial
Services, Manufacturing, Consumer Services, Public Services and Healthcare &
Life sciences. HCL takes pride in its philosophy of Employees First, Customers
Second which empowers its 100,240 transformers to create real value for customers.
HCL Technologies, along with its subsidiaries, had consolidated revenues of
US$ 5.7 billion, for the Financial Year ended as on 31st December 2014 (on LTM
basis). For more information, please visit www.hcltech.com
Hello there! I am an Ideapreneur. I believe that sustainable business outcomes are driven by relationships nurtured through
values like trust, transparency and flexibility. I respect the contract, but believe in going beyond through collaboration,
applied innovation and new generation partnership models that put your interest above everything else. Right now 105,000
Ideapreneurs are in a Relationship Beyond the Contract with 500 customers in 31 countries. How can I help you?