Data Driven Test Automation Framework

Data Driven Test Automation Software Testing

Page 1 5/26/2011
 Data Driven Test Automation Framework

Table of Contents

1. INTRODUCTION.
2. DATA DRIVEN TEST AUTOMATION APPROACH
3. AUTOMATED TESTING
4. DATA DRIVEN TESTING
5. TEST AUTOMATION FRAMEWORK
6. SUMMARY

Introduction
The tools used for creating applications with graphical user interface (GUI) have improved
dramatically over the past several years and this helps programmers to increase their productivity
and to develop software in very short time. This improvement requires testing of more code in
less time and an increasing productivity of the testers: .Because of the varied possibilities for user
interaction and the number of control elements (buttons, pull-down menus, toolbars, etc.)
available with GUI’s, their testing is extremely time-consuming and costly. Manual testing of GUI’s
is labor-intensive, frequently monotonous, and not well liked by software engineers or software
testers. [7].

With the progress of a project, the number of features added to a product line will increase and
the testing has to ensure that the new features meet the defined requirements by adding newer
test scenarios. At the same time, regression testing has to be increased to ensure that existing
features and functionality that has not been modified has not been effected by introduction of new
features and functionality. The best way to do this is to automate as much as possible. The
primary objective of Test automation is to reduce the testing cycle time and to ensure high
software quality delivered to the customer. Through test automation error will be avoided that
humans make after multiple repetitions. The test programs can work at any time independently;
without human intervention and can record the test results more accurately.

The design and implementation of the test automation framework follows the main principles of
the Data Driven Test Automation approach: lower maintenance, increased reusability, higher
coverage, parallel execution and uniform test environment. In conclusion the main classes,
modules and test drivers are implemented using testing software and the sample test cases can
be ran and evaluated. The test data database is designed and implemented according to the
Data Driven Test Automation concept.

1. Data Driven Test Automation Approach

The new business requirements and needs to accelerate the software development and reduce
the time to market in the fast changing business environment demand new testing methods.
Automated testing is essential for performing thorough and fast testing, even thought the manual
testing can not be completely replaced. Test automation has its own advantages and
disadvantages, but also involves a lot of challenges.

1.1. Automated Testing

Automated testing is here basically an automation of manual testing procedures. It is about

developing test automation programs to test other software and replace or supplement manual

Page 2 5/26/2011
 Data Driven Test Automation Framework

testing. A lot of effort and time for testing of a system is saved using test automation. The tests
can be run fast and can be repeated over and over again without increasing the testing costs.
Using test automation, the quality of the product can be increased and the software can be
delivered in shorter time to the market.

Automated testing saves costs, drudgery and repetition associated with manual testing tests. It
makes the tests more consistent, repeatable and ensures uniformity in the testing process each
time the test is executed. The test scripts are written using some of the test automation tools and
execute keyboard and mouse actions and background processing. They basically drive the
application under testing and verify its behavior and responses.

In order to start developing automated testing, it is required that a manual testing process exists
and there are well defined detailed test cases including test data and expected results.
Before starting working on the test automation and also during the automation process, there are
many factors to be considered.

It is important to estimate the effort and cost required by every automation activity. The effort
being saved through automation needs to be compared with the effort needed to achieve this.
One of the main objectives of automation is to reduce and avoid similar types of effort again and
again. That is why the automation is recommended only if there are many software releases.
Automated testing assumes certain stability in the product or system and a certain maturity is
required before starting test automation. The most important factor driving the automation project
should be the productivity of the automation. The productivity is the effort saved due to
automation divided by the effort required to make the test automation. The value of the
productivity should be as high as possible in order to get the benefits from the automation.

The applications Automating tests for a graphical user interface presents significant difficulties not
found in character based interfaces, much less command line interfaces or programming
interfaces (APIs). Graphical user interfaces tend to be made of complex components and tend to
be constantly redesigned during the development process. Significant successes have been
made in delivering test tools that are able to identify and manipulate graphical user interfaces.
One of these tools is for example Compuware.s TestPartner and it is used for this project.
TestPartner provides a range of benefits for Windows-application and Web-application
development organizations:

• Shorter development and testing cycles. Because Test Partner can run test suites
unattended or at night, it can accelerate the development and testing of applications,
helping the applications to reach the market faster.

• Consistency. Because automated testing runs tests the same way every time, Test-
Partner produces consistent, reproducible results.

• Increased productivity. Because Test-Partner can run test suites without manual
intervention, programmers and testing staff are freed for other tasks.

• Rapid test development. The user can create useful tests quickly and easily just by
navigating through the application. It can be also set up templates to provide a standard
framework for the test scripts.

2.2. Data Driven Testing

Most of the testing tools support the record/play method but it is often problematic and costly for
all the methods over a long period of time. Basically it is simply recording the actions and playing
back the recorded scripts. The problem that always crops up are that the layouts are changed,
invalidating the screen captures and then the interface controls change making playback fail. Now
the scripts must be re-recorded from scratch. The scripts contain hard-coded values which must

Page 3 5/26/2011
 Data Driven Test Automation Framework

be changed if anything changes in the application and their maintenance is very complex and
expensive.

Even if the application has not changed, the scripts often fail on replay because of events that did
not occur during the recording. The test must be rerecorded if an error occurs while entering the
data or if the application changes. The real value in the automated testing only comes when data
driven automated tests are used. Data driven testing means that the automated tests read the
test data and expected results from an external source, such as an Excel sheet or data base,
7rather than having the values hard coded into the scripts. The test scripts take parameters and
these parameters define the way in which the test functions. This allows the user to test a variety
of scenarios with the same automated test scripts simply by changing the data. An additional
coding is not required. The test framework enables the generation of new tests scenarios only by
entering data in the data tables. Even non Software developers can enter new data and manage
the test workflows.

The Data Driven Methodology used in this project is Functional Decomposition [4] and its main
concept is to reduce to their most fundamental tasks, and write user defined functions, scripts and
sub-routine. They perform these tasks independently of one another. The fundamental areas
include navigation, specific functions, data verification, return navigation. In order to accomplish
this, it is necessary to separate Data from Function. This allows an automated test script to be
written for a business function, using data-file to provide the both the input and the expected-
results verification.

3. Test Automation Framework

The most important assets supported by TestPartner are driver scripts, modules and classes. The
scripts are the executable programs and depending on the current workflow different test
scenarios are implemented and executed. They are the engine of the test and drive it through the
steps of the test case. The common functions are implemented in modules and they can be
called from the driver scripts by including the module.

Summary
The main approach is Data-Driven Test Automation and the goal is to achieve a lower
maintenance, an increased reusability, a higher coverage and a uniform test environment. The
application is still in development and the existing test automation framework enables easier
maintenance and reusability by new features and functionalities of the application. Only the
specific functional scripts need to be updated.

A manual testing is replaces through test scripts and the regression testing can be run without
human intervention. At the same time following the concept of data driven testing, a Record / Play
technique is avoided. If different test data needs to be used, then all we have to do is update the
data table, and not the actual scripts. The input data and the expected results are stored as easily
maintainable database. The actual result of a test execution as well as test steps in details are
written to a log file. Tests can be evaluated by analyzing the logs. Utilizing a modular design, and
using database to both input and verify data, reduces redundancy and duplication of effort in
creating automated test scripts. The scripts are written to perform and test individual business
functions and can be easily combined in order to accommodate complex test scenarios.

The advantages of the Data-Driven approach can be proofed in the following example. If we want
to run the same test with 50 different data inputs in order to verify all of the possible conditions,
using the Record/Play method, we would need 50 scripts, each containing hard-coded data, that
would have to be maintained. Using the data driven testing, we need only one script that contains
all 50 workflow ids and calls the same subroutine with the different ids. We can re-use the
subroutine script as-is without any modifications at all.

Page 4 5/26/2011