AutomatedSoftwareTestingMagazine September2011

An AUTOMATED TESTING INSTITUTE Publication - www.automatedtestinginstitute.
com
AutomAted ....... S t
oftware
September 2011 $8.95
eSting
MAGAZINE
Tools, services and daTa: AutomAtion in ParTs

ADvAnceD insight From conFerence sPeAkers
Verify/ATI 2011 Prequel
removing roADblocks by virtuAlizing services
Virtualization of Services
Selenium and TestNG

DAtA Driven test AutomAtion
Wins AnD loses: Why Some of the BrighteSt open Source toolS loSt their Shine
Verify/ATI Conference 2011

The Test Conference Youve Been Waiting For
EntEr for ComplimEntarty rEgistration @ omplimEntArty EgistrAtion

http://www.VErifyAtiComp.AutomAtEdtEstinginstitutE.Com/
Register Now!
September 26 - 28, 2011 Crowne Plaza Hotel Arlington, VA
Venture Beyond the Abstract

www.verifyati.com
AutomAted S t
oftware eSting
September 2011, Volume 3, Issue 3
Contents
toolS, ServiceS and data
We often speak of test automation as though it is this single, rigid element through which testing is mechanically conducted. Not so. Successful test automation is normally a product of several different working parts. The challenge for test automators is to understand some of the available working parts, how they function and when to employ them. This issue is dedicated to the automated testings working parts.
featureS
How Service virtualization removeS teSting roadblockS 12

This article reveals how components created via service virtualization provide a new way for developers & testers to exercise their applications in incomplete, constantly evolving, and/or difficult-to-access environments. By Wayne Ariola
make your own data-driven arcHitecture uSing Selenium & teStng! 18

This article offers a detailed approach, complete with sample code, for using Selenium and TestNG for separating test data from test scripts.
By Jailton Alkimin Louzada
automation tecHnical expertiSe, management pHiloSopHy & tHe Future 26 ,

This offer provides insight on various automation topics from many of the speakers that will be at the Verify/ATI 2011 Conference from September 26-28.
By Various
columnS & departmentS

editorial
tools, Services & data: automation in parts Breaking automation into parts. Learn about AST authors and upcoming events.
Read featured microblog posts from the web.
go on a retweet 38
autHorS and eventS open Sourcery 8
i blog to u 34
Read featured blog posts from the web.
Hot topicS in automation
open Source wins, loses & puns Find out why some tools lost their popularity.
Joining Forces is Hot! Exploring the trend of merging tools and tool organizations
40
The Automated Software Testing (AST) Magazine is an Automated Testing Institute (ATI) publication. For more information regarding the magazine visit http://www.astmagazine.automatedtestinginstitute.com
september 2011
www.automatedtestinginstitute.com
Automated software testing magazine
editorial
tools, Services & data: automation in parts

by Dion Johnson
We often speak of test automation as though it is this single, rigid entity through which testing is mechanically conducted. Not so. Successful test automation is normally a product of several different working parts. The parts themselves are not of one singular type but are diverse in nature. The challenge for test automators is to understand some of the available working parts, how they function and when to employ them. One part to consider encompasses the various tools that may be employed as part of an automated test framework. Tools are typically considered a very important part of an automated test implementation, and rightly so. It is difficult to imagine how test automation can be conducted without some sort of programmable automated utility.
Another important part of test automation is the data. Data is typically critical to test automation implementations that are geared towards the verification of some application under test. Therefore,
incomplete, constantly evolving, and/or difficult-to-access environments. In our second feature, Jailton Alkimin Louzada discusses both tools and data
Successful test automation is normally a product of several different working parts.

Although tools are important, they are not irreplaceable. For this reason, we must stay abreast of what is going on with the tool industry. This means understanding tool trends, and tool organizational changes that may affect the tool choices that we have made or will make for our frameworks. In addition, we must appropriately map out the working parts or components that we choose to develop using the tools that we employ. These workings parts could be test scripts or components used by test scripts. 4 the data must be dealt with in an effective manner. This issue of the Automated Software Testing Magazine is dedicated to the automated testings working parts. In our first feature entitled, How Service Virtualization Removes Testing Roadblocks, Wayne Ariola discusses a different type of virtualization than the testing masses may be used to. In this article, Ariola reveals how components created via service virtualization provide a new way for developers and testers to exercise their applications in www.automatedtestinginstitute.com with an article entitled, Make your own Data-driven Architecture Using Selenium & TestNG! Louzada offers an approach, complete with sample code, for using Selenium and TestNG for separating test data from test scripts. Our third feature includes insights on various automation topics from many of the speakers who will be at the Verify/ ATI 2011 Conference from September 26-28 (register now at www.verifyati. com!) In this article, you are able to read content from speakers before you hear from them live at the event. september 2011
Authors and events

Whos In This Issue?
Jailton alkimin louzada lives in Goiania, Brazil. He currently works as a Test Analyst on Politec Global IT Services (www. politec.com.br). Jailton has 3 years commercial IT experience in the public and private software industry, mainly in ERP software. His experience covers Java and C development, requirements analysis, system architecture, test management and quality analysis. Jailton holds the ISTQB/BSTQB Tester-Foundation Level (CTFL) certification. You can find out more about Jailton A. Louzada through his personal website http:// www.jailtonalkiminlouzada.com. wayne ariola leads the development and execution of Parasofts long-term strategy. He leverages customer input and fosters partnerships with industry leaders to ensure that Parasoft solutions continuously evolve to support the everchanging complexities of real-world business processes and systems. Ariola has contributed to the design of core Parasoft technologies and has been awarded several patents for his inventions. A recognized leader on topics such as SOA and RIA quality, quality policy governance, and application security, Ariola brings more than 15 years strategic consulting experience within the technology and software development industries. He has a BA from the University of California at Santa Barbara and a MBA from Indiana University. verify/ati 2011 Speakers are internally known, skilled practitioners that have chosen to all converge on the Washington, DC metropolitan area for a testing and test automation event organized by ATI along with Verify and conducted from September 26-28. This event is a conference that features professionals such as Joe Jarzombek of the Department of Homeland Security (DHS), Linda Hayes of Worksoft, author and testing expert Dorothy Graham, and many more. This issue specifically features those speakers that have contributed to issues of AST, along with some of the concepts theyve shared in those contributions.
6 Automated software testing magazine
AutomAted S t
oftware eSting
Managing Editor Dion Johnson Contributing Editor Donna W. Vance Director of Marketing and Events Christine Johnson
A PUBLICATION OF THE AUTOMATED TESTING INSTITUTE
CONTACT US AST Magazine astmagazine@automatedtestinginstitute.com ATI Online Reference contact@automatedtestinginstitute.com
Ati and Partner events

September 21, 2011 Verify/ATI Complimentary Registration Entry Period Ends September 26-28, 2011 Verify/ATI 2011 Conference
www.verifyati.com
http://www.verifyaticomp.automatedtestinginstitute.com
September 27, 2011 3rd Annual ATI Automation Honors Awards Live Ceremony
www.verifyati.com
september 2011
Are You Contributing Content Yet?

The Automated Testing Institute relies heavily on the automated testing community in order to deliver up-to-date and relevant content. Thats why weve made it even easier for you to contribute content directly to the ATI Online Reference! Register and let your voice be heard today!
As a registered user you can submit content directly to the site, providing you with content control and the ability to network with like minded individuals.
Community Comments Box
>> Community Comments Box - this comments box, available on the home page of the site, provides an opportunity for users to post micro comments in real time. >> AnnounCements & Blog Posts - if you have interesting tool announcements, or you have a concept that youd like to blog about, submit a post directly to the Ati online reference today. At Ati, you have a community of individuals that would love to hear what you have to say. Your site profile will include a list of your submitted articles. >> AutomAtion events - Do you know about a cool automated testing meetup, webinar or conference? let the rest of us know about it by posting it on the Ati site. Add the date, time and venue so people will know where to go and when to be there.
Announcements & Blog Posts
Automation Events
learn more today at http//www.about.automatedtestinginstitute.com
Open Sourcery
open Source wins, loses & puns

Why Some of the Brightest Open Source Tools Lose Their Shine
eligible version for last year was 0.9.8 which was released in September 2009. While cfix and Google C++ Testing Framework made it back into the running this year, Check hasnt had an updated release, which keeps it out of the running for now. Well be checking for Check in the future, though. csUnit (http://www.csunit.org/) csUnit, finalist in last years Best Open Source Unit Automated Test Tool .NET category, is a free and open source unit testing tool for the .NET Framework. Like Check, csUnit was in a category last year with two tools that made it into this years list of finalist: NUnit and xUnit. net. csUnits 2.6 version released in March 2009, however, did not receive an update, so for now, it has to be content with its 2010 nomination. As the Automated Testing Institute gets ready to crown the winners of the 3rd Annual ATI Automation Honors, we take a moment to look at some of the finalists from last year that didnt make the finals this year. Many of the 2nd Annual ATI Automation Honors finalists but rules are rules! This year, however, instead of simply excluding these tools and moving on, we decided to shine a pun-riddled spotlight on some of those in the open source category that reached the pinnacle of glory last year, but failed to meet the constraints this year. Canoo WebTest (http://webtest.canoo.
com/webtest/manual/WebTestHome.html)
Canoo WebTest is a free, open source tool that boasts of being unrestricted by server side technology. Regardless of whether Servlets, JSP, ASP, CGI, or PHP is used, Canoo can test it as long as it
...but it appears that the fun may now be over. Not only because the tool didnt make it into this years awards, but because it seems as though it is now defunct.
didnt make the 2011-2012 awardee list because they didnt have eligible releases. This is nothing new. Every year there are some tools that dont make the cut due to the fact that they dont have a release that fits within the period of eligibility. Last year, in the 2nd Annual ATI Automation Honors the ever popular Hewlett Packard QuickTest Professional (QTP) fit into this category, and there was plenty of blow-back because of it. Sorry, 8 Check (http://sourceforge.net/projects/check/) First up is Check. Check is a unit testing framework for C, created by Chris Pickett. Tests are run in a separate address space, so Check can catch both assertion failures and code errors that cause segmentation faults or other signals. Last year Check landed in the Best Open Source Unit Automated Test Tool C/C++ category along with cfix and Google C++ Testing Framework. The results in html. Canoo WebTest contains a set of Ant tasks that drive a simulated, faceless browser known as HtmlUnit. These features were enough to also drive Canoo WebTest version 3.0 into the Best Open Source Functional Automated Test Tool Web category of last years awards. With no major updates to this 3.0 release, which came out in March 2009, WebTest put the breaks on its chances of making it into this years list september 2011
of finalists. ATI was able to reach Dierk Knig, WebTest committer, for comment, however, and he had this to say:
With Canoo WebTest, every build has release quality and we every now and then tag one of the builds as a release - just for the convenience of the ones that are looking for it. With 10 years of development, Canoo WebTest has proven to be very stable and the current changes that we have in mind are mainly about the inner structure (moving to git and gradle) but [not] so much about new features. However, we would very [much] like to hear about any wishes for improvements or new features!
wiki/FunFX-and-Flex) believed to be the same Aslak Hellesy that was a project admin for the FunFX project no further releases were deployed because FunFX is [defunct]. Check out Melomel instead. ATI reached out to Aslak Hellesy and he did confirm that FunFX is defunct. When asked why, Hellesy indicated that:
The reason its defunct is that nobody has maintained it. The reason nobody has maintained it is that nobody needed it enough to bother.
open source tool for testing performance and scalability of web services. It flew onto the scene, snagged an ATI Honors nod, and seemed to fly off just as quickly. Pylot generates concurrent load (HTTP Requests), verifies server responses, and produces reports with metrics. It apparently didnt generate enough interest, however, to win it an ATI Honors award or even warrant an updated release given the fact that the last release was 1.26 which came in July 2009. This delayed delivery may have something to do with the limitations that even the tool producer admits the tool has. On his blog back in February 2010 (http://www.coreygoldberg. blogspot.com/search?updated-min=2010-0101T00%3A00%3A00-05%3A00&updatedmax=2011-01-01T00%3A00%3A0005%3A00&max-results=17), Corey Goldberg, Pylot producer states:
I am working on a new tool/framework for web load testing. [Yes], thats right, another one. My last tool, Pylot, was very limited because of the fact that you had to create a test script in a declarative format. I have come to the conclusion that you *need* a scripting language for defining test cases, and decided to develop a scriptable performance testing tool.
So it seems as though there is still plenty of activity going on with WebTest, but it was determined that none of it needed to be tagged as a major release. So we may be able to look for Canoo WebTest to paddle itself back into the ever-flowing river of finalists in the future. Flash Selenium (http://code.google.com/p/
flash-selenium/)
This would explain why FunFx didnt gain enough votes to win its category in the past ATI Automation Honors awards. Hellesy also went on to reaffirm his suggestion from the web site that users try Melomel instead. Well, apparently people did try Melomel because it is currently in the 3rd Annual ATI Automation Honors awards sitting in the category once occupied by FunFX. Hopefully they will be just as fun, but fairs a little bit better in the voting. fwptt (http://fwptt.sourceforge.net/) fwptt, which stands for Fast Web Performance Test Tool, is a program for load testing web applications, confirmed to work with asp.net applications, but believed to also work with jsp, php and other applications. By using a proxy server, fwptt allows users to record browsing actions into an XML file or a C# class that may be modified and executed to automate the submission of HTTP requests. The use of a proxy server allows a user to record browsing actions using any browser of choice (e.g. IE, Firefox Opera etc). While the f in fwptt stands for fast, the open source project has not been fast enough when it comes to delivering a new version of the software to the community. Since its December 2009 0.8 release that earned it a Best Open Source PERFORMANCE Automated Test Tool Web category nomination in the 2nd Annual ATI Automation Honors, fwptt has been pretty quiet. ATI will be keeping an eye on this tool, however, to see how fast it can makes its way back into the list of awards finalists in the future. Pylot (http://www.pylot.org/) Pylot (Python Load Tester) is a free
While Selenium did make it into the 3rd Annual Awards, its brethren Flash Selenium did not. Flash Selenium is a separate project from Selenium, the popular open source web test tool, but it extends the Selenium RC clients by adding Flash communication capabilities for the Java, .Net, Ruby and Python Selenium RC client drivers. This tool flashed into last years list of finalists, but its 2009 update was not recent enough to get it a return trip to the finals this year. FunFX (http://rubyforge.org/projects/funfx/) It was fun having FunFX in the 2nd Annual ATI Automation Honors, for obvious reasons, but it appears that the fun may now be over. Not only because the tool didnt make it into this years awards, but because it seems as though it is now defunct. FunFX was a framework for functional testing of Adobe Flex applications and provided a Ruby API for accomplishing that task. Unfortunately, it didnt have an updated release to bring it back to the Best Open Source Functional Automated Test Tool Flash/Flex category of the ATI Automation Honors awards that it enjoyed last year. According to a github wiki post from a user named aslakhellesoy (https://github.com/cucumber/cucumber/ september 2011
This left the impression that Goldberg was deciding to abandon the Pylot airspace in favor of a completely different tool project. In the same post, he directs people to a project called Multi-mechanize that he is a member of. To confirm this shift, we went directly to the source. Goldberg issued the following statement:
I am no longer developing Pylot, but it is still very useful and has many users. I am focusing on [Multi-mechanize] development and maintenance.
Multi-mechanize is an open source framework for API performance and load testing. It allows you to run simultaneous python scripts to generate load (synthetic transactions) against a web site or API/ service. It has had more recent releases than Pylot, but unlike Pylot, it has not yet made the ATI Honors Awards list of finalist. Well be on the lookout for more from Multi-mechanize in the future, though, unless Goldberg decides to create a new tool instead. 9
september 2011
september 2011
11
How service Virtualization

By Wayne Ariola
ith large complex systems (mainframes, large
exercise end-to-end transactions involving these components, teams usually need to schedule (and pay for) access to a shared resource. development processes (e.g., prevents the team from performing the level and breadth of testing that they would like. testing increases the severity of these delays and fees exponentially.
in
ment and test teams are commonly vying for access to limited staged testing environments. many cases the dependent systems are complex to manage and maintain for a test lab.
erps,
third party systems), multiple develop-
to
this
approach commonly causes test efforts to be delayed and/or
agile
or parallel development), the demand for frequent and immediate
for iterative
ven if organizations manage to use virtualization for these complex systems, proper configuration for the teams distinct testing needs would require a tremendous amount of work. And once that obstacle is overcome, another is right on its heels. Developing and managing the necessary set of test data can also be overwhelming. service virtualization provides a new way for developers & testers to exercise their applications in incomplete, constantly evolving, and/or difficultto-access environments. rather than virtualizing entire applications and/or databases, service virtualization focuses on virtualizing only the specific behavior that is exercised as developers and testers execute their core use cases. the concept of service virtualization was conceived to emulate the behavior of unavailable
and evolving web services, then evolved beyond the canonical service protocols to support multiple message types and protocolsJDbc, mQ, Jms, and more. this new breed of virtualization radically reduces the configuration time, hardware overhead, and data management efforts involved in standing up and managing a realistic and sustainable dev/test environment. When teams use service virtualization in such contexts, they only need to access the dependent resources long enough to capture the specific functionality related to the components and transactions they are working on. With this behavior captured in virtual assets, developers and testers can then access it continuously, allowing them to exercise end-to-end transactions at whatever time they want (without scheduling) and as frequently as they want (without incurring exorbitant transaction/access fees).
september 2011
Service Virtualization
n removes TesTing roadblocks

Why Move Beyond Hardware and OS Virtualization? Todays complex, interdependent systems wreak havoc on parallel development and functional/ performance testing efforts significantly impacting productivity, quality, and project timeliness. As systems become more complex and interdependent, development and quality efforts are further complicated by constraints that limit developer and tester access to realistic test environments. These constraints often include: Missing/unstable components Evolving development environments Inaccessible 3rd party/partner systems and services Systems that are too complex for test labs (mainframes or large ERPs) Internal and external resources with multiple owners architectures have grown broader, larger, and more distributedwith multiple endpoints and access points. For example, you might have a thick client, a web browser, a device, and a mobile application all accessing the same critical component. Not surprisingly, testing in this environment has become very difficult and time consuming. Furthermore, the number and range of people involved with software quality is rising. Advancements in development methodologies such as Agile are drawing more and more people into quality matters throughout the SDLC. For instance, Business Analysts are increasingly involved with user acceptance testing, QA has become responsible for a broader and more iterative quality cycle, and the development team is playing a more prominent role in the process of software quality and validation. Moreover, todays large distributed teams also exhibit a similar increase in team members involved with quality. Also increasing are the permutations of moving parts not only hardware and operating systems, but also client server system upgrades, patches, and dependent third-party application. As the service-oriented world broke apart many monolithic applications, service orientation also increased and distributed the number of connections and integration points involved in executing a business process. In an attempt to provide all of the necessary team members ubiquitous access to realistic dev/ test environments in light of these constraints, many organizations have turned to hardware and OS virtualization. Virtualizing the core test foundationsspecific operating systems, configurations, platforms, etc. has been a tremendous step forward for dev/test environment management. This virtualization provides considerable freedom from the live system, simultaneously reducing infrastructure costs and increasing access to certain types of systems. Moreover, leveraging the cloud in concert with virtualization provides a nearly unlimited bandwidth for scaling dependent systems. Nevertheless, in terms of development or test environments, some significant gaps remain. First of all, some assets cannot be easily virtualized. For example, it is often
The scope of what needs to be tested is increasing exponentially. With multiple new interfaces and ways for people to access core technology, systems and
september 2011
13
unfeasible to leverage hardware or OS virtualization technology for large mainframe applications, third-party applications, or large ERPs. Moreover, even when virtualization can be completed, you still need to configure and manage each one of those applications on top of the virtualized stack. Managing and maintaining the appropriate configuration and data integrity for all the dependent systems remains an ominous and time-consuming task. It is also a task that you will need some outside help withyou will inevitably be relying on other groups, such as operations or DevOps, to assist with at least certain aspects of the environment configuration and management. Service Virtualization reduces this configuration and data management overhead by enabling the developer or tester to rapidly isolate and virtualize just the behavior of the specific dependent components that they need to exercise in order to complete their end-to-end transactions. Rather than virtualizing entire systems, you virtualize only specific slices of dependent behavior critical to the execution of development and testing tasks. It is completely feasible to use the cloud for scalability with Service Virtualization. Nevertheless, since youre virtualizing only the specific behavior involved in dev/test transactions (not entire systems), the scope of whats being virtualized is diminished and so is the need for significant incremental scalability. What is Service Virtualization? Service Virtualization is a more focused and efficient strategy for
eliminating the system and environment constraints that impede the teams ability to test their heterogeneous component-based applications. Instead of trying to virtualize the complete dependent componentthe entire database, the entire third-party application, and so forthyou virtualize only the specific behavior that developers and testers actually need to exercise as they work on their particular applications, components, or scenarios. For instance, instead of virtualizing an entire database (and performing all associated test data management as well as setting up the database for each test session), you monitor how the application interacts with the database, then you virtualize the related database behavior (the SQL queries that are passed to the database, the corresponding result sets that are returned, and so forth). This can then be accessed and adjusted as needed for different development and test scenarios. To start, you designate which components you want to virtualize, thenas the application is exercised the behavior of the associated transactions, messages, services, etc. is captured in what we call a virtual asset. You can then configure this virtual asset by parameterizing its conditional behavior, performance criteria, and test data. This virtual asset can then emulate the actual behavior of the dependent system from that point forwardeven if the live system is no longer accessible for development and testing. Test data can be associated with these virtual assets, reducing the need for a dependent database and the need to configure and manage the dependent database that, if shared, usually gets corrupted.
By applying Service Virtualization in this manner, you can remove the dependency on the actual live system/ architecture while maintaining access to the dependent behavior. This ultra-focused approach significantly reduces the time and cost involved in managing multiple environmentsas well as complex test data management. What Does Service Virtualization Involve?
Service Virtualization is achieved via the following phases: Capture or model the real behavior of dependent systems Configure the virtualized asset to meet demands of the test scenarios Provision the virtualized asset for the appropriate team members or partners to access and test on their schedule
Phase 1: Capture
Real system behavior is capturedusing monitors to record live transaction details on the system under test; by analyzing transaction logs; or by modeling behavior from a simple interface.
The intent here is to capture the behavior and performance of the dependent application for the system under test and leverage that behavior for development and testing efforts. This capturing can be done in three ways: If you have access to the live system, you can capture behavior by monitoring live system traffic. With a proxy monitoring traffic on the dependent system, the related messages are monitored, and
september 2011
Service Virtualization
then the observed behavior is represented in a virtualized asset. This capturing can cover simple or composite behavior (e.g., a call to transfer funds in one endpoint can trigger an account balance update on another). If you want to emulate the behavior represented in transaction logs, virtual assets can be created by analyzing those logs. This is a more passive (and less politically volatile) approach to capturing the system behavior. If youre working in an environment that is evolving to include new functionality, you might want to model the behavior of the not yet implemented functionality within the Service Virtualization interface. Leveraging the broad scope of protocol support available to facilitate modeling, you can rapidly build a virtual asset that emulates practically any anticipated behavior. For instance, you can visually model various message formats such as XML, JSON, and various legacy, financial, healthcare, and other domain-specific formats. behave from the performance (timing, latency, and delay) perspective. You can also apply and modify test data for each particular asset to reproduce specific conditions critical for completing dev/test tasks. For example, you can configure various error and failure conditions that are difficult to reproduce or replicate with real systems. By adding data sources and providing conditional response criteria, you can tune the virtualized asset to perform as expectedor as unexpected (for negative testing). Phase 3: Provision and Test
The environment is then provisioned for secure access across teams & business partners. The virtualized asset can then be leveraged for testing.
certain set of conditional responses. You can instantly construct an additional virtual asset that inherits those original conditions, and then you can adjust them as needed to meet the needs of a similar test scenario.
Phase 2: Configure
The virtualized assets behavior can be fine-tuned, including performance, data source usage, and conditional response criteria.
After you use any of the three above methods to create a virtual asset, you can then instruct that asset to finetune or extend the behavior that it emulates. For instance, you can apply Quality of Service metrics so you can alter how you would like the asset to
september 2011
Once a virtualized asset is created, it can be provisioned for simplified uniform access across teams & business partnerseither locally or globally (on a globally-accessible server, or in the cloud). They can then be used in unit, functional, and performance tests. Since virtual assets leverage a wide array of native protocols, they can be accessed for manual testing or automated testing by any test suite or any test framework, including Parasoft Test, HP Quality Center suite, IBM Rational Quality Management suite, Oracle ATS, and more. It is also easy to scale virtualized assets to support large-scale, high-throughput load and performance tests. Even after the initial provisioning, these virtual assets are still easily modifiable and reusable to assist you in various dev/test scenarios. For instance, one of your test scenarios might access a particular virtual asset that applies a
Removing Performance Testing and Agile/Parallel Development Process Roadblocks
At the start of this article, I introduced how Service Virtualization removes roadblocks in complex, difficultto-access systems (mainframes, large ERPs, 3rd party systems). To conclude, Id like to explore two other common use case scenarios for Service Virtualization: Performance/capacity-constrained environment Parallel development (Agile or other iterative processes)
Performance/CapacityConstrained Environments Staged environments frequently lack the infrastructure bandwidth required to deliver realistic performance. Placing multiple virtualized
15
Nevertheless, since youre virtualizing only the specific behavior involved in dev/test transactions (not entire systems), the scope of whats being virtualized is diminished and so is the need for significant incremental scalability.
applications on a single piece of hardware can increase access to a constrained resource, but the cost of this increased access is often degraded performance. Although the increased access could technically enable the execution of performance and load tests, the results typically would not reflect real-world behavior, significantly undermining the value of such testing efforts. Service Virtualization allows you to replicate realistic performance data independent of the live system. Once you create a virtual asset that captures the current performance, you can adjust the parameters to simulate more realistic performance. Performance tests can then run against the virtual asset (with realistic performance per the Quality of Service agreement) rather than the staged asset (with degraded performance). Controlling the virtual assets performance criteria is simply a matter of adjusting controls for timing, latency, and delay. In addition to simulating realistic behavior, this can also be used to instantly reproduce performance conditions that would otherwise be difficult to setup and control. For instance, you can simulate various levels of slow performance in a dependent component, and then zero in on how your application component responds to such bottlenecks. Even when it is possible to test against systems that are performing realistically, it is often not feasible to hit various components with the volume typical of effective load/ stress tests. For example, you might need to validate how your application responds to extreme traffic volumes simulating peak conditionsbut how do you proceed if your end-to-end transactions pass through a third-party service that charges per-transaction access fees? If your performance tests pass through a component that you cannot (or do not want to) access under extreme load testing conditions, Service Virtualization enables you to capture its behavior under a low-volume test (e.g., a single user transaction), adjust the captured performance criteria as desired, then perform all subsequent load testing against that virtualized component instead of the actual asset. In the event that the constrained component is not available for capture, you can create a virtual asset from scratchusing Service Virtualization visual modeling interfaces to define its expected behavior and performance. Parallel Development (Agile or other Iterative Processes) Even for simple applications, providing continued access to a realistic test environment can be challenging for teams engaged in parallel development (Agile or other iterative processes). A wide range of team membersincluding developers, testers, sometimes business analystsall need easy access to a dev/test environment that is evolving in synch with their application. If the team decided to take the traditional virtualization route here, they would not only face all the initial setup overhead, but also be mired in constant work to ensure that the virtualized systems remain in step with the changes introduced in the latest iteration. When the team ends up waiting for access to dependent functionality, agility is stifled. Service Virtualization reduces these constraints and associated delays by giving developers and testers the ability to rapidly emulate the needed behavior rather than having to wait for others to upgrade, configure, and manage the dependent systems. Even if anticipated functionality or components are not yet implemented, their behavior can be modeled rapidly then deployed so team members can execute the necessary end-to-end transactions without delay. And if the dependent functionality recently changed, previously-captured behavior can be easily modifiedeither by re-capturing key transactions or
september 2011
by adjusting behavior settings in a graphical interface (without scripting or coding). For example, many organizations are developing mobile applications, and this development is typically performed by a separate mobile development team. Since mobile applications commonly depend on core application components developed and maintained by other teams, the mobile team is often delayed as they wait for the other teams to complete work on the core components that
their own mobile apps need to interact with. Service Virtualization can eliminate these delays by allowing the mobile development team to emulate the behavior of the dependent componentseven if the actual components are incomplete, evolving, or otherwise difficult-to-access during the parallel development process. Key Takeaways Leveraging Service Virtualization, teams reduce the complexity and the costs of managing multiple
environments while providing ubiquitous access for development and test. Service Virtualization helps you: Reduce infrastructure costs Improve provisioning/maintenance of test environments Increase test coverage Reduce defects Improve predictability/control of software cycle times Increase development productivity Reduce 3rd party access fees
Software Test Automation Training

www . training . automatedtestinginstitute . com Software Test Automation Foundations Automated Test Development & Scripting Designing an Automated Test Framework Advanced Automated Test Framework Development Virtual courses Automated Test Development & Scripting Designing an Automated Test Framework Advanced Automated Test Framework Development Test Automation Framework Workshop: A Shortcut to Success
september 2011 www.automatedtestinginstitute.com Automated software testing magazine 17
Training Thats Process Focused Yet Hands On
Public courses
M
by
ake your own
Data-Driven
using
architecture
& TestNG!
Jailton alkimin louzada
Once I started automating projects, I realized the necessity of keeping my captured scripts from the data used to test applications. So, I ended up asking myself How do I keep and reuse these data on my projects? After almost two years working with Selenium - RC, I took that knowledge, plus the knowledge that I already had about the TestNG framework and I decided to use the best of what these two powerful tools could provide to create an Automated Test Architecture that would facilitate my activities, by separating my captured scripts from the reusable data of my projects.
Selenium
The main idea was to create a DDT (Data-driven Testing) Architecture using open-source tools, facilitating the automation activity on projects.
Data Script
18 Automated software testing magazine www.automatedtestinginstitute.com september 2011
Data Driven
How Selenium rc works Selenium Remote Control (RC) is a test tool that allows you to write automated web application UI tests in any programming language against any HTTP website using any mainstream JavaScript-enabled browser. Selenium RC comes in two parts: 1. A server which automatically launches and kills browsers, and acts as an HTTP proxy for web requests from them; and 2. Client libraries for your favorite computer language. The RC server also bundles Selenium Core, and automatically loads it into the browser.
(SOURCE: SeleniumHQ)
What
is
selenium?
Selenium is a suite of tools which automate web browsers across many platforms. Its includes: Selenium IDE, Selenium Remote Control (RC) and Selenium Grid. In this case we will only be working with the Selenium Remote Control. Selenium Remote Control is great for testing AJAX-based web user interfaces, and very useful for Continuous Integration System. Selenium RC has easy integration with plug-ins like JUnit and TestNG, allowing you to run test suites in your favorite IDE, such as Eclipse.
of test: unit, functional, end-to-end, integration. In this article, I use TestNG for endto-end functional tests with Selenium, because it is very flexible and easy to use, and it is also a very powerful tool. For example, if you need to run your tests in a specific order, you can either define test dependencies between methods/groups or set TestNG to run your methods in the same order that you have specified. Furthermore, TestNG has a nice HTML report too. Everything can be done via annotations or through an XML file.
What
is
testnG?
hoW does
TestNG is a Java testing framework, inspired by JUnit and NUnit, but it introduces some new functionalities. TestNG (NG stands for Next Generation) was written originally by Cdric Beust, and allows the coverage of all categories
the architecture Work?
This data-driven architecture uses CSV (comma separated values) files as a data source to provide data for the application, in order for it to be tested. The first step in using the framework is to figure out what data youll use in your test, and change its
Figure 1: Selenium Configuration Property File

Figure 3: Data Source Used For testCase1 Figure 2: Java project

values in the CSV files. Once the data is established, the next step is to update the class named Tests.class, which has all the frameworks test scripts, and then update a property file that has all the selenium configuration information (host address, port, browser and URL), as shown in Figure 1. Later on I will explain how to configure the CSV data file, so it can be read by the application.
package com.jailtonalkiminlouzada.data; /** * @author Jailton Alkimin louzada * www.jailtonalkiminlouzada.com */ import java.io.Filereader; import java.io.ioexception; import java.util.Arraylist; import java.util.regex.Pattern; import au.com.bytecode.opencsv.csvreader; public class csvreader { public Arraylist<string> getvalues(string data, string ctname) throws ioexception{ csvreader reader = new csvreader(new Filereader(data),,); string[] nextline = null; Arraylist<string> valuesFromcsv = new Arraylist<string>(); boolean setstatus = false; Pattern myPattern = Pattern.compile(^+ctname); java.util.regex.matcher mysearch = null; Pattern patterncoment = Pattern.compile(^#.*); java.util.regex.matcher nocomments = null; nextline = reader.readnext(); do{ nocomments = patterncoment.matcher(nextline[0]); mysearch = myPattern.matcher(nextline[0]); if (mySearch.find()) { nextline = reader.readnext(); setstatus = true; } if(nextline[0].trim().equals(})){ setstatus = false; } if (nextline[0].trim().equals({)) { nextline = reader.readnext(); } if (setstatus == true) { if (!noComments.find()) { if (nextline[0].trim().length() != 0) { valuesFromcsv.add(nextline[0]); } } } }while ((nextline = reader.readnext()) != null); return valuesFromcsv;
settinG
up your
environment
The architecture and the examples discussed in this article were developed using the Eclipse IDE among other tools. Here is a list of what you need to set up on your automation environment: 1. Eclipse (My version is Galileo 3.5) - You will need a Java Runtime Environment (JRE) to use Eclipse (Java SE 5 or greater is recommended). 2. TestNG (My version is 6.1.1) (a) You can use a link from the website to install the plugin for running yours tests. (b) You will need to download the jar file to use TestNG annotations and others features.
Figure 4: CSV reader Algorithm

3. Selenium Server (formerly the Selenium RC Server) Selenium Server 2.3.0 is the latest version (as of the time this article was written), but old versions work too.
4. Selenium Client Drivers Selenium client drivers to create scripts that interact with the Selenium Server (Selenium RC, Selenium Remote Webdriver), in this case Im using Java.
september 2011
Data Driven
5. OpenCSV (version 1.8 or superior) - A Simple CSV parser for Java. creating an eclipSe project: First you will need to create an Eclipse project and import all Selenium, TestNG and OpenCSV libraries. To do this go New Java Project to: File (in this example my project name is DataDrivenTesting) and import all libraries mentioned above, then create a folder named DataStorage for example. It will contain all CSV files created by you, to be used in your target test application, once everything is finished your project might look like figure 2.
package com.jailtonalkiminlouzada.testcases; /** * @author Jailton Alkimin louzada * www.jailtonalkiminlouzada.com */ import java.util.Arraylist; import com.jailtonalkiminlouzada.connection.*; import org.testng.annotations.Aftersuite; import org.testng.annotations.beforesuite; import org.testng.annotations.test; import com.jailtonalkiminlouzada.data.DataDrivenProvider; import com.jailtonalkiminlouzada.util.Propertiesloaderimpl; import com.thoughtworks.selenium.Defaultselenium; import com.thoughtworks.selenium.selenium; public class tests { public static string Datadir = system.getProperty(user.dir) + \\Datastorage\\; protected static selenium selenium; @beforesuite (alwaysrun= true) public void setup() throws exception { connection.startseleniumserver(); selenium = new Defaultselenium( Propertiesloaderimpl.getstringvalue(hostnAme), Propertiesloaderimpl.getintvalue(Port), Propertiesloaderimpl.getstringvalue(broWser), Propertiesloaderimpl.getstringvalue(url)); selenium.start(); selenium.windowmaximize(); selenium.windowFocus(); } @Aftersuite public void tearDown() throws exception { selenium.stop(); connection.stopseleniumserver(selenium); } @test(dataProvider=DataDriven,dataProviderclass = DataDrivenProvider.class) public void testcase1(Arraylist<string> valuesFromcsv){ for (string eachvalue : valuesFromcsv) { selenium.open(/); selenium.type(q, eachvalue); selenium.click(btng);
WritinG
source code to
read
csv
file
Reading data from CSV files is an integral part of the architecture, and the way in which the data is stored in these files must first be understood before the source code can be written to read the data. 1. This architecture is designed as follows: The tests will be data driven using the CSV file that contains the class that has the test scripts. In other words, we are using the Tests.class class to run the Selenium scripts, with data from the CSV file named Tests.csv (which should be located in the DataStorage folder). Thus, the class works as a test suite, where we put all the test cases. 2. Since we can have several test cases (methods with @Test annotation) in the Tests.class, I write the methods that will use the CSV Reader functionality to only use the data source that is defined for it on the CSV file. In other words, the data in the CSV file should be set up within a block that is enclosed by brackets ({
september 2011
}
Figure 5: Test.class Source Code

package com.jailtonalkiminlouzada.data; /** * @author Jailton Alkimin louzada * www.jailtonalkiminlouzada.com */ import java.io.ioexception; import java.lang.reflect.Method; import java.util.Arraylist; import org.testng.annotations.DataProvider; import com.jailtonalkiminlouzada.testcases.tests; public class DataDrivenProvider extends tests { @DataProvider(name = DataDriven) public static object[ ][ ] Datastorage(method m ) { string testcasename = m.getname(); string classname = m.getDeclaringclass().getsimplename(); Arraylist<string> values = new Arraylist<string>(); csvreader obj = new csvreader(); try { values = obj.getvalues(Datadir+classname+.csv, testcasename ); } catch (ioexception e) { } } system.out.println(File +classname+.csv not Found!!! +e);
return new object[][]{{values}};
Figure 6: DataDrivenprovider class Source Code

www.automatedtestinginstitute.com Automated software testing magazine 21
Verify/ATI 2011 Co
www.veri
september 2011
onference Schedule
ifyati.com
september 2011
23
and }) and named after the test case method that uses it, as shown in Figure 3. 3. The algorithm in the source code implementation (illustrated in Figure 4) relies on the following rules to effectively read a CSV file: (a) The values from each test case should be in one column (See Figure 3); (b) Values should be separated by commas (of course); (c) The algorithm discards lines that begin with #; in other words, they are treated as comments; (d) The blocks of data must always start and close, respectively, with open ({) and close (}) brackets; (e) The name of the block that defines the values for each test case should be the same as the methods (test cases) that will use them; (f) Remember the CSV filename should be the name of the class containing the scripts that will use the dataProvider and that should be in the DataStorage folder.
package com.jailtonalkiminlouzada.connection; /** * @author Jailton Alkimin louzada * www.jailtonalkiminlouzada.com * credits: http://seleniumhq.org/ */ import org.openqa.selenium.server.RemoteControlConfiguration; import org.openqa.selenium.server.seleniumserver; import com.jailtonalkiminlouzada.util.Propertiesloaderimpl; import com.thoughtworks.selenium.selenium; import java.net.bindexception; import java.net.serversocket; public class connection { public static seleniumserver server; public static void startseleniumserver() throws exception { try { getintvalue(Port_server)); serversocket serversocket = new serversocket(Propertiesloaderimpl. serversocket.close(); //server not up, start it try { RemoteControlConfiguration rcc = new RemoteControlConfigurarcc.setPort(Propertiesloaderimpl.getintvalue(Port_server)); rcc.setsingleWindow(Propertiesloaderimpl. server = new seleniumserver(false, rcc); } catch (exception e) { system.err.println(e.getmessage()); e.printstacktrace(); } try { server.start(); } catch (exception e) { system.err.println(e.getmessage()); e.printstacktrace(); } } catch (bindexception e) { } }
tion(); getboleanvalue(single_Win));
public static void stopseleniumserver(selenium selenium){ selenium.stop(); if (server != null) { try { selenium.shutDownseleniumserver(); server.stop(); server = null; } catch (exception e) { e.printstacktrace(); }
WritinG
a test case usinG

}
dataprovider
} This is the easiest part, because we just need to write a simple test class where we put all test cases in Figure their respective suites. The most important thing here is to identify all of the methods that will use data from our CSV file. For this you will need to establish methods (test cases) with the dataProvider configuration, as shown in Figure 5.
7: Connection Class Source Code

dataProvider: The name of method that will provide the data; dataProviderClass: Name of the class that contains the data provider method; Return Value: The data provider method returns an ArrayList object, so we need to received this object in an ArrayList object (in this case the variable named ValuesFromCSV receives the values from the CSV file).
WritinG
a data provider class
The purpose of the dataProvider is to call the CSVReader class. The next section offers more information on the dataProvider class, but for now, lets take a moment to understand the dataProvider parameters illustrated in Figure 5:
This data provider class will manage all of the data sources that you intend to develop. While this article only focuses on data stored in a CSV file, data could be stored in several other locations such as an excel file or a relational database. Therefore, rather than have the tests directly call the CSVReader class, they call the dataProvider, which is responsible for calling the appropriate reader class
september 2011
Data Driven
O
package com.jailtonalkiminlouzada.util; /** * @author Jailton Alkimin louzada * www.jailtonalkiminlouzada.com */ import java.io.Fileinputstream; import java.io.ioexception; import java.io.inputstream; import java.util.Properties; public class Propertiesloader { private Properties props; public static String CONFIG = config.properties;
data
(CSVReader for our purposes) based on the data store that is selected for use by the test.
public Propertiesloader(string propertie){ props = new Properties(); inputstream in; try { in = new Fileinputstream((new java.io.File(propertie)).getAbsolutePath()); props.load(in); in.close(); } catch (ioexception e) { e.printstacktrace(); }
WritinG a connection class

The connection class is just to separate the connection methods from other classes. Like I said previously, you just need change properties file to set the configurations that you will use to run your tests. In Figure 8 you can see the class that gets information from source file.
conclusion
Selenium is a great open-source tool for automating your tests and combined with TestNG you can easily create an automated framework to facilitate your endto-end test execution. The advantage of using data-driven testing is that it makes it easier to add additional data when you need to use other values or use another product or System Under Test. This makes test automation a much more cost effective solution.
public string getvalue(string key){ return (string)props.getProperty(key); } } package com.jailtonalkiminlouzada.util; /** * @author Jailton Alkimin louzada * www.jailtonalkiminlouzada.com */ public class Propertiesloaderimpl { private static PropertiesLoader config = new PropertiesLoader(PropertiesLoader.CONFIG); public static string getstringvalue(string key){ string value = ; value = config.getValue(key); return value; } public static int getintvalue(string key){ return integer.parseint(getstringvalue(key)); } public static boolean getboleanvalue(string key){ return boolean.parseboolean(getstringvalue(key)); } }
wait! theres more!

Want more Selenium? Come to the Verify/ATI 2011 Testing and Test Automation conference held in Arlington, VA from September 26-28. There will be several talks on various tools including Selenium. Learn more at:
http://www.verifyati.com
Figure 8: Property File Reader Source Code

www.automatedtestinginstitute.com Automated software testing magazine 25
september 2011
AutomAtion
Technical experTise,
ManageMenT, philosophy & The FuTure

Verify/AtI 2011 testing
speak on Automations Most Pressing topics
& TesT AuTomATion ConFeRenCe sPeAkeRs
before They Speak on Even more Pressing Topics
in
L inda
hayes
d orothy G raham n ick o Livo e ric P uGh t om W imsatt d ion J ohnson e Lfriede d ustin
The speakers in the Verify/ATI 2011 Testing & Test Automation Conference coming this month are experts in their fields, and are among the best that the testing and test automation disciplines have to offer. Many of these speakers have already shared information with AST, however, so you can benefit from their expertise before the conference even begins.
www.automatedtestinginstitute.com september 2011
Dion
Johnson
One
Of the mOst pOWerful apprOaches tO making yOur autOmatiOn mOre agile invOlves expanding yOur vieW Of What test autOmatiOn is.
hOW
really
much
WOuld the
speeding prOduct
up
the
critical path, autOmating it Will prObably nOt impact delivery time much.
if
impact
delivery
timeline?
testing
a particular test activity is nOt really in the
the BusIness cAse For AutomAted soFtwAre testIng ast volume 1, issue 1
eggplant, thrOugh the use Of
Is Your AutomAtIon AgIle or FrAgIle ast volume 1, issue 1 risk reductiOn, a third apprOach fOr calculating rOi, is One Of the mOst impOrtant yet undervalued rOi calculatiOns. its impOrtant because this is Where mOst autOmatiOn benefit exists. A-commerce mArketIng ast volume 2, issue 1 diOn
fOllOWing issues: as has alsO
vnc, interacts With all gui elements Of the sut as images, independent Of the gui technOlOgy used. this is
a cOmmOn feature Of many image based autOmated test tOOls, and shOuld nOt be cOnfused With bitmap, cOOrdinate-based checks.
ImAge BAsed AutomAted soFtwAre testIng ast volume 3, issue 1 JOin elfriede cOnference.
as she cO-chairs the
verify/ati
v1 i2, v1 i3, v2 i5.

he cO-chairs the
cOntributed
features
in
the
JOin diOn
cOnference.
verify/ati
Dustin
ElfriEDE
september 2011
27
linDa
hayes
hOrrified, i
On the secOnd day she Openly burst intO tears. asked her What Was WrOng.
in
typical
Waterfall
develOpment
cycle,
develOpers build a backend system that Will becOme the fOundatiOn fOr an applicatiOn. can cOnsist Of stOred prOcedures, Web
Or dlls; but the thing that these have in cOmmOn is that they typically dOnt prOvide an easy Way fOr testers tO access them.
this services,
this
prevents testers
frOm being able tO dO their JOb until much later in the develOpment cycle. creativity and access tO
she
explained that she Was afraid she WOuld lOse her JOb since there Was nO Way she cOuld use Our tOOl because she Was nOt a prOgrammer.
hOWever, With a little visual studiO, yOu can
build yOur OWn tOOl that can learn abOut the assemblies and Web services that yOur develOpers effectively. this via have created, and help yOu test them mOre
evolutIon oF AutomAted soFtwAre testIng ast volume 1, issue 2 the

cOncept Of accessibility applies tO the applicatiOn internals such as naming cOnventiOns and class implementatiOn. autOmatiOn Will be difficult Or even unWOrkable.
this article Will shOW yOu hOW tO dO .nets reflectiOn capabilities.
WithOut
accessibility,
testIng wIth reFlectIon ast volume 2, issue 3 JOin nick as he delivers a presentatiOn On getting it right the first time: hOW tO evaluate a test autOmatiOn tOOl.
developments role In soFtwAre test AutomAtIon ast volume 2, issue 3 JOin linda as she delivers a cOnference tutOrial On agile functiOnal test autOmatiOn. in additiOn, she delivers a presentatiOn On hOW tO becOme an autOmatiOn entrepreneur.
olivo
september 2011
nicK
Eric
Pugh
may prOve yOur cOde is syntactically cOrrect, the intent Of the develOper Was met. running autOmated tests On an
anOther
technical aspect Of test autOmatiOn is
the design Of the testWare architecture
the
structure and relatiOnship Of all Of the items Of testWare that cOmprise the artifacts required fOr autOmated tests tO successfully run.
the
design Of the testWare architecture is a critical
aspect fOr successful test autOmatiOn, and the
cOntinuOus integratiOn WithOut autOmatiOn
skills needed fOr this include technical expertise, as Well as knOWledge Of hOW the tests are tO be used. architecture may be called a test autOmatOr, test architect, Or lead autOmatOr.
and can cOmpile, but dOes nOthing tO prOve that
the
persOn WhO designs the testWare
likeWise, irregular, Or
infrequent schedule means yOu lOse much Of the value Of having thOse tests, since by the time the On tO Other tasks. tests identify an issue, the develOper has mOved
technIcAl vs. non-technIcAl skIlls In test AutomAtIon ast volume 2, issue 5 JOin dOrOthy
tutOrial On
the prOmises Of bOth cOntinuOus integratiOn

and test autOmatiOn Only cOme tO fruitiOn When they are perfOrmed in cOnJunctiOn With each Other!
successful test autOmatiOn. in additiOn, she delivers a keynOte On test autOmatiOn: past, present & future.
as
she
delivers
cOnference
contInuous IntegrAtIon And AutomAtIon go together lIke peAnut Butter And JellY ast volume 2, issue 3 JOin eric
as cOnference On he speaks at the verify/ati better search engine testing.
graham
Dorothy
september 2011
29
tom
Wimsatt
One thing i had tO dO Was simulate netWOrk traffic fOr the devices tO capture statistics. in Order tO validate the statistics cOllected, i had tO prOduce
predictable netWOrk traffic Over a periOd Of time.
wait! theres more!

The foremost software testing and test automation event of 2011 is here! Held in Arlington, VA from September 26th through the 28th, this conference discusses many of the most pressing topics in the realm of software quality, testing and automation. Topics include discussions of the most proven open source tools and how they work. Also, learn about the techniques that help to improve test automation ROI, how the cloud alters a testing approach, how security concerns may be addressed in an affordable manner and much more. The conference agenda is packed with skilled practitioners and internationally known speakers including Joe Jarzombek of the Department of Homeland Security (DHS), Linda Hayes of Worksoft, author and testing expert Dorothy Graham, and many more. This high powered conference features numerous educational and networking opportunities including: Keynote Presentations from Industry Leaders In-depth Tutorials Numerous Concurrent Presentations High Interest Tool Specific Sessions Live presentation of the 3rd Annual ATI Automation Honors TABOK Certification Training and Exam http://www.verifyati.com To learn about an opportunity for a complimentary registration visit: http://www.verifyaticomp.automatedtestinginstitute.com
a rather expensive piece Of brOadband test equipment Was used tO accOmplish this task. it came With a fairly nice gui prOgram tO drive the unit but i cOuld nOt get the granularity and cOntrOl i needed tO evaluate the data. fOrtunately, the manufacturer Offered device drivers in c++ and tcl! aWesOme! i quickly began learning hOW tO
prOgram the machine With the precise type and make-up Of netWOrk traffic
i needed.
how to get Into AutomAtIon ast volume 2, issue 2 JOin tOm as he speaks On building a cOst effective test management sOlutiOn by integrating ms Office.
To Learn more about the conference or to Register visit:
september 2011
Software Test Automation Training

www . training . automatedtestinginstitute . com Software Test Automation Foundations Automated Test Development & Scripting Designing an Automated Test Framework Advanced Automated Test Framework Development Virtual courses Automated Test Development & Scripting Designing an Automated Test Framework Advanced Automated Test Framework Development
Training Thats Process Focused Yet Hands On
Public courses
Featured Course Come participate in a set of test automation courses that address both fundamental and advanced concepts from a theoretical and hands on perspective. These courses focus on topics such as test scripting concepts, automated framework creation, ROI calculations and more. In addition, these courses may be used to prepare for the TABOK Certification exam.
september 2011
Public and Virtual Training Available

31


Register Now!

www.verifyati.com
For more information, visit http://www.networking.automatedtestinginstitute.com

latest From the b
I bLog to U
Automation blogs are one of the greatest sources to-date test automation information, so the Autom Testing Institute has decided to keep you up-to-d some of the latest blog posts from around the we interesting posts, and keep an eye out, because y your post will be spotlighted.
Blog Name: ABAKAS Post Date: July 27, 2011 Post Title: On Performance Author: Catherine Powell
Post Title: Test Automation: Beyond Rudimentary Script-lets Author: Bj Rollison Blog Name: I.M. Testy Post Date: May 19, 2011
Thats the dirty little secret of performance: you can almost always be faster if youre willing to pay more. At some point, you need to decide to stop, that more performance isnt worth the cost. So have fun with performance. Make it faster. Make it better. Look in several places and get some more speed (or scalability or load) out of your app. But watch your costs, and when youre paying more than the benefit youre getting, stop.
But, I think one of the reasons that test automation generally gets a bad rap is because of overly simplistic scripts that mindlessly perform the same thing over and over again and provides little real value to the team beyond emotional feel-good or frustration that often accompanies automation projects. So, building upon the example from last week, lets look at how we might consider a few simple design principles and craft a more robust functional automated test...
Read More at: http://www.testingmentor.com/imtesty/category/test-automation/
Read More at: http://blog.abakas.com/2011/07/on-performance.html
september 2011
blogosphere
s of upmated date with eb. Read below for some you never know when
Blog Name: szabgab Blog Post Date: July 21, 2011 Post Title: Modern Perl tutorial part 04 Author: szabgab Blog Name: InfoStretch Blog Post Date: August 29, 2011 Post Title: Why We Make Mistakes in Test Automation Author: Jeff Lucas
I think the most [interesting] function I am going to show now is substr. It is basically the opposite of index(). index() will tell you where is a string while substr will give you the substring at a given locations. Normally substr gets 3 [parameters]. The first one is a string. The second is a 0-based location, also called offset and the third is the length of the substring we would like to get.
A fix is made to the application, you run a scripted test suite (which passes), and you then declare the application as ready for user testing. Five minutes into the test turns up a bug that was not caught during the scripted run. A typical solution is to expand the test suite to check for the bug the next time. Coverage complete again, job well done! The actual cause was probably the reliance on scripted tests as the only validation. A more realistic solution would be to investigate ways of providing effective, targeted exploratory testing in association with the scripted checks...
Read More at: http://blog.infostretch.com/why-we-make-mistakes-in-test-automation
Read More at: http://szabgab.com/blog/2011/07/modern-perl-tutorial-part-04-string-functions-length-lc-uc-undex-substr.html
september 2011
35
Verify/ATI Con
Venture Beyond
www.ver
nference 2011
d the Abstract
Register Now!
The Test Automation Conference Youve Been Waiting For
rifyati.com
Go On A retweet
paying a visit to t
Microblogging is a form of communication based on the concept of blogging (also known as web logging), that allows subscribers of the microblogging service to broadcast brief messages to other subscribers of the service. The main difference between microblogging and blogging is in the fact that microblog posts are much shorter, with most services restricting messages to about 140 to 200 characters. Popularized by Twitter, there are numerous other microblogging services, including Plurk, Jaiku, Pownce and Tumblr, and the list goes on-and-on. Microblogging is a powerful tool for relaying an assortment of information, a power that has definitely not been lost on the test automation community. Lets retreat into the world of microblogs for a moment and see how automators are using their 140 characters.
@arinsime Congrats on speaking at #verify/ati conference. Car pool?
Twitter Name: hudsonci Post Date/Time: Aug 8 Topic: Hudson/Jenkins at Verify/ATI
automated testing with #Hudson:Security with SoapUI, Browser with Selenium - VerifyATI conf: Sept Arlington VA http://bit.ly/peaC9Z
Twitter Name: dep4b Post Date/Time: July 27 Topic: Verify/ATI 2011 Conference Speaker
the microblogs
If your site takes more than 0.8 seconds to load Ive already thrown myself out the window.
Twitter Name: CloudTest Post Date/Time: Aug 24 Topic: Load Tests
6 Ways to Kill Your Servers http://j.mp/nKkqGL < Easily avoidable with some planned load testing #webperf #softwaretesting
Twitter Name: SaraJChipps Post Date/Time: Aug 31 Topic: Website Performance
Much like todays earthquake (east coast, USA) the Verify/ATI will shake up the world of testing! Register today @verifyati
Twitter Name: automatedtest Post Date/Time: Aug 13 Topic: ATI Honors at the Verify/ATI
@verifyati 3rd Annual ATI Honors winners will be announced live at the Verify/ATI 2011 Conference: Sept Arlington VA verifyati.com
Twitter Name: verifyati Post Date/Time: Aug 23 Topic: Verify/ATI 2011 Shaking Up Things
september 2011
39
Hot topics in Automation
Joining Forces is Hot!

The Trend of Merging Tools and Tool Organizations
not affect the tools themselves, but may serve to connect and increase collaboration among the large user communities that each organization individually enjoyed. According to Joe Krivickas, CEO, SmartBear, they are thrilled to welcome the eviware team and user community to the SmartBear family. The addition of the worlds most widely used Web services testing tool soapUI, aligns extremely well with our other popular tools to build, test and monitor some of the best software applications and websites, anywhere. This is yet another important step this year on our mission to provide one of the worlds top three platforms for software quality and to deliver Quality Anywhere to our users on the desktop, mobile and in the cloud. While the eviware acquisition is relatively recent, SmartBear is no stranger to unification. SmartBear Software, after all, was already a combined company that brought together AutomatedQA (the
upe
r r
ool
League, tools and tool organizations have found that the act of teaming with other tools and/or tool organizations is hot! Two organizations that have recently decided to come together are eviware
The automated test tool market continues to grow and expand to such a great extent, that rarely do you have to ask the question, Is there a tool to meet my needs? The prevailing question is more times than not, Which tool will best meet my
Merging great tools and tool organizations to make even greater tools and tool organization seems to be hot trend.
needs? With this growth, a few select tools tend to rise to the top of the test tool heap as leaders and trend setters that establish the mark by which all other tools in the market are measured. This natural selection, or sorts, is an interesting phenomenon to watch, but perhaps even more interesting to watch is when such tools and/or the organizations behind the tools join forces to become even more powerful. Like the Super Friends or the Justice and SmartBear Software. SmartBear Software, a software company that provides tools for software developers, quality assurance engineers and IT professionals, is perhaps best known in QA circles for its TestComplete functional automated test tool. Eviware is the creator of soapUI and loadUI, the worlds leading testing software for Web Services. SmartBear announced its acquisition of eviware on July 5, 2011, a move that may previous owner of TestComplete), Pragmatic Software (producers of Software Planner, now known as ALMComplete, QAComplete and DevComplete, and free tools, QAPlanner and DevPlanner), and the previous Smart Bear Software (makers of CodeCollaborator). SmartBear seems smart indeed and they leave one to wonder who they will add to their Justice League of Tools next. This joining of forces is not just
(Continued on page 42) 40 Automated software testing magazine www.automatedtestinginstitute.com september 2011
ATI Automation Honors

Celebrating Excellence in the Discipline of Software Test Automation
Winners Announced Live at Verify/ATI 2011 Conference on September 27!
www.atihonors.automatedtestinginstitute.com
3rd Annual
(Continued from page 40)
limited to organizations, however, but is actually seen with tools themselves. The recent joining of Selenium and WebDriver to make Selenium 2.0 has the automated testing community buzzing. Selenium and WebDriver were both tools for automating web applications, but the way in which each accomplished this goal was different. Selenium was a suite of tools underscored by Selenium Core (a JavaScript library embedded within a browser). Through the use of a proxy, Selenium tests, written in one of several available languages of the automators choosing, were able to communicate with the JavaScript library in order to drive browser activity. WebDriver, conversely, did not use a JavaScript embed or a proxy, but instead accomplished the same goal by talking natively to the browser. This native communication may be different for each browser. For example, in Firefox, this meant using JavaScript wrapped in an XPCOM component, while in IE, this may involve using C++ to drive IEs Automation Controls. Each of the tools had their own advantages and challenges. Seleniums JavaScript-based approach made simple the process of adding support for additional browsers, but presented some problems when attempting to use Selenium on an application that imposed security restrictions on JavaScript. WebDriver offered a user-friendly API, and averted many security related restrictions imposed by a given browser, but didnt support nearly as many languages for test development as Selenium. For these reasons and more, the two camps felt that there was definite potential for synergy between the tools; a synergy to be realized by a merger. The Selenium website offers a quote from Simon Stewart, creator of WebDriver, stating the following:
Why are the projects merging? Partly because Webdriver addresses some shortcomings in selenium (by being able to bypass the JS sandbox, for example. And weve got a gorgeous API), partly because selenium addresses some shortcomings in webdriver (such
as supporting a broader range of browsers) and partly because the main selenium contributors and I felt that it was the best way to offer users the best possible framework.
So with Seleniums name recognition powered by some of WebDrivers improved implementation choices, this new super team will definitely be one to watch. While some tools and organizations join forces invariably changing or enhancing the power of both, other mergers are a little more one-sided. Such is the case, it would seem, with the unification of JUnit and well every other tool you can think of! Maybe not every tool, but a lot of tools have connected with JUnit to offer an improved solution. JUnit an open source unit testing framework for the Java programming language has, for example, been enlisted by a tool known as JSystem. JSystem is an open-source automated test framework that deploys JUnit as part of its framework for writing the automated tests. Jsystem is not alone in this repurposing of JUnit for test development. Selenium, which supports many languages for writing tests, counts JUnit as one of its more popular mechanisms for test creation. FitNesse, another opensource framework, also employs JUnit for some of its purposes. Not to the extent that JSystem or even Selenium do, but it has found JUnit to be useful. Fitnesse has its own engine for executing its Java-based tests, but offers the flexibility for those tests to be executed in a different developer IDE. This is made possible via a set of JUnit helpers included as part of Fitnesse that make it possible to execute a test or a suite of tests using JUnit. Merging great tools and tool organizations to make even greater tools and tool organization seems to be hot trend. Weve seen it with SmartBear and eviware, Selenium and WebDriver, as well as JUnit and Everyone! Whos next to join together and form a new league of super tools?
C ontent C oday T
Community Comments Box
ontribute
Announcements & Blog Posts
Automation Events
As a registered user you can submit content directly to the site, providing you with content control and the ability to network with like minded individuals.
learn more today at

http//www.about.automatedtestinginstitute.com
september 2011
http://www.googleautomation.com


Register Now!

www.verifyati.com

AutomatedSoftwareTestingMagazine September2011

Enviado por

Dados do documento

Descrição original:

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

AutomatedSoftwareTestingMagazine September2011

Enviado por

Direitos autorais:

Formatos disponíveis

An AUTOMATED TESTING INSTITUTE Publication - www.automatedtestinginstitute.

Tools, services and daTa: AutomAtion in ParTs

Verify/ATI 2011 Prequel

removing roADblocks by virtuAlizing services

Selenium and TestNG

Verify/ATI Conference 2011

EntEr for ComplimEntarty rEgistration @ omplimEntArty EgistrAtion

Venture Beyond the Abstract

How Service virtualization removeS teSting roadblockS 12

make your own data-driven arcHitecture uSing Selenium & teStng! 18

By Jailton Alkimin Louzada

automation tecHnical expertiSe, management pHiloSopHy & tHe Future 26 ,

columnS & departmentS

Read featured microblog posts from the web.

autHorS and eventS open Sourcery 8

Read featured blog posts from the web.

Hot topicS in automation

Automated software testing magazine

tools, Services & data: automation in parts

Successful test automation is normally a product of several different working parts.

Automated software testing magazine

Authors and events

CONTACT US AST Magazine astmagazine@automatedtestinginstitute.com ATI Online Reference contact@automatedtestinginstitute.com

Ati and Partner events

Are You Contributing Content Yet?

Community Comments Box

Announcements & Blog Posts

learn more today at http//www.about.automatedtestinginstitute.com

open Source wins, loses & puns

Automated software testing magazine

Automated software testing magazine

10 Automated software testing magazine

Automated software testing magazine

How service Virtualization

ith large complex systems (mainframes, large

third party systems), multiple develop-

approach commonly causes test efforts to be delayed and/or

or parallel development), the demand for frequent and immediate

12 Automated software testing magazine

n removes TesTing roadblocks

Automated software testing magazine

14 Automated software testing magazine

Removing Performance Testing and Agile/Parallel Development Process Roadblocks

Automated software testing magazine

16 Automated software testing magazine

Software Test Automation Training

Training Thats Process Focused Yet Hands On

ake your own

the architecture Work?

Figure 1: Selenium Configuration Property File

Figure 3: Data Source Used For testCase1 Figure 2: Java project

Figure 4: CSV reader Algorithm

Figure 5: Test.class Source Code

return new object[][]{{values}};

Figure 6: DataDrivenprovider class Source Code

22 Automated software testing magazine

Automated software testing magazine

a test case usinG

7: Connection Class Source Code

a data provider class

WritinG a connection class

wait! theres more!