Escolar Documentos
Profissional Documentos
Cultura Documentos
This is the
first tutorial in three-part HP LoadRunner training series.
In first two tutorials we will introduce you to performance testing and in last tutorial we
will share Load Testing with LoadRunner video tutorials.
See Also => List of all LoadRunner Video Tutorials.
Why Performance testing?
Performance testing has proved itself to be crucial for the success of a business. Not only
does a poor performing site face financiallosses, it also could lead to legal repercussions
at times.
No one wants to put up with a slow performing, unreliable site in cases of purchasing,
online test taking, bill payment, etc. With the internet being so widely available, the
alternates are immense. It is easier to lose clientele than gain them and performance is
a key game changer.
Therefore, performance testing is no longer a name sake checkpoint before going live. It
is indeed a comprehensive and detailed stagethat would determine whether the
performance of a site or an application meets the needs.
Introduction
The purpose of this test is to understand the performance of application under load,
particularly users.
Load Testing
Load testing is a type of performance test where the application is tested for its
performance on normal and peak usage. Performance of an application is checked with
respect to its response to the user request, its ability to respond consistently within
accepted tolerance on different user loads.
The key considerations are:
1. What is the max load the application is able to hold before the application starts
behaving unexpectedly?
2. How much data the Database is able to handle before system slowness or the
crash is observed?
3. Are there any network related issues to be addressed?
Stress Testing
Stress testing is the test to find the ways to break the system. The test also gives the
idea for the maximum load the system can hold.
Generally Stress testing has incremental approach where the load is increased gradually.
The test is started with good load for which application has been already tested. Then
slowly more load is added to stress the system and the point when we start seeing
servers not responding to the requests is considered as a break point.
During this test all the functionality of the application are tested under heavy load and
on back-end these functionality might be running complex queries, handling data, etc.
The following questions are to be addressed:
What is the max load a system can sustain before it breaks down?
How is the system break down?
Is the system able to recover once its crashed?
In how many ways system can break and which are the weak node while handling
the unexpected load?
Volume Testing
Volume test is to verify the performance of the application is not affected by volume of
data that is being handled by the application. Hence to execute Volume Test generally
huge volume of data is entered into the database. This test can be incremental or steady
test. In the incremental test volume of data is increased gradually.
Generally with the application usage, the database size grows and it is necessary to test
the application against heavy Database. A good example of this could be a website of a
new school or college having small data to store initially but after 5-10 years the data
stores in database of website is much more.
The most common recommendation of this test is tuning of DB queries which access the
Database for data. In some cases the response of DB queries is high for big database, so
it needs to be rewritten in a different way or index, joints etc need to be included.
Capacity Testing
=> Is the application capable of meeting business volume under both normal and peak
load conditions?
Capacity testing is generally done for future prospects. Capacity testing addresses
the following:
1. Will the application able to support the future load?
2. Is the environment capable to stand for upcoming increased load?
3. What are the additional resources required to make environment capable enough?
Capacity testing is used to determine how many users and/or transactions a given web
application will support and still meet performance. During this testing resources such as
processor capacity, network bandwidth, memory usage, disk capacity, etc. are
considered and altered to meet the goal.
Online Banking is a perfect example of where capacity testing could play a major part.
Reliability/Recovery Testing
Reliability Testing or Recovery Testing is to verify as to whether the application is able
to return back to its normal state or not after a failure or abnormal behavior- and also
how long does it take for it to do so(in other words, time estimation).
An online trading site if experience a failure where the users are not able to buy/sell
shares at a certain point of the day (peak hours) but are able to do so after an hour or
two. In this case, we can say the application is reliable or recovered from the abnormal
behavior.
In addition to the above sub-forms of performance testing, there are some more
fundamental ones that are prominent:
Smoke Test:
How is the new version of the application performing when compared to previous
ones?
Is any performance degradation observed in any area in the new version?
What should be the next area where developers should focus to address
performance issues in the new version of application?
Endurance Test:
Whether the application will able to perform well enough over the period of time.
Any potential reasons that could slow the system down?
Third party tool and/or vendor integration and any possibility that the interaction
makes the application slower.
In a system involving multiple applications that interact with one another, all the
interface components must be deemed as critical for performance test.
Example: E-commerce sites interface with online banking sites for payments, which is an
external third party application. This should be definitely the part of Perf testing.
HP LoadRunner,
Jmeter,
Silk Performer,
NeoLoad,
Web Load,
Rational Performance Tester (RTP),
VSTS,
Loadstorm,
Web Performance,
LoadUI,
Loadster,
Load Impact,
OpenSTA,
QEngine,
Cloud Test,
Httperf,
App Loader,
Qtest,
RTI,
Apica LoadTest,
Forecast,
WAPT,
Monitis,
Keynote Test Perspective,
Agile Load, etc.
Conclusion
In this article we have covered most of the information required to build a base to move
ahead and understand the Performance testing. In the next article we will apply
these concepts and understand the key activities of Performance testing.
Deliverable, etc.)
e) In-Scope and Out-of-Scope
f) Test Environment (Configuration, Tool, Hardware,
Server Monitoring, Database, test configuration, etc.)
g) Reporting & Communication
h) Test Metrics
i) Role & Responsibilities
j) Risk & Mitigation
k) Configuration Management
Purpose of test
Scenario summary
Duration of test
Throughput
Graphs
Graphs comparison
Response Time
Error occurred
Recommendations
#8. Report
Test results should be simplified so the conclusion is
clearer and should not need any derivation. Development
Team needs more information on analysis, comparison of
results, and details of how the results were obtained.
Test report is considered to be good if it is brief,
descriptive and to the point.
The following guidelines will smooth this step out:
Use appropriate heading and summary
Report should be presentable so that it can be used
in the management meetings.
Provide supporting data to support the results.
Give meaningful names to the table headers.
Share the status report periodically, even with the
clients
Execution Summary
System Under test
Testing Strategy
Summary of test
Results Strategy
Problem Identified
Recommendations
Along with the final report, all the deliverable as per test
plan should be shared with the client.
Conclusion
We hope this article has given a process oriented,
conceptual and detailed information on how performance
testing is carried out from beginning to end.
In the past tutorials we have seen the basics of Performance testing and LoadRunner
video tutorials. This article is going to focus on the most important commonly asked
LoadRunner interview questions and answers that will help you be successful in
performance testers interview using LoadRunner.
LoadRunner is one of the best licensed Performance testing tools in the market. It is best
suited for most upcoming technologies because of the wide range of supported protocols.
A few basic pointers before we begin:
#1) LoadRunner interview questions can be categorized into 3 main types Scripting,
Execution and Analysis. It is important for beginners to focus more on the scripting
part.
#2) Http/html is mostly used protocol, for a start try to perfect this protocol.
#3) Be sure to know the exact version of LoadRunner that you worked on. In case of
work experience with a previous version, try to keep yourself updated with the features
that are part of the newer/current versions.
#4) Performance Testing interviews are more practical than they used to be.
Scenario oriented questions are common rather than straight forward ones. Some
companies, even make scripting tests a part of the interview process. So, be prepared
for the same.
#5) Even in scripting, it is preferred that you be able to customize code, instead of
just record and replay.
#6) Expect questions on think time, transactions, comments, recording options,
run time settings, etc. these are to test your knowledge of scripting best practices.
The following are some of the performance testing interviewquestions that will need
some experience to answer. Try to keep these questions in mind while working on your
performance test projects, so the interview preparation activity becomes a continuous
process.
1. What are the different scripting issues you faced so far?
2. What are the performance bottlenecks that you found in projects you were
working? What are the recommendations made to overcome those issues?
3. Have you applied Littles law in your project? If so, how?
4. What is your approach for analysis?
5. What do you monitor while execution?
6. How to extract server data for test execution and how to analyze that?
7. How to identify performance bottlenecks?
Key question areas are:
Challenges that you face during scripting
Correlation function
Error handling
Different recording modes for Web HTTP/HTML protocol.
Scenario creation
Challenges during execution
Analysis
while replaying needs correlation. Any user input while recording should be
parametrized.
Q #11. What is parameterization & why is parameterization necessary in the
script?
Ans => Replacing hard coded values within the script with a parameter is called
Parameterization. This helps a single virtual user (vuser) to use different data on each
run. This simulates real life usage of application as it avoids server from caching results.
Refer this tutorial for more details.
Q #12. How you identify Performance test use cases of any application?
Ans => Test cases/Uses cases for Performance test are almost same as any
manual/functional testing test cases where each and every step performed by the user is
written. The only difference is that all manual test cases cant be Performance testing
use cases as there are few criteria for the selection as:
I. The user activity should be related to critical and most important functionality of the
application.
II. The user activity should be having good amount of database activity such as search,
delete or insert.
III. The user activity should be having good user volume. The functionality having less
user activity is generally omitted from Performance testing point of view. e.g admin
account activity.
Any of the manual test cases that fulfill the above criteria can be used as performance
testing use case/test case. If manual test cases are not written step by step ,
Performance team should create dedicated documents for them.
Q #13. While scripting you created correlation rules for automatic correlation.
If you want to share the correlation rules with your team member working on
the same application so that he/she can use the same on his workstation, how
will you do that?
Ans => Correlation rules can be exported through .cor file and the same file can be
imported through VuGen.
-----------Q #14. What are different types of vuser logs which can be used while scripting
and execution? What is the difference between these logs? When you disable
logging?
Ans => There are two types of Vuser logs available Standard log and Extended log.
Logs are key for debugging the script. Once a script is up and running, logging is enabled
for errors only. Standard log creates a log of functions and messages sent to the server
during script execution whereas Extended log contains additional of warnings and other
messages. Logging is used during debugging and disabled while execution. Logging can
be enabled for errors in that case.
Q #15. What is Modular approach of scripting?
Ans => In Modular approach, a function is created for each request (e.g. login, logout,
save, delete, etc.) and these functions are called wherever required. This approach gives
more freedom to reuse the request and saves time. With this approach it is
recommended to work with web custom request.
(Note- This question needs practical knowledge. So please this practically and formulate
your answer).
Q #25. What is difference between pacing and think time?
Ans => Pacing is wait time between the action iterations whereas think time is wait time
between the transactions.
Q #26. What are the number of graphs you can monitor using Controller at a
time? What is the max of them?
Ans => One, two, four and eight graphs can be seen at a time. The maximum number
of graphs can be monitored in at a time is 8.
Q #27. You have an application which shows the exam results of the student.
Corresponding to name of each student its mentioned whether he passed or
failed the exam with the label of Pass and Fail. How will you identify the
number of passed and failed student in VuGen script?
Ans => For this text check is used for the web page for the text Pass and Fail.
Through the function web_reg_find, we can capture the number of texts found on the
web page with the help of SaveCount. SaveCount stored the number of matches
found. For example1
web_reg_find("Text=Pass",
2
"SaveCount=Pass_Student",
3
LAST);
4
web_reg_find("Text=Fail",
5
"SaveCount=Fail_Student",
6
LAST);
Q #28. During the load test what is the optimum setting for Logs?
Ans => For the load test log level is set to minimal. This can be achieved with setting
the log level to the standard log and selecting the radio button Send message only
when an error occurs.
Q #29. How will you handle the situation in scripting where for your mailbox
you have to select any one mail randomly to read?
Ans => For this we will record the script for reading the first mail. Try to find what is
being posted in the request to read the first mail such as mail ids or row no. From the
post where a list of mails is reflecting, we will try to capture all the email ids row no with
correlation function and keeping Ordinal as All i.e. ORD=All . Replace the requested
email id in the read post with any of the randomly selected email id from the list of
captured email ids.
Refer this Scripting Tutorial.
Q #30. What is the Think Time? What is the Threshold level for think time and
how can be this changed?
Ans => Think time is the wait time inserted intentionally between the actions in the
script to emulate real user`s wait time while performing activity on the application. The
Threshold level for Think time in the level below which recorded think time will be
ignored. This can be changed from Recorded options->Script->Generate think time
greater than threshold.
Q #31. How is Automated Correlation configured?
Ans => Any setting related to Automated Correlation can be done byGeneral Options>Correlation. Correlation rules are set fromRecording options->Correlations.
Q #32. How you decide the number of load generator machine required to run a
test?
Ans => Number of load generator required totally depends on the protocol used to
create the script and configuration of the load generator machine. Each protocol has
different memory print and this decides how many virtual users can be generated from
the give configuration of the machine (load generator).
Q #33. What are the capabilities exactly you look for while selecting the
performance testing tool?
Ans => Performance testing tool should capable of: Testing an application built using multiple technologies and hardware platforms.
Determine the suitability of a server for testing the application
Testing an application with load of tens, thousand and even thousands virtual
users.
Q #34. How concurrent users are differing from simultaneous users?
Ans => All simultaneous users are concurrent users but vice versa is not true.
All the vusers in the running scenario are Concurrent users as they are using the same
application at the same time but may be or may not be doing the same tasks.
Simultaneous users perform the same task at the same time. Concurrent users are made
Simultaneous users through rendezvous points. Rendezvous points instruct the system
to wait till a certain number of vusers arrive so that they all can do a particular task
simultaneously.
Q #35. How do you identify which values need to be correlated in the script?
Give an example.
Ans => This can be done in ways:
a) Record the two scripts with similar steps and compare them using WDiff utility. (See
tutorial Correlation).
b) Replay the recorded script and scan for correlation. This gives a list of values that can
be correlated.
Session Id is a good example of this. When two scripts are recorded and compared using
WDiff utility. Session ids in the two scripts should be different and WDiff highlight these
values.
Q #36. How does caching affect performance testing results?
Ans => When data is cached in server`s memory, the server need not fetch the result
and no server activity triggered. Test result does not reflect the same performance of
real user using the application with different data.
Q #37. How will you stop the execution of script on error?
Ans => This can be achieved through lr_abort function. The function instructs the vuser
to stop executing Action section and end the execution by executing the vuser_end
section. This function is helpful in handling a specific error. This can also be used to
handle a situation rather than error where execution is not possible. The function
assigned Stopped status to the vuser which stopped due to lr_abort function. In RunTime setting, Continue on error should be unchecked.
LoadRunner Interview
Questions -1
1) What is Performance Testing?
The process of testing to determine the performance of software product.
2) What is Load Testing?
A type of performance testing conducted to evaluate the behavior of a component or
system with increasing load, e.g. numbers of parallel users and/or numbers of
transactions, to determine what load can be handled by the component or system.
3) What is Stress Testing?
A type of performance testing conducted to evaluate a system or component at or
beyond the limits of its anticipated or specified work loads, or with reduced availability of
resources such as access to memory or servers.
4) What is Spike Testing?
Verify the Systems performance under sudden increments and decrements.
5) What is Data Volume Testing?
Testing where the system is subjected to large volumes of data.
6) What is Endurance Testing?
Verifying the Systems performance under continues load in terms of users and
transactions.
7) What is LoadRunner?
It is a Performance Test Tool from HP. It supports all aspects of Performance Testing like
Load, Stress, Endurance, spike and Data volume testing.
8) What are the tools available in the industry for Load Testing?
LoadRunner from HP
RPT (Rational Performance Tester) from IBM
Silk Performer from Micro Focus
JMeter (Open source Tool) Etc
17) What is the difference between Overlay graph and Correlate graph?
Overlay Graph:
It overlay the content of two graphs that shares a common x-axis. Left Y-axis on the
merged graph shows the current graphs value & Right Y-axis show the value of Y-axis of
the graph that was merged.
Correlate Graph:
Plot the Y-axis of two graphs against each other. The active graphs Y-axis becomes Xaxis of merged graph. Y-axis of the graph that was merged becomes merged graphs Yaxis.
18) How did you plan the Load? What are the Criteria?
Load test is planned to decide the number of users, what kind of machines we are going
to use and from where they are run. It is based on 2 important documents, Task
Distribution Diagram and Transaction profile. Task Distribution Diagram gives us the
information on number of users for a particular transaction and the time of the load. The
peak usage and off-usage are decided from this Diagram. Transaction profile gives us the
information about the transactions name and their priority levels with regard to the
scenario we are deciding.
19) What does vuser_init action contain?
Vuser_init action contains procedures to login to a server.
20) What does vuser_end action contain?
Vuser_end section contains log off procedures.
21) What is Performance Test Tool?
A tool to support performance testing that usually has two main
Facilities: load generation and test transaction measurement.
Load generation can simulate either multiple users or high volumes of input data. During
execution, response time measurements are taken from selected transactions and these
are logged.
Performance testing tools normally provide reports based on test logs and graphs of load
against response times.
22) What are the phases in LoadRunner Test Process?
I) Planning the Test
II) Creating VUser Scripts
III) Creating the Scenario
IV) Running the Scenario
V) Monitoring the Scenario
Web 2.0
Step 3
lr_measure_stop("trans2)
Step 4
trans1 and trans2 are transaction names, which will have the response
times to measure periodically and those response times will be displayed
from the graphs in controller during the load test execution and those
transaction response times can be analyzed from LoadRunner analysis
after load test execution.
8) What is think time in LoadRunner?
Think time is nothing but the user delay between two subsequent
requests.
Assume that a user opened page1 and he is filling the data on page1.
During filling the page the user has spent 10 sec, and he has submitted
the page1, then page2 is loaded. In this case the user wait time between
pag1 and page2 10 sec is called think time.
9) What is tuning in LoadRunner? How to use this tuning option in
any project?Any one pls explain me in detail.?
We are having different tunings like DB Tuning, Network Tuning, and
Server Tuning and so on.
10) What is VUGen?
The VUGen stands for Virtual User Generator. VuGen is used to generate
vuser script (here we record a business operation performed by a single
user).
11) What is the analyzer in LoadRunner?
This gives you the results of the load runner test. These results can be
viewed in graphs and reports.
12) Do we see much difference in load testing for web
applications versus traditional software?
Yes. From own experience, in traditional applications, the developers
know more about how it all works, if its in house then developers are
easy to access and know the environment. With Web and CMS, there is so
much the developers dont know about, this is especially true when they
integrate out of the box solutions. So many software developers now are
using solutions they buy from someone else and they are slow to turn
Hits per second means the number of hits the vserver receives in one
second from the vuser and the request per second is the no. of request
the vuser will request from the server.
25) What is the advantage of using LoadRunner?
Advantages are:
1. With help of vusers reduces the human users
2. Reduces the reqirement of the systems
3. Helps in the better usage of time and money
4. Effective utilization of automation
5. Everything done from a single point.
Network communicators
Server response
4) How to identify the memory leakage using Loadrunner?
In Load runner, every application has a processor running in the system. The processor
needs to be identified. Using the performance tab we can check the memory
consumption of the processor. Continuous tracking needs to be done while load testing.
However, if the memory keeps increasing even on stopping the test, a memory may have
occurred. Also, if the memory is not released on stopping the test, a memory may have
occurred.
5) How do we debug a LoadRunner script?
Debugging a LoadRunner script:
VuGen contains two options to help debug Vuser scripts-the Run Step by Step command
and breakpoints. The Debug settings in the Options dialog box allow us to determine the
extent of the trace to be performed during scenario execution. The debug information is
written to the Output window. We can manually set the message class within your script
using the lr_set_debug_message function. This is useful if we want to receive debug
information about a small section of the script only.
6) What are the performance tester roles and responsibilities using loadrunner?
There won't be load runner roles and responsibilities. It is the roles of performance
engineer and there won't be difference between the roles what ever tool you use for
performance testing.
7) When do you do load and performance Testing?
We perform load testing once we are done with interface (GUI) testing. Modern system
architectures are large and complex. Whereas single user testing primarily on
functionality and user interface of a system component, application testing focuses on
performance and reliability of an entire system. For example, a typical application-testing
scenario might depict 1000 users logging in simultaneously to a system. This gives rise
to issues such as what is the response time of the system, does it crash, will it go with
different software applications and platforms, can it hold so many hundreds and
thousands of users, etc. This is when we set do load and performance testing.
8) Do you feel like performance testing is an accepted critical part of the
development life cycle?
It is getting that way yes, with more and more crashes getting exposure on the news
here and with it happening more and more it has become a critical part of testing.
10) What are the key KPIs you track for performance testing and tuning?
The key KPIs that we track for performance testing are transaction response time,
memory usage, disk space, and CPU time.
Tuning: Network delay and stored procedure times.
Scalability testing is a process of evaluating the systems behaviour when the number of
simultaneous users is increasing, and the hardware and software resources are fixed.
This testing will be conducted for comparing the response times and system resource
utilization of AUT when the number of users are increased.
19) What are the results reported after capacity Testing?
Results Reported after capacity Testing are:
The hardware resource related bottlenecks will be reported.
Recommendation on new hardware.
Up-gradation of existing hardware.
Change in the existing application deployment architecture to support the future
growth.
Introducing new servers in the application deployment architecture.
20) What are the results reported after Stress Testing?
Results Reported after Stress Testing are:
1. The maximum number of users supported by AUT.
2. Let us assume that, the system resources are utilized beyond the expected limits.
After the stress test, when the normal amount of users are running, the status of the
application in terms of the resources utilization will be reported.
3. Assume that there are many errors are coming when the stress testing is been
conducted. After the stress test, when the normal amount of users are running the
status of the application in terms of the errors will be reported.
21) What are the disadvantages of using commercial performance / load
testing tools?
Disadvantages of using commercial performance/load testing:
We need to understand the need for any commercial tool with respect to the kind of
technologies we use in your organisation. A tool that do not support the technologies is
waste of time and money. If it can fit into our requirements, there is a considerable ROI.
22) How do you test an application if it is going production for the first time?
For testing the application, you need to have the basic scenarios done first.
Second step will be to do the End to End testing with E2E scenarios.
Third test will be to do the rigorous testing.
Final step will be to do the load testing.
23) Why should we automate the performance testing?
Its a discipline that leverages products, people and processes to reduce the risk of
application, upgrade or patch deployment. It is about applying production work loads to
pre-deployment systems while simultaneously measuring system performance and enduser experience.
24) What are the results reported after the Endurance/Longevity/Soak
Testing?
Results Reported after the Endurance / Longevity / Soak Testing Test:
When the endurance test is conducted on a multi tier web based enterprise level
applications the following kind of results will be reported.
Memory leaks on the application servers
JVM Heap size utilization on the application servers
Connection leaks on the database server
Cursor leaks on the data base servers
Response time (consistency or degradation) comparison for start of the load test to
end of the load test
Systems Resource (Memory, CPU, Network and Disk usage etc) comparison for
beginning of the load test to end of the load test
Application errors occurrence over the period of time.
25) What are all the things will be considered while doing performance testing?
Does the application respond quickly enough for the intended users?
Will the application handle the expected user load and beyond?
Will the application handle the number of transactions required by the business?
Is the application stable under expected and unexpected user loads?
Are we sure that users will have a positive experience on go-live day?
26) What are the results reported after the load Testing?
Results Reported after the load Test:
The system will be validated to ensure whether the service level agreements or
performance objectives are met.
Average, max, min and standard deviation of response times for each scenario will be
measure and reported.
Resource utilization of each of the systems which are part of AUT will be monitored and
reported.
If there is any application break point below the peak load condition, it need to be
identified and reported.
27) What are the results reported after Spike Testing?
Results Reported after Spike Testing:
The systems resources are utilization comparison for, with and without spikes.
The response times comparison for, with and without spikes.
Observation on errors for, with and without spikes.
28) What is remote command launcher?
The remote command launcher enables the controller to start applications on the host
machine.
29) How to determine the Stress Point?
Determining the Stress Point:
Transaction response times are exponentially increased
The application started throwing the errors for many users
The system stopped responding
At least one of the server in AUT architecture got crashed
The system resource utilization went beyond the acceptable limits.