WHITEPAPER

IdentifyingSoftwarePerformance BottlenecksusingDiagnostictools

Abstract
Inthiswhitepaper,ImpetusTechnologiesfocuseson softwareperformancediagnostictoolsandhowthey enablerapidbottleneckidentification.Itdiscusses serversideandclientsidediagnostics,including pluggablediagnostictoolsandcustomsolutionsacross variousscenarios. Thewhitepaperalsotouchesonthebenefitsof adoptingdiagnostics,suchasenhancedcollaboration andapplicationvisibility.

ImpetusTechnologies,Inc. www.impetus.com September2011

Iden ntifyingSoftwa arePerformanc ceBottlenecks susingDiagnos stictools

Tableo ofConte ents

Introduction..... ...................... ............................................................... .......2 eimportance eofDiagnostictools.................................................... .......3 The Howdiagnostictoolswork?.. ............................................................... .......4 Imp petusapproa achtoserver sidediagnos .................................... stics .......6 Imp petussolutio onforclientsidediagnostics...................................... .......8 Ben nefitsoffered dbyDiagnosticstools................................................. .......9 Sum mmary.......... ...................... ............................................................... .....10

In ntroduct tion
Identifyingbottlenecksa andperforma ancesnagsinapplicationsisbecoming criticalforo organizations stoday.Typic cally,thisisun ndertakenusingloadtest results.The eperformanc ceengineering gteamdetec ctsproblemareasbyexecu uting performanc cetests,andthetestresults,includingproblemarea as,transactio on responsetimes,andsystemmonitoringdataaret thensharedw withthe developme entteam.The edevelopmen ntteamidentifiessystemb bottlenecksu using resourcem monitoringdat taandhelpsi inthetuningofsystemcomponents.Th hese recommendationsareth hensharedw withtheperfo ormanceengin neeringteam m, thatvalidat testhesamewithperform mancetests. However,id dentifyingcodelevelbottl lenecksisachallengingtas skforthe developme entteam.Itm mayfinditdiff ficulttoanaly yzeandcomp prehendthetest results,ast theyarebase edonusertra ansactions.Also,developm ment environmentsarevastly ydifferentfro omtestenviro onmentsinte ermsofmemory andprocessingpower. Organizatio onsaretheref foreonthelo ookoutforsolutionsthatc canhelpthem m reducetheiterationsbe etweenthepe erformancee engineersand dthedevelopment teamsandspeedupthe eprocessofid dentifyingand disolatingco odebottlenecks. Theyareke eentofindwa aysbywhichtheycanmap pusertransactionresultsto themethod dandqueryle evel,sothatt thedevelopm mentteamcanbehelpedi inits analysis.Di iagnosticstoo olscanhelpcompaniesaddresstheseis ssues.

Iden ntifyingSoftwa arePerformanc ceBottlenecks susingDiagnos stictools

T Theimpo ortanceofDiagn nosticto ools

Diagnostictoolsmapthe eusercentric cbusinesstra ansactionstomethodcalls s, therebypro ovidingadevelopercentri icviewofthe etestresults.Thesetoolsa are beneficialf forthefollow wingreasons: The eycandrilldo ownthetrans sactionrespo onsetimeinto ounderlying me ethods,querie esandservice ecallstocom mponents;and dprovide res sponsetimeb breakdownacrossdifferen ntsubcompo onents. The eyfollowthepathoftheu usertransactionandgathe erperformanc ce dat tathroughoutitslifetime. The eyenablethe edevelopmen ntteamtoanalyzethesestatisticsalong g wit ththemethod dtrace,andisolatethebo ottlenecks.Th hisprovidesan intuitive,easyto ouseview,o ofhowindivid dualtiers,com mponents, me emory,andSQ QLstatement tsimpacttheoverallperfo ormanceofa bus sinessproces ssunderloadconditions;a andfacilitates sdeeperanalysis. The eycanbeinte egratedwiththeavailableloadtestingsolutionsand d effe ectivelyusedintestenviro onments. The eycanbeuse edinproducti iontosetupacontinuous sperformance e mo onitoringmec chanism,duetotheirlowo overheads.

Howeveritisimportanttodifferentia atebetweenthetermsdiagnostictool ls, andprofile ers,whichareoftenusedbyteamsinte erchangeably y.Thefactisthat thesearetw wodifferentsetsoftoolsw withdifferentpurposes. Profilersar remoresuited dforfinegrainedanalysisandusuallyh havemore overheadsthandiagnos stictools.This smakesthem minappropriateformultiu user tests.Diagn nostictoolsontheotherh hand,canbeu usedformult tiuserloadte ests, aswellas,p productionm monitoring.Th heirmainpurposeistoiso olatethecode e bottlenecks sandprovide erelevantinfo ormation.The edevelopmentteamcanu use Profilersto oexplorethes seisolatedpro oblemareasandfigureou uttheirrootc cause. Essentially,Profilersand dDiagnostict toolscomplem menteachother.

Iden ntifyingSoftwa arePerformanc ceBottlenecks susingDiagnos stictools

Howdiag H gnostict toolswo ork?

Diagnostictoolscanbeusedinmultipleenvironm mentsincludin ngdevelopme ent, test,stagingandproduc ction.Howeve er,theyarem mostwidelyusedinaload testingenv vironment.

Figure1H HowDiagnost ticToolsWork scanbeappli iedtotheapp pserver,asw wellastheclie entside,inclu uding Diagnostics thebrowse erandmobile edevice.Diagnostictoolsh havetwomajorcomponen nts theDiagno osticagentan ndDiagnosticserver. TheDiagno osticagentrunsonthetarg getcompone entandhooks supwithit,using bytecodeinstrumentati ion.Itgathersperformanc cestatisticsandstarts transmittin ngthedatato otheserver.T Theagentrun nsinthesame eruntimeast that ofthetarge etcomponent.

Iden ntifyingSoftwa arePerformanc ceBottlenecks susingDiagnos stictools

TheDiagno osticserverisresponsiblef forcollectingtherawdata a,aggregating git andstoring gitintheresu ultrepository y.Theuserca anthengener ratemultiple graphsandreports;and dmonitortheapplicationp performanceinrealtimea as wellasperf formanofflin neanalysis.

Serversidediagnostic cs
Thereared differentscen narioswhered diagnostictoolscanbeused.Keyamon ng themisser rversidediagnostics.Inthi isparticulars scenario,Diag gnostictoolsa are usedtoide entifycriticali issuesinthes servercompo onents.These emayinclude ethe applicationserver,databaseserver,W Webserver,m messagingserver,among others.The eperformanceengineering gteamcanch hoosebetwee enplugintoo olsor customtoo ols. PluginDiagnostictools s PluginDiag gnostictoolsareintegrate edwiththelo oadtestingtool.Therearealso situationsw wherecompa anieshavetocreateacust tomdiagnosti icharnessto supportsom mespecificte echnologyorrequirement t.Thesetoolsdonotintegrate withanyloadtestingtoo olandworkinisolation. Plugintoolsareusedas sanextension ntoexistingl loadtestingtools.Theirm major advantageisthattheyc canintegratediagnosticda atawithtestr results.The businesspr rocesses,inth hiscase,canbemappedto omethodand dquerycalls. Dependingonthepathoftheusertr ransactionintheapplicatio on,the distribution nofresponse etimeisprovi ided. Thismakes sanalysiseasi ierforthedev velopmentte eam.Theteam mcannotonly identifythe emethodexe ecutiontimesbutalsoverifyanyredund dantmethod calls. Thediagno osticsconfigur rationishand dledbythelo oadtestingtool.Themetho ods thatneedt tobemonitor redareinclud dedasapartofthetestsc criptsthatare e executed. agnostictools CustomDia Standalone eDiagnostictoolsworkind dependently,monitoringtheperforman nce statisticsof fthetargetapplication,as swellasmappingthemethodandquer ry detailstoth herequestURLsasreceive edbytheserv vercompone ent. Thesetools sareinstalled dandconfiguredseparatelyinthetarge etenvironme ent. Theyarewellsuitedtow workwithexi istingloadtes stingtoolsan ndtestscripts s. nologyrequire ements,acus stomtoolmig ghtbecreated Dependingonthetechn orkduringthe eloadtestsandprovideperformancem monitoring. thatcanwo Therearea afewOpenSo ourceutilities sthatarealso oavailableforthesame. Glassbox,a andInfrareda aresomeofp populartoolsinthiscatego ory.
5

Iden ntifyingSoftwa arePerformanc ceBottlenecks susingDiagnos stictools

Imp petusap pproach htoserv versided diagnostics

ImpetusTe echnologieshasevolvedan napproachto oserversidediagnosticsth hatit hasusedef ffectivelyinm multiplecusto omerengagem ments. Thefirstan ndthemostim mportantstep pareselectin ngtheapprop priateDiagnos stic tool.Thec choiceisofco oursebetweenapluginto oolorcreating gacustom solution,w whichinturnd dependsonse everalfactors s,includingth hefollowing: The etechnology ybeingused. Ifnoexistingtoolisavaila ableforthe tec chnologystackbeingusedintheapplica ation,itbeco omesimperativeto cre eateacustom msolutionusin ngavailableA Aspectoriente edframeworks. The eobjectiveof fusingDiagn nostictools.If ftheobjectiv veistousethese dur ringloadtest tingactivities, ,organization nscanchoose eeitherpluginor cus stomtools.However,ifthe eobjectiveistocontinuou uslymonitor per rformance,th hereisneedf forstandalonetools. The ebudget.Ift thebudgetisaconstraint,organization nscanoptfor r OpenSourceorfreelyavailab bletools.Else e,theycango oforspecializ zed olsthatprovid derichanalys siscapabilitie esandaremo oreuserfriend dly. too

Afterselect tingthetoola andconfiguringitintheta argetenvironment,Impetu us recommendsthatperformanceteam msmeasureth heoverheadi intermsofCP PU utilizationa andimpactonresponsetimes.Itispos ssibletocontr rolthisoverhead bysettingu uptheapprop priatefilterss sothatonlythespecificm methodscanb be monitored. .Thisisveryi importantincaseoftheproductionmo onitoring mechanism mbecausecom mpaniesmaynotwanttohaveahighim mpactofthe Diagnostictoolontheenvironment. oolisproperly yconfiguredandcalibrate ed,itispossib bletorunload d Aftertheto testswitht thediagnostic cson.Thetes stresultsalon ngwiththedi iagnosticresu ults canthenbe eanalyzedto ogetinsightso ontheperfor rmanceofthe ecomplete application. esinformatio onsuchastop p10methods sbyexecution ntime,metho od Thisinclude hotspots,c costliestquer ries,etc.

Clientside ediagnostics s
Diagnostictoolscanalso obeusedtoi identifyclient tsideperform mancesnags, esktopclients s,suchasbro owsers;andth heappsrunni ingonmobile e coveringde devices.Thisisextremelyimportantasclientsideperformance egreatlyimpa acts theenduse erexperience e.

Iden ntifyingSoftwa arePerformanc ceBottlenecks susingDiagnos stictools

MobileDia agnostics Mobilediag gnosticsrefer rstothemeasurementoftheperforma anceofamob bile applicationonadevice.Thisisachallengingtaskf forperformanceengineer rs. Duetothelackofresourcesonthed device,thepe erformanceof fanapplication candegradedrastically.Also,theapp plicationmay ybehavediffe erentlyon differentde evices,aswellastechnolo ogies. Byusingdia agnostictools s,performanceengineerin ngteamscanarriveatmet thod executiont timesonthemobiledevice eandlogthesame.Thism mayhelpin analyzingth heexpensivemethods. Therearentmanydiagn nostictoolsav vailableinthemobilespac ce.Profilers, includingJ2 2MEprofilers s,AndroidTra ace,etc.havetobeused,w whicharespe ecific todifferent ttechnologiesandaremo ostlydeployed dbythedeve elopmentteam ms. However,inordertoaddressspecific crequiremen nts,itisprefer rablethattea ams createcust tomdiagnosti icsolutions. BrowserDi iagnostics Internettec chnologiesru unningapplica ationsalsofaceissues.Tod day,richInternet applicationsareavailabl lethathaveb beenbuiltusingAJAX,HTM ML5,FLEX, SilverLight,jQueryands similartechno ologies.These ealsopresentperformanc ce challengestoWebdeve elopers.Itisre eallydifficulttoidentifybo ottleneckson nthe browserthatcanimpact ttheenduse erexperience.Diagnostict toolscanhelp pin thisprocess. Mostofthe eclientsided diagnostictoo olsareplugin nstopopularbrowsers.Th hese canbeinte egratedwitht thebrowsera andprovidea allthestatistic csrelatedtopage rendering,Javascriptex xecutions,me emory,CPUutilization,etc c.Someofthe e populardia agnosticplug insincludesFirebugforM Mozilla,Speed dtraceforChr rome, HttpWatchforIE. Thereareo othersetofto ools,including gDynatraceA AjaxEdition,t thatwork independen ntlyandextra acttheabove einformationfromthebro owser.Thesehave morefeatu uresandprovidedashboar rdsandfinegrainedanalys sisforidentify ying anyexisting gissues.

Iden ntifyingSoftwa arePerformanc ceBottlenecks susingDiagnos stictools

Im mpetussolution nforclie entsided diagnostics

Nottooma anytoolsareavailableformobilediagn nostics,asmu ultiple technologie esandplatformsexist,and dnosingleto oolcanservet thepurpose.Itis bettertocr reatecustomdiagnosticso olutionsthataddressspec cificrequirements ofmobilea applications. Infewenga agements,Im mpetushasusedAspectorientedframeworkstobuil lda diagnosticutilityformobileapplicationsonthede evices.TheAO OPframewor rks likeAspectMeforJ2MEandAspectLU UAforLUAca anbeusedfo orthispurpose. Thesefram meworksuseb bytecodeinst trumentation ntoinjecthoo oksintothetarget applicationandmeasure eruntimesta atistics,sucha asmethodtim mingandmem mory utilization.Asetofreus sableaspects scanbecreat tedthatcanb behookedint tothe importantm methodcallsorUIrenderi ingcallstotra acktheirperf formance statistics. Theaspects scanbeinser rtedusingapplicationbina ariesandhen ncethereisno o needtoundertakeanyc codechanges s.Themethod dstobemonitoredare specifiedat tthetimeofinstrumentat tion. Inordertoperformdiag gnosticsforbrowserstoid dentifyperformance bottlenecks s,werecomm mendinstallin ngproperplug ginsandcon nfiguringtheseto collectperf formancestat tisticsduringbrowsingses ssion. Theplugin ngathersallth herelevantd detailsandma akesthemavailabletothe e performanc cetestingtea am.Theteam mcanthenana alyzetheresu ultstoidentif fy bottlenecks srelatedtop pageloading,Javascriptex xecution,mem moryleaks,C CPU utilization,etc. Inoneofourengageme entbyjusttun ningclientsid decomponents,application performanc ce,especially ytheenduser rexperience,wasimprove edbyalmost4 40%.

Iden ntifyingSoftwa arePerformanc ceBottlenecks susingDiagnos stictools

Benefitsof fferedb byDiagno osticsto ools

Diagnostictoolsofferarangeofbene efitstoperformancetestin ngand developme entteamswithinorganizat tions,includin ngthefollowi ing: Ide entificationan ndisolationo ofperformanc cebottleneck ks,whichmay ybe com mponents,methods,orSQ QLstatements s.Thisinturn n,assistsin res solvingmemo ory,exception n,orotherpro oblemsswiftl ly,thereby, red ducingtheeff fortsofthede evelopmentt team. Com mpleteapplic cationvisibilit tyacrossthelifecycle,ensuringhigher qua alitywhenap pplicationsgo olive.Theyals sohelpreduc cetheMeanT Time toResolutionintheapplicati iondeployme entenvironment. Documentationoftherootc causeofanyp performanceorstabilityissues, fendinglineo ofcode,evenincomplex,highlydistributed includingtheoff app plications.Theperformanc cetestersdonotneedany yprogrammin ng kno owledgeorac ccesstosourc cecodetode eliveranofflin nediagnosis rep portthatallow wsdevelopmenttoefficientlyresolveis ssues. Effe ectivecollabo orationbetwe eenperforma anceengineersandthe dev velopmentte eam,duetoth heavailability yofdetailedt testresults.T This red ducesthemultipleiterationsbetweenb boththeteam msandspeedsup the eproblemres solutiontime. Imp provementin ntheproductivityoftheDevelopmentt team,including dev velopers,arch hitectsandda atabaseengin neers.Thetim meintensive, ted dious,errorproneandofte enimpossible eprocessofm manually cor rrelatinglogf files,fromdiff ferentbacke endsystemsw withtheclient sideloadtesting gresults,beco omesobsolet te.Thissignif ficantly celeratestheproblemreso olutiontime. acc Inb builtintelligen nceandsugge estionsforteamsbasedon nbestpractic ces. Als so,someofth hetoolscanb bedirectlyintegratedwiththedevelope ers env vironmentan ndpinpointth hebottleneck katthecodelevelitself.

Iden ntifyingSoftwa arePerformanc ceBottlenecks susingDiagnos stictools

Summ mary
Insummary y,itcanbesa aidthatDiagn nostictoolsca andoalottoimprove applicationperformance e,byidentifyingthebottle enecks.Also,companiesm must fferentiatebe etweenDiagnostictoolsan ndProfilersth hatarenot learntodif interchange eable,butrat thercomplem mentarytoea achother.Pro ofilershavehigh overheadsandarebette ersuitedforf finegraineda analysis. Organizatio onscanuseD Diagnostictoo olsinmultiple eenvironmen ntsincluding developme ent,test,stagingandprodu uction.Itisve eryessentialforcompanie esto embraceDiagnosticsintheirperform manceengineeringstrategiestoenhanc ce visibilityacrossapplicati ionsandimpr rovecollabor rationamongvarious stakeholders. Itispossibletobuildcus stomdiagnos sticssolutions stoaddresss specific requiremen nts.Thesecan nbeinstalled dandconfigur redseparatel lyinthetarge et environmentandarewe ellsuitedtow workwithexis stingloadtes stingtoolsandtest scripts. Itisalsoad dvisabletominimizetheov verheadsbya applyingfilter rsonthe methods,o orURLsthath havetobemo onitored.Itisagoodpract ticetocalibra ate theoverheadsoftheDia agnostictools sintheteste environmentbyrunningafew sanitytests s

AboutImpet tus ImpetusTech hnologiesoffersProductEng gineeringandT TechnologyR& &Dservicesforsoftwareprod ductdevelopment. Withongoing ginvestmentsinresearchan ndapplicationo ofemergingte echnologyarea as,innovativeb businessmode els,and anagileappr roach,wepartnerwithourclientbasecom mprisinglarges scaleISVsandt technologyinn novatorstodeliver cuttingedge esoftwareprod ducts.Ourexpertisespansth hedomainsofBigData,SaaS,CloudCompu uting,Mobility Solutions,Te estEngineering g,Performance eEngineering,andSocialMediaamongoth hers. ImpetusTechnologies,Inc. vard,Suite450 0,SanJose,CA95129,USA 5300StevensCreekBoulev @impetus.com Tel:408.252.7111|Email:inquiry@ velopmentCentersINDIA:NewDelhiBangaloreIn ndoreHydera abad RegionalDev Toknowmo orevisit:http:/ //www.impetus.com

Di isclaimers
Theinformationcon ntainedinthisdocumentistheprop prietaryandexclus sivepropertyofIm mpetusTechnologi iesInc.exceptaso otherwiseindicate ed.Nopartof isdocument,inwh holeorinpart,ma aybereproduced, ,stored,transmitted,orusedforde esignpurposeswithoutthepriorwri ittenpermissiono ofImpetus thi 10 0 TechnologiesInc.