Escolar Documentos
Profissional Documentos
Cultura Documentos
Flint McGlaughlin
Director, MECLABS
GettingtheMostFromMetrics g INTRODUCTION
spius
AboutMarketingExperiments
spius
AboutMarketingExperiments
KeyfindingsarefeaturedintheMarketingExperiments Journalandinlivewebclinics Journal and in live web clinics Contributedmorethan$10milliondollarsofresearchfor freeaccessatwww.MarketingExperiments.com free access at www.MarketingExperiments.com Theorganizationgeneratesrevenueintwoways:
ProfessionalCertificationCoursessuchasLandingPage Professional Certification Courses such as Landing Page Optimization,EmailMessagingOptimization,andThe FundamentalsofOnlineTesting ResearchPartnershipswithgroupssuchastheNewYorkTimes, TheRoyalBankofCanada,andJohnson&Johnson
spius
TodaysPresenter FlintMcGlaughlin
Director,MECLABSGroup , p
FlintMcGlaughlinistheDirectorofMECLABSGroup.The organizationhaspartneredwithkeymarketleadersincluding,The NewYorkTimes,MicrosoftCorporation,andReutersGroup.Dr. New York Times Microsoft Corporation and Reuters Group Dr McGlaughlinalsoservesastheDirectorofEnterpriseResearchat theTransformingBusinessInstitute,UniversityofCambridge(UK), astheChairmanoftheBoardofGovernorsforSt.Stephens University,andasaTrusteeforWestminsterTheologicalCentre.Dr. McGlaughlinoriginallystudiedPhilosophyandTheologyatthe UniversityofLondonsSpecialistJesuitCollege.Today,hisprimary researchisfocusedonenterpriseastransformativeagent.Hiswork research is focused on enterprise as transformative agent His work haswonmultipleawardsandhasbeenquotedinmorethan13,000 onlineandofflinesources.
spius
Experiment:Background
EXPERIMENTID: (Protected) EXPERIMENT ID (P d) LOCATION:MarketingExperimentsResearchLibrary TESTPROTOCOL:#TP1092 RESEARCHNOTES: BACKGROUND:Companyisapublisherofelectronic BACKGROUND: Company is a publisher of electronic marketinginformationandoffersrelatedservices. GOAL:Increaseregistrationsforafreeemailnewsletter. GOAL: Increase registrations for a free email newsletter RESEARCHQUESTION:Whichsignuppagewillyieldthe highestconversionrate? highest conversion rate? TESTDESIGN:A/B/CMultivariatetestinvolvingchangesin headline,credibilityindicators,andimagesaccordingto headline, credibility indicators, and images according to optimizationbestpractices.
spius 2010 Cisco Systems, Inc. All rights reserved.
Experiment:OriginalPage
Original
Weradically redesignedthis redesigned this pagebasedupon commonbest practices inlanding pageoptimization. Theresultingpages gp g werethefollowing:
spius
Experiment:Treatment1
Treatment2 Treatment 2 Treatment1
spius
Experiment:Treatment1
Treatment1 Cl Clearerheadline,emphasizing h dli h ii thevalueproposition FeatureClientlistaddedto emphasizethevalueand reduceanxiety. Easy to read and scan body Easytoreadandscanbody copy Formfieldsreducedtothe bareminimum CostumerTestimonialsto reduceanxiety reduce anxiety Antispamsealtoreduce anxiety
spius
Experiment:Treatment2
Treatment2 Treatment 2 Headline is stronger and more quantitative to emphasize the value proposition. Added more testimonials Rearranged page to give better site flow. Add d personal f l with Added l feel i h images and hand-written signature. Tell me where to send language used.
spius
10
Experiment:Results
Original Treatment1 Treatment 1
Vs.
Conversion=14.26%
Conversion=6.74%
53%
spius 2010 Cisco Systems, Inc. All rights reserved.
11
Experiment:Results
Original Treatment2 T t t2
Vs.
Conversion=14.26%
Conversion=6.84%
52%
spius 2010 Cisco Systems, Inc. All rights reserved.
12
Experiment:Results
53%DecreaseinConversion
Treatment1decreasedconversionby53% d d b
CTR
14.26% 6.74% 6.84%
53%
spius
13
Experiment:Background
EXPERIMENTID: (Protected) LOCATION:MarketingExperimentsResearchLibrary TESTPROTOCOL:#TP1115 TEST PROTOCOL: #TP1115 RESEARCHNOTES: BACKGROUND:Companyisapublisherofelectronic marketinginformationandoffersrelatedservices. GOAL:Increaseregistrationsforafreeemailnewsletter. GO i i f f il l RESEARCHQUESTION:Whichsignuppagewillyieldthe highestconversionrate? hi h t i t ? TESTDESIGN:A/Bmultifactorialtestwithaminimalist strategyofreducingpageelements. t t f d i l t
spius 2010 Cisco Systems, Inc. All rights reserved.
14
Experiment:Control
Original
Afteraseriesoftestsin Aft i ft t i whichcommonlanding pagebestpracticesfailedto improveconversiononthis improve conversion on this originalpage. Ouranalystsbegantesting Our analysts began testing theremovalofelements fromthepagetotestvisitor motivationlevels. Ifaddingelementsto increasethevalue propositionhurt conversion,thetrafficto thispagemightalreadybe highlymotivated hi hl ti t d
15
spius
Experiment:Treatment
Treatment
Muchofthecopy onthispageis removed,leaving d l i simpleform submissionfields. Norealselling pointsareincluded inthisdesign. in this design.
spius
16
Experiment:Control&Treatment
Original Treatment
spius
17
Experiment:Results
78%IncreaseinConversion
Thenewtreatmentincreasedconversionby78% h b
CR
12.09% 21.54%
78%
spius
18
Experiment:Results
WHATYOUNEEDTOUNDERSTAND:Itisnotenoughto WHAT YOU NEED TO UNDERSTAND: It is not enough to graspahandfulofbestpracticerulesandthenfling thematapage.Wemustunderstandthemotivations ofthevisitorscomingtoourpagesandcraftour f th i it i t d ft messagestomatchthosemotivations. Thekeytounderstandingmotivationisunderstanding metrics.
spius
19
GettingtheMostFromMetrics g Whataremetrics?
spius
CommonQuestions
Wefrequentlyhearthefollowingmetricquestionsfromthe W f l h h f ll i i i f h MarketingExperimentscommunityofmarketers: 1. Isourconversionrate goodenough? 2. Howcanwegetour marketingdepartmentsto testandmeasure test and measure everything? 3. Howcanweaddmore reportingcapabilitiesto ourmetrics?
Butarewereallyaskingtherightquestions?
spius 2010 Cisco Systems, Inc. All rights reserved.
21
GettingtheMostFromMetrics g Experiment
spius
Experiment:Background
EXPERIMENTID: (Protected) LOCATION:MarketingExperimentsResearchLibrary TESTPROTOCOL:TP1214 TEST PROTOCOL: TP1214 RESEARCHNOTES: BACKGROUND:Aleadingbusinesssolutionsoftwareprovider g p GOAL:Toincreasetotalleadsgeneratedfromsearchengine campaigns p g RESEARCHQUESTION:Whichprocesswillgeneratethemost leads? TESTDESIGN:Radicalredesignofthecompletelead generationprocessfromthePPC aditselftothefinalpageof theleadform.
spius 2010 Cisco Systems, Inc. All rights reserved.
23
Experiment:OriginalPage
Original
spius
WebsiteData
Andmuchmore
spius
Experiment:Developingadatadrivencampaign
Throughaprocesswe callChannel Mapping we can look Mappingwecanlook atdatainsuchaway astoclearthroughthe clutteranddiscover l tt d di thewaystocreate effectiveonline marketingcampaigns.
spius
26
Experiment:Results
54%IncreaseinClickthrough
Theoptimizedlandingpageincreasedclickthroughby54% LandingPage Original Optimized p CTR
7.17% 11.06% 06%
RelativeDifference
54.26%
WHATYOUNEEDTOUNDERSTAND:byanalyzingthedata andcreatingacampaignbasedonunderstandingfromour metrics,wewereabletoincreaseclickthroughbyover54%. metrics we were able to increase clickthrough by over 54%
spius 2010 Cisco Systems, Inc. All rights reserved.
27
Understandingmetrics
Sohowdowegofrommoundsofdatatoeffective campaigns? W ill l k Wewilllookspecificallyattheprocessmentioned ifi ll h i d inthepreviousexperimentcalledChannel Mapping which will be the key method for taking Mapping,whichwillbethekeymethodfortaking dataandproducingresults.
spius
28
GettingtheMostFromMetrics g ChannelMapping
spius
ChannelMapping
Useaprinted(ordigital)copyoftheMarketingExperiments ChannelMappingspreadsheetaswewalkthroughtherestofthis session
spius
30
ChannelMapping
Step1:Determinetherightobjective
spius
31
Step1:Determinetherightobjective
Design
Traffic
Branding
RESULTS
Usability SocialMedia
Video
spius
32
Step1:Determinetherightobjective
Top Directories - Companyname (within /portal/ directory)
1 2 3 4 5 6 7 8 9 10 11 12 13 14
homepage /products/ /resource/ /industries/ [otherrootportalpages] /services/ /customers/ [otherdirectories] /press/ /partners/ /developers/ /investors/ /landing/ /security/
PVs 1,786,086 1,596,918 446,431 401,004 332,328 249,275 249 275 211,505 198,386 177,454 157,209 152,606 59,293 45,119 6,092
UPVs 1,520,578 1,131,961 370,528 293,057 235,785 185,865 185 865 170,434 162,696 147,026 113,354 116,092 46,702 37,613 4,056
%ofAll 33.5% 25.0% 8.2% 6.5% 5.2% 4.1% 4 1% 3.8% 3.6% 3.2% 2.5% 2.6% 1.0% 0.8% 0.1% 100.0%
Inourcurrenteconomic situation,resultsmustbe quantifiableandofreal valuewhendetermining ROI.(ormarketing s ROI (or marketings contributiontopipeline, revenue,etc.) Nothingjustifiesyour marketingexpendituresor budgetrequestsmore effectivelythanrealresults.
spius
33
ChannelMapping
Step1:Determinetherightobjective Step2:Simplifyyourmetrics
spius
34
Understandingmetrics
Iwouldntgiveafigforsimplicityonthisside ofcomplexity,butIwouldgivemylifefor of complexity but I would give my life for simplicityontheothersideofcomplexity Oli OliverWendellHolmes,Jr. W d ll H l J
spius
35
Step2:Simplifyyourmetrics
Yourcompany'stimeandenergyisatapremium.Youneedtogetthe "most"forthe"least."Thisrigorousfocusnotonlyconcentratesyour resources;italsoclarifiesyourobjectives. resources; it also clarifies your objectives Whenyoucutthroughalltheconfusion,therearereallyonlyfour elementsyoucanmeasurefortheactivityon yourwebsite:
THEAMOUNT(pageviews,visitorsessions,returningvisitors,etc.) THESOURCE(referrers,searchterms,languages,countries,organizations, ( , , g g , , g , etc.) THENATURE(entrypages,exitpages,browsers,platforms,screen resolutions,pagerefreshes,pageloaderrors,averagetimeperpage,etc.) resolutions page refreshes page load errors average time per page etc ) THERESULTS(clicktrails,mostrequestedpages,numberofpageviews, signups,orders,etc.)
spius
36
Step2:Simplifyyourmetrics
Whatmetricsyouchoosetofocusonwilldependgreatlyonyour objectiveinthefirststep.However,theprovidedChannel Mappingspreadsheetlaysoutthebasicminimummetricsyouwill Mapping spreadsheet lays out the basic minimum metrics you will wanttotrackforanyonlinecampaign.
Youcanusealmostanymetricsprogramandshoppingcart programtoextractyourdataforthefollowinginformation. t t t d t f th f ll i i f ti
37
spius
Step2:Simplifyyourmetrics
Onceyouhave determinedwhat youaretracking,it you are tracking it isnowtimetolist themetricsforyour channels.
spius
Step2:Simplifyyourmetrics
Youmayhaveto trackthefollowing information internallywithyour ownfinancial department.
spius
spius
ChannelMapping
spius
41
Step3:Profileyourkeychannels
Profilingchannelsrequiresthatyouprofilethe Profiling channels requires that you profile the following: 1. The MAGNITUDE ofDemand 2. TheNATURE ofDemand 2 The NATURE of Demand
spius
Step3:Profileyourkeychannels
The MAGNITUDE ofDemand(RankingChannels)
Whenlookingatmagnitude,wefirstrankchannelsbydescending performance.(Formostbusinesses,thiswillmeanrankingthemby performance. (For most businesses, this will mean ranking them by descendingprofitordescendingrevenue.) Rankingyourchannelsinorderofdescendingperformancehelpsto identifypatternsandtoprioritizeyourchannelprofilingefforts. identify patterns and to prioritize your channel profiling efforts
spius
Step3:Profileyourkeychannels
The MAGNITUDE ofDemand
Next,wefocusonanalyzingatourtopperformingchannels(inthis case,wewilllookatourtop4channels) , p )
spius
Step3:Profileyourkeychannels
Profilingchannelsrequiresthatyouprofilethe Profiling channels requires that you profile the following: 1. The MAGNITUDE ofDemand 2. TheNATURE ofDemand 2 The NATURE of Demand
spius
Step3:Profileyourkeychannels
The Th NATURE ofDemand fD d
Whenlookingatnature,firstwelookforanyimmediatelynoticeable corecommonalitiesamongthetop performingchannels. core commonalities among the topperforming channels.
Let snowexplorethenature ofthecommonalitiesinchannelprofiles. Lets now explore the nature of the commonalities in channel profiles
spius
Step3:Profileyourkeychannels
The Th NATURE ofDemand fD d
Next,throughconductingdeeperresearchandanalysisofeachchannel (keyphrase data, segment demographics, surveys, etc.), we begin to data,segmentdemographics,surveys,etc.),webeginto identifyadditionalsimilarmotivationsamongthetopperforming channels.
spius
Step3:Profileyourkeychannels
The Th NATURE ofDemand fD d
Finally,weusethesekeyinsightstodevelopheadlinesandkeyphrases thataddressthesecoremotivations.
NatureofDemandforbusinesssoftwarekeyword
spius
Step3:Profileyourkeychannels
The Th NATURE ofDemand fD d
Also,notethatpropensitytobuy(P2B)modelscanbeahelpfulwayto calculatethemotivationlevelsofyourchannels. AP2Bisastatisticalmodelthatcalculatestheprobabilityofacertain companytobuyacertainproductinagiventimeframeinthefuture. YoucanfindmoreaboutP2Binyourresources. You can find more about P2B in your resources. Statistical Model
ProbabilitytoBuy Technologyinthe Technology in the next2quarters
Firmographics PastPurchaseBehavior
pastpurchasebehavior
Q4 Q1 Q2 Q3 Q4
firmographics
Q1
Q2
(mostrecent closedquarter)
spius 2010 Cisco Systems, Inc. All rights reserved.
ChannelMapping
spius
50
Step4:Optimizeaselectcampaign
Atthispoint,youintegratewhatyourchannelspecific profilesintochannelspecificmarketingmessages.
Whytheypurchase: CEOs Likecompanywide moduleinteraction, impressedwithdashboard i d i h d hb d capabilities,seepotential tosignificantlyreduce technologyandoperations costs,likethefactthatso manyotherlarge companiesareusingit,like theexpandabilityofthe system. CFOlikeabilityofmodules tointeractdirectlywith accounting,like automationcapabilities, impressedwithreporting features
spius
51
Step4:Optimizeaselectcampaign
Tomorrow,wewillspenda2hourslookingmorein ll d h l k depthathowtooptimizingonlinemarketingcampaigns. Thisisacrucialpartoftheprocessandwarrantsmore teaching.Fornow,justmakenoteoftheprocessandbe preparedtolookfurtheratthisinournextsession prepared to look further at this in our next session together.
spius
52
ChannelMapping Step1:Determinetherightobjective Step2:Simplifyyourmetrics Step3:Profileyourkeychannels Step4:Optimizeaselectcampaign l Step5:Refinewithtesting Step 5: Refine with testing
spius
53
Step5:Refinewithtesting
EXPERIMENTID: NewYorkTimes LOCATION MarketingExperiments Research Library MarketingExperimentsResearchLibrary RESEARCHNOTES: BACKGROUND: TheNewYorkTimescametouswitha subscriptionproductthattheywerehavingdifficulty marketingcalledtheElectronicEdition. g GOAL:Toincreasethenumberofsubscriptionsforthe ElectronicEdition. RESEARCHQUESTION:Whichpagedesignwillgeneratethe mostsubscriptions? TESTDESIGN:A/Bmultifactorialsplittest
spius 2010 Cisco Systems, Inc. All rights reserved.
54
Step5:Refinewithtesting
Original Page Test Campaign #1
Step5:Refinewithtesting
Original
Test Campaign #1
spius
Step5:Refinewithtesting
64%IncreaseinConversions
Theoptimizedversionincreasedtotalconversionsby64%.
Original Page Test Campaign #1
spius
Step5:Refinewithtesting
WHATYOUNEEDTOUNDERSTAND:Wedidnotstop WHAT YOU NEED TO UNDERSTAND: We did not stop there,afterreceivinganinitialresultsof64%,our teamstudiedthedataagainandcreatedournext roundoftestsforthiscampaign.
spius
Step5:Refinewithtesting
(New) Original Test Campaign #2
Step5:Refinewithtesting
(New) Original
Test Campaign #2
spius
Step5:Refinewithtesting
541%IncreaseinConversions
Theoptimizedversionincreasedtotalconversionsby541.6%.
(New) Original Page Optimized Page #2
spius
Step5:Refinewithtesting
WHATYOUNEEDTOUNDERSTAND:Againourdid WHAT YOU NEED TO UNDERSTAND: Again our did notstopthere,afterreceivingadditionalresultsof 541%,ourteamagainstudiedthedataandput togetherplansforournextroundsoftests.
spius
Step5:Refinewithtesting
Test Campaign #3
spius
63
Step5:Refinewithtesting
47%IncreaseinConversions
Theoptimizedversionincreasedtotalconversionsby47%.
(New) Original Test Campaign #3
spius
Step5:Refinewithtesting
(New) Original
Test Campaign #4
spius
Step5:Refinewithtesting
12%IncreaseinConversions
Bychangingafewwordsweincreasedtotalconversionby12% By changing a few words we increased total conversion by 12%
(New) Original Test Campaign #4
spius
Step5:Refinewithtesting
Test Campaign #5
spius
Step5:Refinewithtesting
88%IncreaseinConversions
Thisis88%higherconversionontopofallthepreviousgains This is 88% higher conversion on top of all the previous gains
(New) Original Optimized Page #5
spius
Step5:Refinewithtesting
CumulativeOptimizationGain: 1052%
64%
spius 2010 Cisco Systems, Inc. All rights reserved.
541%
47%
12%
88%
Step5:Refinewithtesting
Itisnotenoughtojustsetupcampaignsbasedonmetrics.That wouldbeanadequatewaytocreateeffectivecampaigns.But adequacy is the enemy of excellenceanditiskeymakesurethat adequacyistheenemyofexcellence and it is key make sure that wearetestingourcampaigns,measuringtheresults,andthen startingthecyclealloveragain.
Testand refine Determine Objective
Results
Optimize Campaign Simplify Metrics
Profile Channels
spius 2010 Cisco Systems, Inc. All rights reserved.
70
GettingtheMostFromMetrics g Summary
spius
Summary
Sohowdosimplygofrommoundsofdatatoeffectivecampaigns?
Welookedspecificallyataprocesscalled ChannelMapping ,whichisakey We looked specifically at a process called Channel Mapping, which is a key methodfortakingdataandproducingresults.
Step1:Determinetherightobjective Step2:Simplifyyourmetrics Step3:Profileyourkeychannels Step 3: Profile your key channels Step4:Optimizeaselectcampaign Step5:Refinewithtesting **Nowthatwehavethisfoundationandstructure,tomorrowwewill drilldownonactuallyoptimizingthecampaign**
spius 2010 Cisco Systems, Inc. All rights reserved.
72
GettingtheMostFromMetrics g Questions
spius
GettingtheMostFromMetrics g End
spius